Norman Walsh recently posted a link to Akiyoshi's illusion pages. I've got to say these are really fantastic. They are excellent illustrations of the physiological factors that make our vision work the way it does.
I recently read Margaret Livingstone's Vision and Art: The Biology of Seeing, and although it's not the sort of book I would have bought for myself (my mother gave it to me as a birthday present), it's excellent. It made me a lot more aware of how human vision works and evolved — things I always had a general idea of, but not much in the way of specifics. This book really made a lot of those ideas a lot more concrete for me, and illustrated them with artwork and optical illusions that demonstrate them.
A number of the book's illustrations are remarkably similar to many of Akiyoshi's illusions.
On a related note, via Dave Shea, for anyone interested in colour and luminance contrasts, and colour blindness (topics covered in Vision and Art), I highly recommend Pixy's Colour Scheme Picker. It's a client-side web application that lets you explore colour combinations, and how they might appear to people with different forms of colour-blindness — well worth bookmarking.