CSS non-reload rollover demo (not sure if this version works in IE?)

It works because the rollover image is already loaded as a background image (view source).

On rollover (CSS hover), the link background is set to none, revealing the image beneath.

Here's another example using a typical 3rd party link: