It's every bit as illegal (I'm not sure how much that might be) to download "abandonware" as it is to download other stuff. You'll almost certainly never be prosecuted for anything old enough not to be making money for a company, but it's still technically illegal. ROM sites out there make up all sorts of justifications ("we only let you download games we actually own!" "you can only download these ROMs for 24 hours if you don't own the game yourself" ), but they're just trying to make people comfortable downloading from them. Personally, my guess is that downloading a ROM would hold up as legal in court if you could show that you own the physical copy, since I believe you're allowed to create backups for personal use, but I don't really know.

It's pretty clear that downloading a game you don't own, though, is illegal. It's just rarely enforced for anything old.