That he comes over while his son is gone doesn't sound so much like co-dependency (if that were the case, I suspect dad would come over while son was home). Or maybe the two are tied together in some way, since it sounds like he doesn't clean your areas of the house/apartment.

Moving forward, could you suggest that his father wean himself off the habit? Instead of coming over every/other day, try to keep it to once a week, then every other week, once a month, etc. I think it would be better to work with dad, get him to admit he has a problem, and work with it pro-actively. Basically, wean dad off his habit. Suggesting professional help might be a good idea as well, since when confronted with it he seems to at least recognize that there's something not right with how things are now even if he can't control it.

I'd say that asking for the key back would only be a last-case scenario, to be done after the above.