"Rand misjudges Taim’s age because when they meet, you might say Taim has been rode hard and put away wet. He has just finished a long and difficult flight to reach Caemlyn, the one place where he might find refuge instead of being hunted — along with other reasons — and that has a wearing effect on anyone. Now that he has recovered, he doesn’t look so old."

That together with the shaved beard explains why Bashere had a harder time to recognize Taim. Probably Taim actually shaved his beard so that his hunters would have a harder time to identify him as well, and lied about the heat.

As said, Aran'gar's comment in PoD that she can hardly remember being a man makes no real sense if she is usually posing as Taim. It makes more sense that Osan'gar would have been ordered to play Taim at the BT (and not Dashiva), or that Demandred poses as Taim, but both ideas are ruled out.

Finally, RJ said that he was surprised that people came up with the Taimandred-theory, but he shouldn't if it's actually just another male Forsaken who poses as Taim. That and all his other comments about Taim indicate the Taim is a separate character, not someone in disguise.