You're correct in saying that the Aes Sedai with their aloofness have definitely contributed to the lower number of novices and accepted in the WT.
On the other hand, the actual number of channelers in this Age has been previously confirmed by RJ that it is lower than that in the Age of Legends. I think it was something like 1% of the population now vs. 3% then.

Another thing that may have contributed to this is definitely the purging of all male channelers. Genetically this would have affected the number a bit. But most important is the strength of the actual channelers. I believe that by not working together with men, Aes Sedai slowly started to diminish in their power, because they no longer could do most of the more wonderful channeling tasks that require both sexes. Couple this with the fact that they are a bunch that don't really like to share channeling knowledge, many things were forgotten and never used as result. Just like if you tie one of your fingers tight with a string all over, that finger would shrink, the same thing sort of happened to the Aes Sedai strength but over a longer period of time with many of those strongest in the power no longer having children and all that.