I think that's very true, and often unfair to the past. Particularly since none of us really have that entire treasure trove of knowledge and often not much at all. There's a website, Archive.org, that keep a lot of old books around, one of my favorites to peruse The 1881 Household Cyclopedia is really something of a mindscrew because it contains so much modern information and science while it explains how to do everything from make ice and photographic plates to milk a cow, very clearly written for the common person. Even when we think of clever historical figures we tend to contrast them against dumb illiterate peasants, which I think is exaggerated. Pretty clearly by then reading wasn't an uncommon skill and didn't imply someone doing the reading might be of a social class that never needed to personally manure a field. I doubt 1881 was a landmark years either, but it gets tricky to read older English than that, for me anyway.

You're right about the uber-smart aliens thing often being used as a crutch to avoid acknowledging people way back were quite clever. The pyramids couldn't have been built by people is popular, but its like swimming after eating, or cold giving you colds, it's thoroughly debunked and has been for decades. People just keep saying they couldn't have been and cobbling alien intervention together to help solve an unnecessary dilemma. I think archaeologists place less effort into debunking these because it is mostly futile and helps with keeping public interest and funding up. I think for some those wilder theories are just to add a little mystery and fantasy to the world, something I think we can all sympathize with, but for others I think their world view just relies on a need to assume ancient humans were dumb and that modern tech, just to have a basic understanding and usage, requires brains. Sort of the flip side to Utopian Primitivism.