See my earlier points about how 'baby' and 'child' are grammatically neuter words (at least in English), so depending on how you phrase sentences, it may be normal to use 'it' at times. As aero then added, there are also some very specific grammatical constructions where 'it' is used for any and all persons, but that doesn't seem very relevant - this conversation is obviously focused on the kind of sentences where different pronouns can be used depending on the person described.

But none of that would make it any less weird if you started referring to children by 'it' in cases where grammar doesn't require it and you'd expect 'he' or 'she'.

So basically, no, the age of the individual is not a factor, but the use of the specific words 'baby' and 'child' is a factor. And obviously those wouldn't be used for adults. Hence that qualifier.