Personally I think it's worse to kill your spouse, though I know some posters believe the opposite. It could also be RJ expressing his personal emotions at the thought of killing his own wife. He had no children, so the emotions would be imagined, and less palpable than his feelings for Harriet.

It's also possible that his kids were adults, especially if LTT was hundreds of years old. I do recall text in the prologue of EoTW mentioning dolls and small broken bodies, but I'll have to check.