I remember reading about an area, I can't remember if it was somewhere in China or another country in the region, where there weren't enough women to go around. Therefore, brothers would often share a wife between them. Any children born of the woman would call the oldest brother "father" and the younger one "uncle."

It can work, but yes, there's probably a reason why it's rarer than polygyny, and I would assume that inability (until recently) to determine who was the biological father of any children born to the wife would be the major reason.