I think it had less to do with refugees than the Fourth Crusade sacking Constantinople before they left (giving the lie to the suggestion it was still a religious rather than political war, if that had ever been true. ) Maybe the walls fall to the worlds first cannon without it (probably, in fact) but the Fourth Crusade left the city vulnerable and exhausted, and the European armies carried back untold treasures on Venetian ships that seem to have been there largely for that purpose from the start. A wealth (in every sense) of art and learning was carried back to Medieval Europe from the dying city. I tend to view the Fourth Crusade like Aeneas carrying Anchises from Troy to Italy--if he'd burned the place himself first.