The Mountains to the west form a natural bulwark. And thought Altara's borders extend towards Andor, this has more to do with Andor being such a large territory - since most of Altara's power is focused along the coast, where their capital is, that territory near Andor's borders may be nothing more than undeveloped wilderness. Given Seanchan damane and their flying creatures, I don't really think a road leading north to the Two Rivers will be such an impressive defense for Ghealdan. Andor already touches the Wall - it would be a greater help to do as Alliandre did and appeal directly to the Andoran monarch, given that the Two Rivers is not that powerful at this time and can't offer more than a sanctuary for refugees at this time if the Seanchan meant to invade Ghealdan.