In LoC, ch.1, when Bashere throws a dagger at him and Rand manages to catch it in time: "Had I been an eyeblink slower, I'd be dead."

So, if Asmo simply wasn't as fast as Rand in this moment, or if the killer's weapon was faster than Bashere's thrown dagger, also a non-channeler could very well kill Asmo (as RJ also made this clear by stating that Fain would have killed Asmo, if he had been there).