That Slayer is the only one able to kill Asmo, but considering the clues he simply makes the most sense. Even Sammael made more sense than Graendal before LoC, because he actually threatens to kill Asmo.

Actually Slayer already appears in Egwene's/Nynaeve's plotline in TDR, when he kills the Gray Man. In the beginning of TSR Slayer is basically in Rand's plotline, when he kills the two BA. Then he hops to Perrin's plotline, when he is hunting Fain, but also appears in Nynaeve's plotline again, when he almost kills her in TSR. In LoC Egwene sees Slayer stalking in TAR. Similarly Slayer is seen spying on the Wondergirls in WH, but also appears in Rand's plotline in Far Madding. Besides that, Slayer (or Luc & Isam) are of course often mentioned by various characters throughout the series.

That shows that Slayer basically jumps between the plotlines, so to speak. And this makes sense, because he is an assassin, who follows the various tasks different patrons give him. Therefore it also makes sense that he killed Asmo, especially if he was sent by a Forsaken linked to Rand's plotline at this time (most likely Graendal or Lanfear; but one also can't completely rule out Mesaana, Sammael or Moridin).

Your argument that Slayer is only tied to Perrin's plotline and therefore couldn't appear in Rand's plotline in TFoH makes obviously no sense and is even refuted in the books. Considering that we had two other mysterious assassinations in the books before Asmo's death - namely the killing of the two BA in the Stone and the killing of the Gray Man in the WT - it makes sense that the same killer stroke again in the Caemlyn Palace to kill Asmo.