A lot of people start hating him after book two or three. I think I enjoyed it through book 5, though. After that came Faith of the Fallen, which I thought was awful. Even so, I read the next one, figuring he had a one book fluke. Pillars of Creation, however, (the next one) was absolutely abysmal. I think I only finished it because I was sick at the time, or something. I quit the series for quite a while. Finally read Naked Empire a couple years down the line, and it wasn't much better. Gave up for another long time, till I made a bargain with a friend. I told him I'd read the next book if he read something decent that I picked out. He agreed, and so I read Chainfire, which was surprisingly readable. Not great, but good enough that I read the next two books, which also improved slightly.

Bottom line: I honestly don't think you'd miss much if you just skipped the three middle books, and went straight from Soul of the Fire to Chainfire.