It isn't supposed to do so in any way involving government action, true, but even the most liberal of societies police speech. Meaning, you can say whatever you like, but there are and should be societal consequences. Make racist anti-Jewish rants? I'm not going to invite you to my parties. Nor will I patronize your business. As a potential employer, I might not hire you. To say that "freedom of speech" is somehow equal to "freedom from consequences for my speech" is ludicrous. A sufficiently liberal society would be reluctant to use such tools; confining it to only those who represent the most extreme and/or dangerous views. And shouldn't be used to silence those with "controversial" views, as you should have a right to "speak". But again, as with many things in life, you should have the freedom to do it, but are not free from the consequences.