I will say that I am one to look too hard at the odds that I might get injured while doing something. I have the tendency to start getting really worked up about things. Then I realize that it's all in my head. I bite the bullet, and do it. Experience is the best teacher. My favorite time to freak out is just before driving in the snow. On the highway. With tractor trailers and poor visibility. I have learned to drive in terrible weather fairly well. I have learned that precautions can be taken to reduce your chances of an accident. I have been stuck in the snow, miles from civilization with no cell service. I now have a winter emergency kit in my trunk. Hitting a deer is also a real concern where I live. Shit, I have to get to work and otherwise live my life. I can't not drive.

I really suggest you take a little vacation in the country. Get a farmer to teach you how to drive. Drive tractors, ATV's, lawn mowers, pickup trucks. The car will seem like a piece of cake. Then visit in the winter and do doughnuts in the mall parking lot before the place opens. It's how all my friends' parents taught them to learn how to steer out of a skid. At the time I was glad my dad never did that to me, but now I wish he'd had.

If the country driving lesson isn't likely, take a defensive driving class. You'll learn a few tools to help you. Good luck!

Edited to add: I've driven in a few places, and hated driving in NYC the most. It sucks.