Do you have a lot of WordPress plugins installed? If you have any inactive plugins or themes, you should look at removing those (can slow things down).
Your server appears to be doing fine. You’ve got a first byte time of 0.351 seconds. That’s not bad at all, and would surely be even faster if you tested with the FIOS connection.
Your problem has more to do with the shear number and size of files on your page. Combining JS files will help and so will CSS sprites.
You can even save some time by creating CSS sprites out of your social media buttons and using direct links as described here…