Sounds like you are on a dedicated server and have the necessary skill set to understand the data it will give you. It should be able to point you to the specific problems that are causing the long byte times.
It looks like static files are being served fast enough so it’s unlikely that you have an Apache configuration problem.
It looks like all of your php requests are pretty painful though (even the ones for content other than the base page - 2 seconds each) so there may be a configuration problem between Apache and PHP. Are you running PHP as mod_php, cgi or fcgi? You might also need to do some tuning on eaccelerator if you are blowing out it’s cache.
If you have any modules that are making external calls that could also cause the huge first byte problem. New Relic should be able to point them out really quickly if there are any.
Patrick, I hadn’t really looked at New Relic before either. I installed it a couple hours ago and it’s pretty impressive. It does a lot of things but I like how you can see the most time consuming database activities. It’s like a profiler for your live server.