Been there, done that Lars. Many of us have.
Issues such as TTFB can be uber frustrating, be dependent on (or influenced by) a myriad of mundane modules, wares, & factors, suddenly change on a dime, require 1000s hours of pen-testing, be susceptible to Murphy’s Law, and may even need a sacrifice or two in the name of GREAT FEM’PUTER!!
I jest, ‘yes’, however it is one of those issues which has racked my brain to the point that my once hippie hair has been reduced to baldness, lol.
But you may consider yourself a tad lucky in the sense of I personally miss the days where I was dealing with a higher TTFB with the ability to blame the hosting provider, over the present where I am the server administrator to be blamed if there is a crappy TTFB!
While there are a few things which your Hosting provider MAY be able to do on their end to help your TTFB, don’t count on it… Your best bet is to deal with the factors which are in the realm of your control, and ‘yes’ there are things within that slice of the Code-Chain. In fact, from my own personal experiences, much larger ranges in the TTFB can be controlled at the USER side over the ranges meldable on the SUDO side. (EX: Reducing TTFB from ~900ms to ~300ms at your end compared to a reduction from ~900ms to ~800ms via the host’s end --although those small changes matter a big deal for those aimed at consistently falling inside Google’s guide of a <200ms TTFB, such as myself.)
My first & best advice for you, based on my experience and on the innumerable rounds of trial/error I’ve played over the years, would be to get a Wordpress-based Cache Plugin (or code one yourself if you’re up to a challenge). I myself presently use “WP-Super Cache” (+ a personally created Plugin to modify/optimize some of its actions for the concerned website’s individual needs) @ [url=https://www.webpagetest.org/result/161004_24_305/]https://www.webpagetest.org/result/161004_24_305/[/url] which, on average, had a TTFB of ~1500ms w/o it and now runs ~400ms on average (but please not that you may notice that the concerned website’s non-cached pages run at ~190ms, which is VERY deceiving, as it uses PHP-based Caching instead of htaccess-based and uses Dynamic-Caching with a tweaked “Late Initialization”; i.e. I am purposely slowing down the caching process for specific reasons which most likely will not be needed in your case, and thus your TTFB would improve all 'round rather than spotty like this).
A Cache Plugin/Program will almost positively be of great aid in reducing your TTFB, as it is during those Pre-1stByte that the HTML page to-be-sent is being constructed in the underbelly via 1,000’s of PHP processes. With Caching, your site will just spit out a pre-made page rather than making the same thing over & over & over. Thus, you’ll save time, in addition to CPU.
While I am presently using WP-Super Cache in my co-administration of the aforementioned site’s systems, I would in-fact recommend “W3 Total Cache” for someone such as yourself, and for a site such as yours; as with the latter here, you’ll have better control over things w/o over-complication, be able to deal with many other issues such as Minification, Cache Control, etc. simultaneously. It was what I had been using on all sites I was overseeing before moving into Server-Side systems. WP-Super Cache is more for those needing some fine-tuning.
I’ve also had experience with the “Falcon Cache” used with the “Wordfence Security” Plugin, and it is good for bare-bones Caching if you just want to literally plug-in-&-get-going. Otherwise, it is just a bit TOO simplistic.
Well… there you have it… my two cents out of my no-sense head for you…
Good luck to ya Lars!
Nate