Your thinking is good, and there are two possible scenarios to consider here.
A) If your ‘CDN’ is something like Cloudflare that takes control at the DNS level, then your TTFB could improve due to 1) better/distributed DNS and 2) better routing to your server
B) If you’re just loading images/css/js/etc on your CDN then your TTFB could be improved because there’s less load on your server - it’s just focusing on generating content, not serving static assets.
Usually though you’re right - using a CDN has a very small (if any) effect on TTFB.
You are correct in that many people use CDNs to load static content faster. However CDNs can also be used to accelerate dynamic content as well. (Disclaimer: I work at Akamai, and while all of these examples are possible w/ Akamai your mileage may vary with other CDNs).
Here are some ways that a CDN can impact TTFB for HTML in addition to what MachMetrics mentioned above -
By routing the user to a nearby CDN server, TCP and TLS connect times are reduced because the round trips are shorter than they would be w/o a CDN.
Redirects often count against the TTFB metric. Many times redirect rules can be configured at the CDN, reducing the round trip time for the redirect. This means the user will be able to request the actual HTML page sooner.
If a HTML page is cacheable, or conditionally cacheable, that can reduce the TTFB time considerably since the processing overhead and transfer time from the origin is eliminated when served from a CDN cache.
Even if the HTML is not cacheable, CDN optimizations across the middle mile can reduce the time it takes to transfer the HTML from your server to the client.
Many times CDNs provide very minor speed ups (the global backbones are very fast now) with massive amounts of caching problem headaches.
When I take on new private hosting clients, my first step is always to strip out any cruft between an Apache server + a visitors browser.
All cruft - load balancers, DOS mitigators, CDNs, proxies (NGINX, Squid, Varnish), session caches (memcached).
Then just tune the LAMP Stack for speed.
I have some servers (hosting WordPress sites) which have been running 250K requests/hour, sustained for years. Average CPU usage is 1/2 of on CPU load unit + very little Disk I/O.
This is all accomplished through LAMP Stack tuning.
Best to focus on tuning your Stack first. Likely this will sufficient.