The problem you’re seeing relates to your LAMP Stack + likely your entire runtime environment.
Asset #1 (HTML component) serve time on your site relates to both your hardware (VPS) + software (LAMP) + WordPress code (core + theme + plugins).
http://www.webpagetest.org/result/171021_1C_83660c92d6f059d0200433427c64a27e/ shows an example of a site I host. Notice the total time required to server Asset #1, after DNS, is 153ms.
I target <300ms for serving Asset #1 for my clients.
Here’s how I do this.
I run Ubuntu on bare metal servers (no VPS + no VMs).
Then LXD at machine level + sites in LXD containers.
This way I can tune each site’s LAMP Stack for each site’s specific work load.
The site in the example above is a WordPress membership site under heavy usage.
Since you’re running in a VPS slice, this likely means your VPS is running inside some sort of slow VM.
This type of setup is highly effected by other sites running in other VMs on the same hardware.
All this said, your likely first step is to change hosting, to a company which understands how to run high speed WordPress site hosting.
That’s your first step.
Once your site’s running in well tuned hosting, then you’d have someone go through your site + fix anything slowing your site down. This list can be long + complex.
If you’re a DIYer, a good starting point for you is to enable SAVEQUERIES + WP_DEBUG + check your WordPress debug log file for any problems + install the Query Monitor plugin + debug/fix all reported issues.
I’d suggest you only go through the Query Monitor debug/fix cycle, after you’ve changed to fast hosting, because you may find all problems vanish when your site runs on a well tuned LAMP Stack.