We recently upgraded to vbulletin 4 and have been fighting a hard battle to get speed back where it used to be.
We’ve done a lot of server side optimisation which has made a big difference, but we still have a long way to go. We now feel that front end optimisations will be where we can yield the big impact for users.
I’d be really really grateful of your expert views on these results and where we might best focus our efforts.
It looks like you may need to do a little work on back-end scaling as well. How is it hosted if you don’t mind me asking (cloud, VPS, shared, dedicated boxes)? Particularly the database. Throwing SSD’s at the database can have a huge impact for something like a discussion forum is it’s an option available to you.
Thanks. Will draw up a list of all those obvious gotchas and start prioritising I guess.
Hoping that someone might spot “the big one” in those results
Is the CPU maxing out an issue? We have seen a particular user activity impact in IE and on lower screen resolutions and we’re wondering if that is suggesting lower end machines are struggling more than faster ones?..
Yeah. The slog lies ahead I guess. Any clues in what might be the biggest wins in this front end work would be very gratefully received. Keen to get the biggest gains first
FYI, there are “Dynatrace” configurations available in the Dulles location (IE 7 and IE 8) which capture a dynatrace run and let you download the session (also lets you share it with everyone else to work off of the same one).
I’m running a test right now but you generally want to look at the hot spots.
It took close to 4 seconds of CPU time to execute in the test that I ran.
If you open up the “Pure Paths” UI and sort by CPU time (descending) it will sort the code execution from most expensive to least.
3k+ DOM elements is a pretty huge DOM (which is why it is so expensive). Not sure if there’s anything you can do about that but it would be worth looking into to see if there are easy options with the layout that may be inflating it.
Ah great. That’s brilliant having it on the Dulles install.
Thanks so much for your help. It’s really shining a light on things
I’ve been having a good dig and am a bit confused about this…using your first test as an example…
When I sort by CPU Time within Pure Paths I am commonly seeing that last child problem at 3400+ ms and a readystatechange event on at 2500ms+ (which seem therefore to be a problem)
What is throwing me is that their start times are 7s and 11.5s roughly. So I looked back at the waterfall to see if that is where we are getting the gaps/spread out items & CPU maxing and the problem seems to begin much earlier i.e. around 3s
I look to see what JS is going on at around 3s and actually it says the start time for the html is 3.38s - which contradicts with the waterfall which shows it running from 0-1.5s
The first significantly slow js seems to start at 6.3s with a CPU time of 765ms. Again too late according to the waterfall
Should the timings from Dynatrace and the Waterfall synchronise?
How can I see which JS is causing problems at the point in time where we are getting the big gaps and CPU max in the waterfall?
Am I missing something and maybe it isn’t JS that is holding up the beginning of these waterfalls.
Probably worth adding that I am more interested in the cached views at the moment as the problem is impacting logged in users most.
Have you tried running NewRelic or Dynatrace on your backend? At your scale it’s probably something you want to leave running all the time (rather than just for a quick trial) but it will tell you where your hotspots are for the backend (specific database calls, etc).
It does sound like a well though-out back-end (a little surprised the times are not better).
I’ll take another look at the front-end (particularly repeat view) in a minute.
Never mind, just noticed the newrelic beacons on the page :-).
Looking at the repeat view of the test you ran, 1.5 seconds is extremely long for the base page - does your server-side instrumentation show that calls are taking that long? That base-page time is going to drive a lot of user experience/feedback in a forum situation where they will be clicking around a lot.
If you scroll the filmstrip, a red line will track in the waterfall with the left edge of the filmstrip window (time-wise) so you can see what is blocking the UI and when certain events happen.
The top part of the page loads pretty quickly after the HTML comes back
The actual posts are blocked by the ads (at least one of them) and show up at 3.1 seconds. If you can get your ads to load asynchronously then they won’t block the display of the content (adsense and doubleclick will both load async but I didn’t dig too deep to see what you’re doing yet). Looking at the code, it looks like it’s probably the ADVERTPRO ad blocks that use inline document.write’s to write script tags
The ad loading triggers the widgets UI to show which reflows the page. If you set up the main part of the page to have the correct initial width that could be avoided (not sure how easy it is to do with what you have).
Some other notes:
I don’t see evidence of it causing a problem in the repeat view waterfall but it looks like you have some IE conditional comments in the HTML after some external resources are loaded. This can cause blocking behavior in IE and it can be worked around by putting an empty conditional comment before any resources are loaded (like at the top of the head): http://www.phpied.com/conditional-comments-block-downloads/
Thanks Patrick. That’s really great. I’m going to be offline for a few days now for the Queen’s Jubilee antics but will properly digest and come back to you next week. Just wanted to say thanks and let you know I’ve seen your response for now. Much appreciated
The way I attacked vB performance was to first remove all unnecessary forum features (but this is not something everyone may be open to doing). Looking at your site I’d suggest to consider if it is necessary to have the right-column widgets (Discussions, Article Updates), the “What’s Going On” module on the bottom of the home page, the “Display Options” and the “Moderators” modules on the bottom of forums, is it necessary to show how many “Views” each thread has had, gender, reputation, country flag? And I would even remove the “Previous Thread” and “Next Thread” links from the threads bottom, as I think people don’t really use it.
On my forum I removed ALL vBulletin images. The orange reply/post/quote/tools/go buttons could be replaced with html. Every little bit helps.
My philosophy is that people come to forums for content, and all the pretty stuff like images and unneeded features are just a distraction.
Just my 2c
P.S. My vBulletin forum can be viewed at laptopgpsworld [dot] com. I recently changed my pagination settings so that forums contain 200(!) instead of the default 25 threads per page, and threads may go up to 100 posts before the page splits, but it still seems to have OK load speed. Members see a slightly different version than guests.
Found it… Need to tick the right setting before submitting the test…Advanced Settings>Video>Capture Video and then it’s obvious in the results (for anyone else who cant see its benefit).
I’m finding it the best view to start from this one personally.
Great. Thanks Marvin. We are looking to see how we can strip things down. Not sure we’ll get away with going quite as far as you have managed, but it’s certainly a good gauntlet to lay down to focus the mind! Thanks
Sorry, meant to reply sooner (did a day trip to California - I’m too old for that anymore). I’m considering making video capture on by default and including the filmstrip UI in the normal test result UI just because it is so useful. I just need to experiment with it a bit to make sure the storage requirements don’t explode.
Then scroll the filmstrip until the post content is the first frame displayed (4.1s), the Red vertical line will indicate the same time marker in the waterfall so you can see what it looks like blocked the content from displaying.
At the time it looked like it was blocked on the ads php calls but that was for a different run. It looks like there is some code in that area of the page but it’s hard to tell right now since it looks like the structure of the code changed from when the test was initially run.