Akamai Versus NetDNA/MaxCDN

Web Performance Discuss Test Results
1

Here is some preliminary testing between MaxCDN and Akamai:

54 KB file, San Jose CA, IE8, 1.5 Megabit DSL

Akamai
Load Time: 0.516 seconds
First Byte: 0.180 seconds
http://www.webpagetest.org/result/111012_S0_1W1RF/

MaxCDN
Load Time: 0.498 seconds
First Byte: 0.186 seconds
http://www.webpagetest.org/result/111012_8B_1W1NW/

4 KB file, San Jose CA, IE8, 1.5 Megabit DSL

Akamai
Load Time: 0.214 seconds
First Byte: 0.176 seconds
http://www.webpagetest.org/result/111012_M6_1W1SF/

MaxCDN
Load Time: 0.232 seconds
First Byte: 0.188 seconds
http://www.webpagetest.org/result/111012_5M_1W1XW/

I’ve tested at different location and the results are pretty similar. The interesting thing is that Akamai seems to almost always do slightly worse than MaxCDN when it comes to total load time, despite Akamai having the better time to first byte. Though the 4KB file shows Akamai having a better load time.
I’ll keep everyone updated, I have to add some more cnames and I’ll be doing web page speed tests.

2

Great. Could be a difference in TCP stacks between the two. A little over 4KB is the default initial CWND which could explain Akamai starting out faster but falling behind as they introduce more round trips.

MaxCDN is using fastsoft’s TCP “acceleration” which has better congestion window control and can likely deliver a lot more of the content with fewer round trips.

3

Yes, that would make sense. I’m going to contact Akamai and see if there is something they can do to improve performance for larger files, perhaps just have certain rules apply to a single host name of theirs and use the current host name i have for everything else. The CDN will mostly be used for product thumbnails and js/css files, so almost all of those should be within the smaller file range.
I am investigating Akamai due to issues with MaxCDN. One i posted here before, was that a certain data center was performing badly. However, i also got annoyed with the fact that their New York data center was being hit by a DDOS attack… this caused our customers in the range of that data center to have timing out css and such. I personally feel a CDN should be more immune to a DDOS attack.
However, MaxCDN has one main thing going for it, price. I had Akamai put in an escape clause that gives me time to test to see if Akamai’s larger price is truly worth it. Though, it’s not just performance to take into consideration, but availability. We lose quite a bit of money when the CDN performance is sketchy, which reflects badly on us.
I’ll keep everyone here updated as i continue to test.

4

Ok, after a string of DNS issues (on our end), I was finally able to get to the whole site testing. Interestingly, while the TCP stack differences were noticeable in individual testing, I found them to be inconsequential in the whole site testing. This is mostly because the TCP connection is reused, not closed each time, so I found that difference had little effect overall. Below is the speed differences from a few tests:

Kansas City, MO - IE8 - DSL

MaxCDN
Load Time: 2.852 seconds
First Byte: 0.334 seconds
http://www.webpagetest.org/result/111017_AE_1XD76/

Akamai
Load Time: 2.280 seconds
First Byte: 0.334 seconds
http://www.webpagetest.org/result/111017_8X_1XDB8/

Load Time Difference: 0.572 seconds in favor of Akamai

San Jose, CA - IE8 - DSL

MaxCDN
Load Time: 2.681 seconds
First Byte: 0.354 seconds
http://www.webpagetest.org/result/111017_JA_1XDMN/

Akamai
Load Time: 2.069 seconds
First Byte: 0.351 seconds
http://www.webpagetest.org/result/111017_6E_1XDKZ/

Load Time Difference: 0.612 seconds in favor of Akamai

Montreal, Canada - IE8 - DSL

MaxCDN
Load Time: 2.945 seconds
First Byte: 0.293 seconds
http://www.webpagetest.org/result/111017_PE_1XESB/

Akamai
Load Time: 2.903 seconds
First Byte: 0.329 seconds
http://www.webpagetest.org/result/111017_JK_1XESZ/

Load Time Difference: 0.042 seconds in favor of Akamai. The difference here is small, but the start render time is 0.204 seconds faster on the Akamai test, though it is hard to tell if this improvement is because of Akamai or not.

I’m still testing, i’ll keep everyone updated.

Edit: Ignore the cache static content rating, for testing I have the expiration date set to access+24 hours. Also, there is a small difference between transferred bytes, I think this is Akamai serving the css and js files with slightly better compression.

5

Hi Jarrod,

Thanks for sharing your test results. I’m following this thread with great interest.

By the way, why not do 10 test runs instead of just 5? Ten is the maximum, and it only takes a couple of extra minutes.

Have you found an “inexpensive” Akamai reseller? The prices I’ve seen have always been very high.

After testing four or five CDN providers myself (I haven’t tested Akamai because of the price), I’ve settled on using Edgecast myself. Edgecast does have a few inexpensive resellers. If you ever do comparison tests Akamai vs Edgecast (or vs anyone else), I’d be interested in your results.

6

Yeah, I could do 10 tests, I just found 5 to be enough, often giving the same results as 10. As for an Akamai reseller, the best I’ve been able to achieve is $1,000 monthly through Akamai themselves. I tried to negotiate on the contract, and was able to get a 90 trial period added onto contract, but that was the extent of it. They wouldn’t budge on anything else. From what I’ve heard they have a $25,000 minimum contract requirement (in our case, $1,000 monthly with a 2 year contract). If your business can take the $1,000 monthly hit, thus far, it looks like Akamai is the best choice. Keep in mind that a CDN for a business is also an investment. Conversion rates usually increase with speed increases, and depending on your monthly revenue, that increase may automatically pay for Akamai.
Currently I have no plans to test Akamai against Edgecast. I’m only doing Akamai versus MaxCDN because we’re already using MaxCDN, so the switch back and forth was pretty easy. But, you never know.