Slow PNGs and GIFs ?

Web Performance Discuss Test Results
1

Hi,
after migration to new machine, we did some optimalisation of the pages. The biggest issue seems to be with the image files, as it takes several seconds to download them (altho having only some 17Kb …). Here is current waterfall

http://www.webpagetest.org/result/111121_D9_28E8D/1/details/

Please the very long bars for GIFs and PNGs :frowning: For Example this

URL: http://img1.valka.cz/images10/homepage_icons/ico-01.png
Host: img1.valka.cz
IP: 89.185.250.36
Location: Czech Republic*
Error/Status Code: 200
Start Offset: 2.952 s
DNS Lookup: 225 ms
Initial Connection: 160 ms
Time to First Byte: 317 ms
Content Download: 1819 ms
Bytes In (downloaded): 17.8 KB
Bytes Out (uploaded): 0.4 KB

The webserver is Lighttpd, multiple 15k SAS disks in raid serving the files, no significant load (8 cores, load average: 0.21, 0.47, 0.41, iostat acting cool as well), 1Gbps network on backbone.

When I do local (same country) test using Google Chrome and Page Speed extension, I get speed for ico-01.png like 10ms instead of those 1800ms. Im connected on shared 10Mbps wifi line.

I wonder, what makes such a horrible difference ? Does it take like two seconds for 17Kb to travel from Europe to webpagetest server ?

(please note that putting amazon CDN instead to serve those files didnt change a thing, please see this result

http://www.webpagetest.org/result/111120_HF_27YR4/1/details/#request44

)

Or is there any problem with recieving PNG / GIF files ? I really cannot understand those results :frowning:

Thanks for any advice !

2

This happens due to TCP slow start.

Wiki:

Try to download images (at least a few) from the main domain, where html comes from.

Here is an example, on how to successfully exploit slow start:
http://www.webpagetest.org/result/111121_6P_28JBV/

3

thank you for your answer, I didnt have any idea about that. But changing to main domain didnt help to serve the files any quicker :frowning:

http://www.webpagetest.org/result/111121_11_28KS6/

i still think that downloading the 17Kb file to the other side of the world still shouldnt take two seconds :frowning:

4

Here is a test result from the Paris node, which is much closer to the origin Servers’:

And I agree, even with TCP Slow Start it shouldn’t take 1 second to download a 14.4 KByte image.
And if you look at objects 5-17, establishing a TCP connection shouldn’t take 1 second as well…

5

I’ve ran a test on the home page of your hosting provider. It seems their web site may be suffering from the same issue like yours. Not all, but a large number of their static files take unusually long time to download:

http://www.webpagetest.org/result/111123_SG_29132ccb0183f6a306df4d0ea05184bd/1/details/

Perhaps you may consider moving to a different host.

6

Thanks again for the tests … I guess I must have done something wrong - Lighttpd webserver is fast especially with static files, so sending images should be easy for it (and my current average on the whole webserver is like 20 requests per second, 300 kbyte/s so I guess nothing terrible). When I run ab command with 30-50-100 concurrent users getting 1000 images, it serves like 4000 static images per second (but I admit, within the same 1gbps network, not from outside internet, havent measured that), so I do not see any harddrive or memory or configuration bottleneck … Might it be keep-alive ON causing this ? I’d say no, but who knows, I’ve learned the hard way that in IT the things are really working the exact opposite you expect it to :slight_smile:

Can someone run the command

ab -c 20 -n 1000 http://forum.valka.cz/images10/homepage_icons/ico-01.png

againts my server from USA and let me know the result ? This is what I see when AB’ing the server from another quadcore machine within the same network :

ab -c 20 -n 1000 http://forum.valka.cz/images10/homepage_icons/ico-01.png

This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking forum.valka.cz (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:        lighttpd/1.4.19
Server Hostname:        forum.valka.cz
Server Port:            80

Document Path:          /images10/homepage_icons/ico-01.png
Document Length:        17933 bytes

Concurrency Level:      20
Time taken for tests:   0.391 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      18221000 bytes
HTML transferred:       17933000 bytes
Requests per second:    2559.37 [#/sec] (mean)
Time per request:       7.814 [ms] (mean)
Time per request:       0.391 [ms] (mean, across all concurrent requests)
Transfer rate:          45541.31 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       2
Processing:     1    1   4.6      1      72
Waiting:        0    0   2.0      0      46
Total:          1    1   4.6      1      72

Percentage of the requests served within a certain time (ms)
  50%      1
  66%      1
  75%      1
  80%      1
  90%      1
  95%      1
  98%      1
  99%      1
 100%     72 (longest request)

Repeated runs, same command

This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking forum.valka.cz (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:        lighttpd/1.4.19
Server Hostname:        forum.valka.cz
Server Port:            80

Document Path:          /images10/homepage_icons/ico-01.png
Document Length:        17933 bytes

Concurrency Level:      20
Time taken for tests:   0.186 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      18221000 bytes
HTML transferred:       17933000 bytes
Requests per second:    5363.51 [#/sec] (mean)
Time per request:       3.729 [ms] (mean)
Time per request:       0.186 [ms] (mean, across all concurrent requests)
Transfer rate:          95438.04 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.2      0       1
Processing:     1    3   0.6      4       6
Waiting:        0    1   0.7      1       3
Total:          1    4   0.5      4       6
WARNING: The median and mean for the processing time are not within a normal deviation
        These results are probably not that reliable.

Percentage of the requests served within a certain time (ms)
  50%      4
  66%      4
  75%      4
  80%      4
  90%      4
  95%      5
  98%      5
  99%      5
 100%      6 (longest request)
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking forum.valka.cz (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:        lighttpd/1.4.19
Server Hostname:        forum.valka.cz
Server Port:            80

Document Path:          /images10/homepage_icons/ico-01.png
Document Length:        17933 bytes

Concurrency Level:      20
Time taken for tests:   0.188 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      18301727 bytes
HTML transferred:       18012287 bytes
Requests per second:    5317.08 [#/sec] (mean)
Time per request:       3.761 [ms] (mean)
Time per request:       0.188 [ms] (mean, across all concurrent requests)
Transfer rate:          95031.08 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.2      0       1
Processing:     1    3   1.0      3       8
Waiting:        0    1   0.8      1       5
Total:          1    4   1.0      4       8

Percentage of the requests served within a certain time (ms)
  50%      4
  66%      4
  75%      4
  80%      4
  90%      5
  95%      5
  98%      6
  99%      6
 100%      8 (longest request)

I do not see any reason the static files shoud have this problem :frowning:

Thanks alot for the help with this topic, I really appreciate it and Im happy to learn new stuff.

7

it seems I may have found the problem, there might be disk speed issue. The gzipped files are being served from disk cache (once they are gzipped, they are stored in a separate directory and served direcly from there if requested again) on another disk, but the images are being served from disk, that seems to have slow response and transfer rate … With repeated requests like AB test, this is not visible, as the file gets cached by the linux system I’d say, but with random access to various files this is not possible and it might cause the above visible problems. Let me investigate this and come back with results

8

well, even after migration to another disk I didn’t achieve any improvement with the PNG files :frowning: still the results look ackward, even when CDN is in the game

http://www.webpagetest.org/result/111209_QP_2F53S/1/details/

seriously, what the heck is wrong with those PNG files ? :frowning: