Sleep Is For The Weak

Continuing my saga of trying to extract every ounce of performance out of my new hosting environment, it’s time to move on to caching.  Up until this point every time a page is loaded the PHP scripts have to query the database for the appropriate records, take the information from the database, and render the page into static HTML that your browser can render.  Computers are fast so this doesn’t take very long, as evidenced by previous benchmarks.  However you’ll notice that even with the APC OpCode Cache in place we’re still hammering the CPU for 50% of its capacity during the benchmarking.  I know this can be improved!

Enter the plugin for WordPress called WP-SuperCache.  Now because I’m running the nginx web server and not Apache I will be unable to use the “super” portion of WP-SuperCache which works with the Apache Module Mod_Rewrite.  However I can still use a static cache of pre-rendered .html to improve site performance.  Installation of WP-SuperCache is pretty straightforward.  You can either upload the folder to your /wp-content/plugins/ folder, or you can use the FTP plugin system within your WordPress Dashboard.  I opted for the manual method as I was able to just wget the file from the server which has a lot more bandwidth than I have at home.

Once WP-SuperCache is installed you have to enable it through the dashboard.  After the plugin is enabled you can adjust its settings.  Because of my selection in webservers I opted to set the plugin to Half-On, Don’t Cache pages for logged in users, and Cache Rebuild.  I then benchmarked the site again from my home computer.

longhammer$ ab -c 25 -t 500 -r http://insa.w9zeb.org/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking insa.w9zeb.org (be patient)
Finished 4667 requests

Server Software:        nginx/0.6.36
Server Hostname:        insa.w9zeb.org
Server Port:            80

Document Path:          /
Document Length:        37195 bytes

Concurrency Level:      25
Time taken for tests:   500.041 seconds
Complete requests:      4667
Failed requests:        8
(Connect: 8, Receive: 0, Length: 0, Exceptions: 0)
Write errors:           0
Total transferred:      174921511 bytes
HTML transferred:       173825455 bytes
Requests per second:    9.33 [#/sec] (mean)
Time per request:       2678.598 [ms] (mean)
Time per request:       107.144 [ms] (mean, across all concurrent requests)
Transfer rate:          341.62 [Kbytes/sec] received

Connection Times (ms)
min  mean[+/-sd] median   max
Connect:       40  785 5663.6    149   67346
Processing:   451 1753 675.3   1605    7765
Waiting:       49  175 126.6    156    1983
Total:        553 2538 5737.4   1767   70928

During the benchmark top reported the following server load

last pid:  7037;  load averages:  0.00,  0.12,  0.20    up 0+00:48:03  21:17:42
42 processes:  1 running, 41 sleeping
CPU:  0.2% user,  0.0% nice,  1.1% system,  0.2% interrupt, 98.5% idle
Mem: 53M Active, 33M Inact, 57M Wired, 1444K Cache, 55M Buf, 848M Free
Swap: 1920M Total, 1920M Free

I’d like to highlight a few of the important numbers in comparison to the APC benchmark.

Completed Requests:  4578 vs 4667 a 1.94% improvement

Requests Per Second: 9.16 vs 9.33 a 1.86% improvement

Time Per Request:  109.218 vs 107.144 a 1.90% improvement

Transfer Rate: 333.78 KB/s vs 341.62 KB/s which is a 2.35% improvement.

CPU Utilization: 56.6% vs 0.2% which is a 99.65% improvement

Thanks to: Bolt of Blue for the image

As I talked about in a previous post, PHP is an interpreted language.  Each page load requires the code to be read, interpreted, and compiled on the fly each time a browser makes a request.  I’m running nginx as my base web server which doesn’t natively support PHP so I have to use FastCGI.  In the previous post, if you could have seen the full output of top you would have seen four PHP-CGI processes adding up to that 75% CPU load.

Just like the vBulletin board tuning I did on Apache earlier I opted to install APC OpCode Cache on my server.  This gives the advantage of pre-compiling the PHP into OpCode and then calling that from cache rather than recompiling every time a page loads.

Installing APC on FreeBSD is a simple task.  I used

[lars@insa]$ sudo portinstall -Rr www/pecl-apc

to install the FreeBSD Port for APC and added the following to my /usr/local/etc/php.ini file

extension=apc.so

apc.enabled = 1
apc.shm_segments = 1
apc.shm_size = 32
apc.filters=wp-cache-config

As I mentioned in previous posts I intend to benchmark my server configuration each step of the way.  Below are the benchmarks run from my home against the remote server.

longhammer$ ab -c 25 -t 500 http://insa.w9zeb.org/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking insa.w9zeb.org (be patient)
Finished 4578 requests

Server Software:        nginx/0.6.36
Server Hostname:        insa.w9zeb.org
Server Port:            80

Document Path:          /
Document Length:        37071 bytes

Concurrency Level:      25
Time taken for tests:   500.001 seconds
Complete requests:      4578
Failed requests:        5
(Connect: 5, Receive: 0, Length: 0, Exceptions: 0)
Write errors:           0
Total transferred:      170893494 bytes
HTML transferred:       169938150 bytes
Requests per second:    9.16 [#/sec] (mean)
Time per request:       2730.456 [ms] (mean)
Time per request:       109.218 [ms] (mean, across all concurrent requests)
Transfer rate:          333.78 [Kbytes/sec] received

Connection Times (ms)
min  mean[+/-sd] median   max
Connect:       47  736 5350.5    143   67352
Processing:   731 1915 673.9   1764   11314
Waiting:      106  285 162.9    239    2093
Total:        850 2651 5429.3   1925   71230

last pid:   903;  load averages:  2.33,  1.79,  0.91    up 0+00:10:56  20:40:35

43 processes:  5 running, 38 sleeping

CPU: 56.6% user,  0.0% nice, 11.7% system,  0.6% interrupt, 31.2% idle

Mem: 52M Active, 18M Inact, 31M Wired, 64K Cache, 20M Buf, 892M Free

Swap: 1920M Total, 1920M Free

Completed Requests:  3542 vs 4578 a 29.25% improvement

Requests Per Second: 7.08 vs 9.16 a 29.38% improvement

Time Per Request:  141.169 vs 109.218 a 22.63% improvement

Transfer Rate: 257.99 KB/s vs 333.78 KB/s which is a 24.73% improvement.

CPU Utilization: 75.2% vs 56.6% which is a 24.73% improvement

Thanks to: LiquidX for the image

Installing nginx as the webserver for a PHP site requires some dedication.  nginx does not interpret PHP with any integration at all.  You must use Fast-CGI.  I will attempt to cover the steps I took to install and configure Fast-CGI on my server.  First we get to install some packages.  I’m on FreeBSD and use portinstall, which is part of the /usr/ports/ports-mgt/portupgrade package.

[lars@insa ~]$ sudo portinstall -Rr nginx
[lars@insa ~]$ sudo portinstall -Rr lighttpd

I installed Lighttpd to get the “spawn-fcgi” package.  Next we add nginx to /etc/rc.conf by adding to the file.

nginx_enable=”YES”

location ~ \.php$ {

fastcgi_pass   127.0.0.1:9000;

fastcgi_index  index.php;

fastcgi_param  SCRIPT_FILENAME  /usr/local/www/nginx$fastcgi_script_name;

include        /usr/local/etc/nginx/fastcgi.conf;

}

fastcgi_param  QUERY_STRING       $query_string;

fastcgi_param  REQUEST_METHOD     $request_method;

fastcgi_param  CONTENT_TYPE       $content_type;

fastcgi_param  CONTENT_LENGTH     $content_length;

fastcgi_param  SCRIPT_NAME        $fastcgi_script_name;

fastcgi_param  REQUEST_URI        $request_uri;

fastcgi_param  DOCUMENT_URI       $document_uri;

fastcgi_param  DOCUMENT_ROOT      $document_root;

fastcgi_param  SERVER_PROTOCOL    $server_protocol;

fastcgi_param  GATEWAY_INTERFACE  CGI/1.1;

fastcgi_param  SERVER_SOFTWARE    nginx;

fastcgi_param  REMOTE_ADDR        $remote_addr;

fastcgi_param  REMOTE_PORT        $remote_port;

fastcgi_param  SERVER_ADDR        $server_addr;

fastcgi_param  SERVER_PORT        $server_port;

fastcgi_param  SERVER_NAME        $server_name;

# PHP only, required if PHP was built with –enable-force-cgi-redirect

fastcgi_param  REDIRECT_STATUS    200;

/usr/local/bin/spawn-fcgi -a 127.0.0.1 -p 9000 -u www -g www -C 4 -f /usr/local/bin/php-cgi

/usr/local/sbin/nginx -c /usr/local/etc/nginx/nginx.conf

Thanks to: ph0t0 {loves you too} for the image.

I’m in the process of moving http://w9zeb.org to a new server.  I’m moving out of a shared environment managed by H-Sphere, hosted on CentOS 5.2 and onto a FreeBSD 7.1-Release Virtual Machine hosted on VMWare ESXi.  It’s not that I was unhappy with my previous host, but rather my interest in tinkering with all of the pieces that deliver a website.  I wanted to be able to tune MySQL, PHP, the file system, etc. and benchmark the effects of those changes without worrying about breaking the other 100 or so sites hosted under H-Sphere.

The technologies that I am planning on using for this site initially are as follows:

• FreeBSD 7.1-Release
• nginx 0.6.36
• varnish 2.0.4
• MySQL 5.1.33
• PHP 5.2.9
• FastCGI

I am intending to benchmark the performance of the server at each step along the way.  Below is the first benchmark which is on a raw, virtually untuned server.

longhammer$ ab -c 25 -t 500 http://insa.w9zeb.org/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking insa.w9zeb.org (be patient)
Finished 3542 requests

Server Software:        nginx/0.6.36
Server Hostname:        insa.w9zeb.org
Server Port:            80

Document Path:          /
Document Length:        37071 bytes

Concurrency Level:      25
Time taken for tests:   500.021 seconds
Complete requests:      3542
Failed requests:        5
(Connect: 5, Receive: 0, Length: 0, Exceptions: 0)
Write errors:           0
Total transferred:      132094346 bytes
HTML transferred:       131356986 bytes
Requests per second:    7.08 [#/sec] (mean)
Time per request:       3529.228 [ms] (mean)
Time per request:       141.169 [ms] (mean, across all concurrent requests)
Transfer rate:          257.99 [Kbytes/sec] received

Connection Times (ms)
min  mean[+/-sd] median   max
Connect:       40  445 4386.8     48   67208
Processing:   651 2982 579.3   2959    8071
Waiting:      342 2173 442.8   2228    3694
Total:        694 3427 4468.5   3015   73897

During this base benchmark top reported the following load on the server

last pid:  9201;  load averages:  4.21,  3.84,  2.55    up 0+04:10:32  20:21:32
42 processes:  5 running, 37 sleeping
CPU: 75.2% user,  0.0% nice, 23.1% system,  1.7% interrupt,  0.0% idle
Mem: 66M Active, 33M Inact, 81M Wired, 1200K Cache, 92M Buf, 811M Free
Swap: 1920M Total, 1920M Free

As you can see the CPU is being hammered and our load averages are > 2.5 during the test.  I know that with some tuning we can improve these numbers.  Be sure to check back for followups on the performance tuning I am doing on the new server.

Thanks to: paradoxperfect for the image