Rapid Application Development with Open Source Software

Next: Tuning Bootstrap 3 UP: TOC

Our objective is to cache html-pages using Memcached which is called directly from Nginx. This eliminates calls to Cakephp and the PostgreSQL database. Hence, performance gains up to 400% can be achieved.

This picture illustrates what we could like to archive:

Measure the current response time

First of all we need to know what is the current response time without caching. This can be evaluated using the curl command.

curl -w "@curl-format.txt" -o /dev/null -s "http://www.logikfabrik.com"

           melookup:  0.004
       time_connect:  0.233
    time_appconnect:  0.000
   time_pretransfer:  0.234
      time_redirect:  0.000
 time_starttransfer:  2.080
                    ----------
         time_total:  2.315

The format file for curl (curl-format.txt) has this content:

        melookup:     %{time_namelookup}\n
       time_connect:  %{time_connect}\n
    time_appconnect:  %{time_appconnect}\n
   time_pretransfer:  %{time_pretransfer}\n
      time_redirect:  %{time_redirect}\n
 time_starttransfer:  %{time_starttransfer}\n
                    ----------\n
         time_total:  %{time_total}\n

Configure Memcached

In FreeBSD memcached parmeters are defined in /etc/rc.conf. We assign 1GB of memory for memcached:

memcached_enable="YES"
memcached_args="-m 1024"

Write html to memcached

We start with writing the generated html to memcached as illustrated in Step 4 of the architecture picture. In CakePHP this will be implemented using a filter. This filter looks as follows:

data['request'];
        $response = $event->data['response'];

        if (strlen($response->body()) === 0) return;
 
        // do not cache admin URLs
        if (substr($request->url,0,5) === 'admin') return;

        // check for mobile....
        if (stripos($_SERVER['HTTP_USER_AGENT'],'iphone') === FALSE and 
            stripos($_SERVER['HTTP_USER_AGENT'],'mobile') === FALSE and 
           ) {
           error_log('Routing  key: '.md5('/'.$request->url).' SizeBody: '.strlen($response->body()).
             ' URL: /'.$request->url .' UA: ' . $_SERVER['HTTP_USER_AGENT']);   
           Cache::write(md5('/'. $request->url), $response->body() , 'short15m');
        } else {
           Cache::write(md5('m/'. $request->url), $response->body() , 'short15m');
           error_log('Routing  key: '.md5('/'.$request->url).' SizeBody: '.strlen($response->body()).
              ' URL: m/'.$request->url .' UA: ' . $_SERVER['HTTP_USER_AGENT']);   
        }
     }
}

In the configuration file app/Config/bootstrap.php we need to indicate to use the above filter:

Configure::write('Dispatcher.filters', array(
	'AssetDispatcher',
	'CacheDispatcher'
        ,'NginxMemcachedFilter'
));

Configure Nginx to use memcached

When nginx processes a request, it should first have a look into the cache before calling CakePHP.

The nginx.conf file needs to have a perl function which generates a md5 hash key for a given url. This function looks as follows:

http {
perl_set $md5_uri '
   sub {
      use Digest::MD5 qw(md5 md5_hex md5_base64);
      my $r = shift;

      #$r->log_error(0,md5_hex($r->variable("request_uri")));
      #$r->log_error(0,$r->variable("request_uri"));
      $r->log_error(0,$r->header_in("User-Agent"));

      my $ua = $r->header_in("User-Agent");
                                                                                                
      if ($ua =~ /iPhone/ or                                                                    
          $ua =~ /mobile/                                                                       
         ) {                                                                                    
         $r->log_error(0,"mobile");                                                             
         return md5_hex("m". $r->variable("request_uri"));                                      
      }                                                                                         
      return md5_hex($r->variable("request_uri"));                                              
   }                                                                                            
';                     

    include       mime.types;
    default_type  application/octet-stream;
.....

Second we need to change the vhosts.d/domain.conf to call memcached

   location ~* \.php$ {                              # added Check memcache first
      set $memcached_key $md5_uri;      # generate the memcached_key (see ngnix.conf)
      memcached_pass 127.0.0.1:11211;  # pass request to memcached
      default_type text/html;
      error_page 404 405 502 = @nocache;  # if not found pass to location nocache
    }

    location @nocache {              # added * 20130522
       try_files $uri =404;
      include /usr/local/etc/nginx/fastcgi_params;
       fastcgi_pass 127.0.0.1:9000;
       fastcgi_index index.php;
       fastcgi_intercept_errors on;
       fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    }
...

Monitoring Memcached

 ( echo "stats" ; echo quit ) | nc localhost 11211 

The above command produces this output:

STAT pid 22717
STAT uptime 7058
STAT time 1430035042
STAT version 1.4.22
STAT libevent 2.0.22-stable
STAT pointer_size 64
STAT rusage_user 0.580456
STAT rusage_system 1.461889
STAT curr_connections 10
STAT total_connections 4900
STAT connection_structures 17
STAT reserved_fds 20
STAT cmd_get 5956
STAT cmd_set 2684
STAT cmd_flush 0
STAT cmd_touch 0
STAT get_hits 4330
STAT get_misses 1626
STAT delete_misses 0
STAT delete_hits 1
STAT incr_misses 0
STAT incr_hits 0
STAT decr_misses 0
STAT decr_hits 0
STAT cas_misses 0
STAT cas_hits 0
STAT cas_badval 0
STAT touch_hits 0
STAT touch_misses 0
STAT auth_cmds 0
STAT auth_errors 0
STAT bytes_read 44708073
STAT bytes_written 57953215
STAT limit_maxbytes 67108864
STAT accepting_conns 1
STAT listen_disabled_num 0
STAT threads 4
STAT conn_yields 0
STAT hash_power_level 16
STAT hash_bytes 524288
STAT hash_is_expanding 0
STAT malloc_fails 0
STAT bytes 18191223
STAT curr_items 622
STAT total_items 2684
STAT expired_unfetched 635
STAT evicted_unfetched 0
STAT evictions 0
STAT reclaimed 655
STAT crawler_reclaimed 0
STAT lrutail_reflocked 0
END

Performance with Memcached

Performance is measured as follows:

curl -w "@curl.txt" -o /dev/null -s http://www.logikfabrik.com

                 timelookup:  0.004
               time_connect:  0.232
            time_appconnect:  0.000
           time_pretransfer:  0.232
              time_redirect:  0.000
         time_starttransfer:  0.487
                            ----------
                 time_total:  0.488

Conclusion

Using major opensource software such as Nginx, CakePHP, Memcached, PostgreSQL and carefully configured will result into highly optimized and performant websites. In this case a reduction of response time from 2.753ms to 488ms has been achieved.

Next: Tuning Bootstrap 3 UP: TOC