Django Cache with Nginx uWsgi backend

Generl Concept:

Cache Improvment:

  • Decrease backend load
  • Scale up to hundreds of requests

Basic scheme

HTTP Client--> Nnginx -->  uWsgi
                 |------> Cache

Configuration

Configuration at HTTP block

uwsgi_cache_path : Path where cache file will be salve.

Subparamenters:

  • levels: number of subdirectory levels in cache.
  • keys_zone: Size of shared memory were will be salved key and metadata.
  • inactive: specify maximum inactive time cache can be stored.
wsgi_cache_path /var/cache/nginx/cache/crazylinux  levels=1:2 keys_zone=crazylinux:16m inactive=60m;

server {
    listen     443 ssl;
    server_name crazylinux.it;
    ...
}

uwsgi_cache_path: /var/cache/nginx/cache/crazylinux: my file will be salved in /var/cache/nginx/cache/crazylinux.

levels=1:2 : 2 subdirectory level for store data.

Content of  /var/cache/nginx/cache/crazylinux wit acctive cache.

[root@myserver crazylinux]# du *
8       2/6d
12      2
8       7/21
12      7
12      9/7d
8       9/9d
24      9
8       b/df
12      b
8       c/c0
12      c

keys_zone=crazylinux:16m : I use 16M of memory for store key and metadata.

inactive=60m: If a page will not call for more than 60 minute the cache will be clear.

Configuration Directive in Server Block

Link at official documentation http://nginx.org/en/docs/http/ngx_http_uwsgi_module.html

  • uwsgi_cache: Define where salve the cache file.
  • uwsgi_cache_revalidate: Enables revalidation of expired cache items.
  • uwsgi_cache_key: Define key name
  • uwsgi_cache_valid: Define how log time items can stay in cache

We can add cache status in the http Header

server {
    ...
    location / {
		...
		include     /etc/nginx/uwsgi_params;
        	uwsgi_pass  unix:///run/uwsgi/agreco_blog.sock;

		#client cache
		uwsgi_cache crazylinux;
		uwsgi_cache_revalidate on;
		uwsgi_cache_key $uri;
		uwsgi_cache_valid any 1h; #Or whatever value you want
		add_header X-Cache-Status $upstream_cache_status ;
	}
}

Other example for uwsgi_cache_valid

uwsgi_cache_valid 200 302 10m;
uwsgi_cache_valid 301      1h;
uwsgi_cache_valid any      1m;

Restart nginx

# check config
nginx -t
# Restart
systemctl restart nginx

Check if all is working find

We can use cUrl for analize the http Header and check the right functions of cache.

$ curl -I https://crazylinux.it/
HTTP/1.1 200 OK
Server: nginx/1.10.2
Date: Mon, 09 Oct 2017 20:51:09 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 5409
Connection: keep-alive
Vary: Accept-Encoding
X-Frame-Options: SAMEORIGIN
Expires: Tue, 10 Oct 2017 20:51:09 GMT
Cache-Control: max-age=86400
X-Cache-Status: MISS

The frist get return paramenter X-Cache-Status: MISS as is not in cache

So remake request

$ curl -I https://crazylinux.it/
HTTP/1.1 200 OK
Server: nginx/1.10.2
Date: Mon, 09 Oct 2017 20:52:33 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 5409
Connection: keep-alive
Vary: Accept-Encoding
X-Frame-Options: SAMEORIGIN
Expires: Tue, 10 Oct 2017 20:52:33 GMT
Cache-Control: max-age=86400
X-Cache-Status: HIT

Now the X-Cache-Status: has change in  HIT the page is in cache

How Bypass the cache

For same page is importat bypass the cache
For example i will try to configure nginx to bypass the cache at the webpage https://crazylinux.it/it/post/installazione-grafana/

Test the webpage wit cUrl  before configure cache_bypass

$ curl -I https://crazylinux.it/it/post/installazione-grafana/
HTTP/1.1 200 OK
Server: nginx/1.10.2
Date: Sun, 22 Oct 2017 12:05:22 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 6539
Connection: keep-alive
Vary: Accept-Encoding
X-Frame-Options: SAMEORIGIN
Expires: Mon, 23 Oct 2017 12:05:22 GMT
Cache-Control: max-age=86400
X-Cache-Status: HIT

The webpage is in cache X-Cache-Status: HIT

First step is configure server block, and add all uri that must be bypassed.
Second step is configure the location block and configure the directive:

  • uwsgi_cache_bypass $nocache
  • uwsgi_no_cache $nocache
if ( $request_uri ~ /it/post/installazione-grafana/ ) {set $nocache 1; }
    location / {
                uwsgi_pass  unix:///run/uwsgi/agreco_blog.sock;

                uwsgi_cache crazylinux;
                uwsgi_cache_revalidate on;
                uwsgi_cache_key $uri;
                uwsgi_cache_bypass $nocache;
                uwsgi_no_cache $nocache;
                uwsgi_cache_valid any 1h; #Or whatever value you want
                add_header X-Cache-Status $upstream_cache_status ;
        }

Restart Nginx and check wit cUrl the cache status.

$ curl -I https://crazylinux.it/it/post/installazione-grafana/
HTTP/1.1 200 OK
Server: nginx/1.10.2
Date: Sun, 22 Oct 2017 12:22:09 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 6539
Connection: keep-alive
Vary: Accept-Encoding
X-Frame-Options: SAMEORIGIN
Expires: Mon, 23 Oct 2017 12:22:09 GMT
Cache-Control: max-age=86400
X-Cache-Status: BYPASS

Cache Purge

A fast system to purge the cache is install this basic bashscript

perusio/nginx-cache-purge
git clone git://github.com/perusio/nginx-cache-purge.git

You can find all instructioon in the README locate in the repo

In alternative for full management of cache we can install cache_purge modules

FRiCKLE/ngx_cache_purge