Blogged by buffi

I run a medium traffic imagehosting site which serves about 8-10000 pageviews per day using Django and I have been using the recommended deployment method (Apache + mod_python) for just under two years and it has mostly worked well. However, about two months ago I was starting to notice increasing delays from the server and Apache would occasionally fail in spectacular ways which brought the CPU load to 100% for long periods of time.

I have always used lighttpd to serve static content, and since I don’t really enjoy the Apache configuration syntax I decided to give the lighttpd + FastCGI deployment method a try. I expected this migration to be complicated, but it took me less than an hour to figure it out. I have some minor documentation of my changes below in case you are interested in the basics of how to handle a Apache -> lighttpd Django migration.

Apache configuration

My old configuration, using Apache looked like this (some stuff omitted).

<VirtualHost *>
        ServerName pici.se
        DocumentRoot /var/www
        ErrorLog /var/log/apache2/pici_error.log
        ServerAlias pici
        <Location "/">
            PythonPath "['/home/buffi/site'] + sys.path"
            SetHandler python-program
            PythonHandler django.core.handlers.modpython
            SetEnv DJANGO_SETTINGS_MODULE pici.settings
            PythonDebug On
        </Location>
        <Location "/css/"> SetHandler None </Location>
        <Location "/js/"> SetHandler None </Location>
        <Location "/im/"> SetHandler None </Location>
        <Location "/picisendfiles/"> SetHandler None </Location>
        <Location "/pictures/"> SetHandler None </Location>
        <Location "/thumbs/"> SetHandler None </Location>

        # Static content served with lighttpd.
        RewriteEngine on
        RewriteRule  ^/pictures(.*) http://static.pici.se:8080/pictures$1
        RewriteRule  ^/thumbs(.*) http://static.pici.se:8080/thumbs$1
</VirtualHost>

lighttpd configuration

The corresponding lighttpd configuration became:

$HTTP["host"] =~ "pici\.se" {
    server.document-root = "/home/buffi/site/pici/"
    fastcgi.server = (
        "/pici.fcgi" => (
            "main" => (
                "socket" => "/home/buffi/site/pici/pici.sock",
                "check-local" => "disable",
            )
        ),
    )

    alias.url = (
        "/css/" => "/home/buffi/site/pici/picipage/css/",
        "/js/" => "/home/buffi/site/pici/picipage/js/",
        "/im/" => "/home/buffi/site/pici/picipage/im/",
        "/thumbs/" => "/var/www/static/thumbs/",
        "/pictures/" => "/var/www/static/pictures/",
        "/picisendfiles/" => "/var/www/picisendfiles/",
       )

    url.rewrite-once = (
        "^(/css.*)$" => "$1",
        "^(/im.*)$" => "$1",
        "^(/js.*)$" => "$1",
        "^(/picisendfiles.*)$" => "$1",
        "^(/thumbs.*)$" => "$1",
        "^(/pictures.*)$" => "$1",
        "^(/.*)$" => "/pici.fcgi$1",
        )
}

You might notice the path of a FastCGI socket.

“socket” => “/home/buffi/site/pici/pici.sock”,

To create this socket, simply use the FastCGI script available through manage.py. I create my socket using the following command.

./manage.py runfcgi method=prefork socket=/home/buffi/site/pici/pici.sock pidfile=pici.pid

All Django requests will then be forwarded from lighttpd to the FastCGI daemon.

Conclusion

By migrating from Apache to lighttpd I noticed a nice performance boost and got a more enjoyable syntax for my configuration. I haven’t really bothered to measure the decrease in CPU load, but it is easily noticeable and my server doesn’t become as sluggish as before during heavy load. I’ve used lighttpd + FastCGI for about a month or so now, and everything seems stable. I’d recommend any Django developer using Apache to give it a try.

buffi Programming & scripting, Python, Web development django, lighttpd, pici, Python, Web development