wylie

Super User

Setting up a deploy system for Python apps on Ubuntu using Supervisor, Upstart and git

September 21, 2013

In the past, I’ve always deployed web apps and side projects to Heroku. But after being bitten by it one too many times, I decided to get a server on DigitalOcean and do it myself. It was surprisingly easy.

Setup

First, I created a user called www-data and gave it a home directory of /var/www/. Inside there, I created this directory structure:

.virtualenvs/
myproject/
myproject.git/
myproject.log

The .virtualenvs folder is used by virtualenvwrapper to manage my environments.

myproject.git is a bare repository that I set up by following these instructions for git deployment from Jeff Hoefs. You can customize the post-receive hook, for example to restart your web process. But first let’s get our process daemonized, so it’ll keep running if it crashes or the server restarts.

Supervisord

Supervisord is a Python-based process management system. It’s similar to Upstart, which is more generic. In fact, I use Upstart to make sure Supervisord starts on boot.

Upstart now ships with Ubuntu, so you just need to install Supervisord. You can install a pre-built package, or use easy_install like I did. I prefer easy_install, because the code generated by the package is hard to modify.

$ easy_install supervisor

Here’s an example program specified in /etc/supervisord.conf. By default, supervisord will start all your programs when it launches.

[program:myproject]
user = www-data
directory = /var/www/myproject
command = /var/www/.virtualenvs/myproject/bin/python main.py
numprocs = 1
stdout_logfile = /var/www/bus.log
redirect_stderr = true

I also created a small Upstart script called supervisor (without the d). This lives in /etc/init/supervisor.conf

description "supervisor"
 
start on runlevel [2345]
stop on runlevel [!2345]
 
respawn

exec /usr/local/bin/supervisord --nodaemon -c /etc/supervisord.conf

That leads us to the last part, which is the modified post-receive hook which will run your newly pushed code:

#!/bin/sh
GIT_WORK_TREE=/var/www/myproject git checkout -f
supervisorctl restart myproject

Running it

Start running your supervisor as a service, if it’s not running:

$ start supervisor

Then check the status of your project:

$ supervisorctl status myproject

You can also use supervisorctl to start, stop, or restart your web process manually.


And that’s it! My app is more reliable now, and deploying is still just a git push.