Provisioning a new Ubuntu server for Django
I've been a long-time satisfied user of Webfaction, but recently I've had a strong urge to move to VPS hosting so that I can have greater control over the environment. After some research, I went with Rackspace Cloud because of the incredibly cheap low-end options. My site doesn't use a huge amount of bandwidth, so Rackspace looks to be the most feature-packed and still cost-effective option.
A friend of mine, Kevin Whitaker, recently posted a great article about getting up and running with Django in a server environment for testing or production. He used Ubuntu, Postgres, Nginx, and FastCGI to make up his stack. I've never set up Nginx before, so his post was a great help in getting Nginx configured. My stack is slightly different, however, since I prefer to use Gunicorn instead of FastCGI and I use supervisord to manage my processes. I also use virtualenv to manage dependencies like Django itself and psycopg2.
I'm including my install notes below so that I can provide a complete picture of one way of provisioning a new server from build to deploy. My site is relatively low-traffic, so I'm not going to go into postgres tweaking, connection pooling, caching, etc. I'll save those topics for another post.
These notes were made on a Rackspace Cloud instance, but they should apply to a variety of other VPS hosts with only minor modifications. If you've got tips on things that need to be changed for other providers, let me know and I'll edit the post. I want this to stay up to date for awhile.
After the build is complete
Configuration
My first step was to log into my new instance using SSH. You should get an email from Rackspace with the IP address and root password.
$ ssh root@123.45.67.890 root@123.45.67.890's password: Welcome to Ubuntu!
I then installed my preferred editor - obviously an optional step.
$ apt-get install emacs23-nox $ export EDITOR=emacs
Then I used visudo to set up sudo access the way I like it.
$ visudo
Under this line:
Defaults env_reset
I added the following lines to preserve my editor and Django environment settings:
Defaults env_keep += "EDITOR VISUAL" Defaults env_keep += "DJANGO_SETTINGS_MODULE PYTHONPATH"
Then I changed this line:
%sudo ALL=(ALL) ALL
To:
%admin ALL=(ALL) ALL
This allows users with the group admin to use sudo for all commands.
Then I added my first user and group. I typically prefer to use a separate user account for each developer or administrator that will be accessing the box, and add them to appropriate groups for access.
$ adduser myuser $ addgroup admin $ adduser myuser admin
Then I switched to my new user, and added my public key so that I can SSH without requiring a password. If you're not familiar with this, the guide at Github is quick and to the point.
$ su - myuser $ mkdir .ssh $ cd .ssh $ wget http://mydomain.com/path/to/my/pubkey/id_rsa.pub $ mv id_rsa.pub authorized_keys
Then, I logged out and made sure I can SSH in using the new user with no problems. Once that's done, I locked down the root account for security (probably because I'm a long time Ubuntu user and it just feels wrong any other way).
$ sudo passwd -l root
I also make my editor choice permanent.
$ sudo update-alternatives --config editor
Nginx
With that taken care of, I was ready to install Nginx.
$ sudo apt-get install nginx $ sudo /etc/init.d/nginx start $ sudo emacs /etc/nginx/nginx.conf
I used this Slicehost article to learn about the basic nginx conf options.
Postgres
Then I installed postgres. I first changed the password on the postgres user for security.
$ sudo apt-get install postgresql $ sudo passwd postgres $ sudo -u postgres psql postgres=# \password postgres postgres=# \q
Then I set up a new user to use for my site.
$ sudo -u postgres createuser myproject $ sudo -u postgres psql postgres=# \password myproject postgres=# \q
I use a Unix domain socket for my postgres connection, so I have to edit pg_hba.conf to allow md5 login from domain sockets.
$ sudo emacs /etc/postgresql/8.4/main/pg_hba.conf
I change the lines that say:
# "local" is for Unix domain socket connections only local all all ident
To:
# "local" is for Unix domain socket connections only local all all md5
Then:
$ sudo /etc/init.d/postgresql-8.4 restart
(Substitute your favorite VCS here)
Now for my VCS. I use the Git PPA.
$ sudo apt-get install python-software-properties $ sudo add-apt-repository ppa:git-core/ppa $ sudo apt-get update $ sudo apt-get install git-core
Setting up a Django site
Setting up the structure
First, I pulled over my git repositories. I use the /code directory for hosting the master copies of my repositories to use as a hub. I keep the live checkouts in /sites/mydomain.com/code/.
$ sudo mkdir /code $ sudo chown myuser:admin /code $ cd /code $ git clone --mirror olddomain.com:/code/repository_name.git $ sudo mkdir /sites $ sudo chown myuser:admin /sites $ cd /sites $ mkdir -p mydomain.com/{code,public,logs,backup} $ cd mydomain.com $ sudo chown :www-data logs public $ sudo chmod g+w logs public $ git clone /code/repository_name.git code/ Initialized empty Git repository in /sites/mydomain.com/code/.git/
Media
I set up the media and made sure www-data could access it and write to the root directory.
$ cd public $ ln -s ../code/projectname/media $ sudo chown :www-data media $ sudo chmod g+w media
Then I synced the media from my old server, so that user-created media wouldn't be lost.
$ rsync -avz --progress old.domain.com:~/path/to/media/ media/
Restoring the database
The next step was backing up and restoring the database.
$ ssh old.domain.com $ pg_dump dbname | bzip2 > dbname.`date +%Y%m%dT%H%M%S`.sql.bz2 $ exit $ ssh new.domain.com $ cd /sites/mydomain.com/backup/ $ scp old.domain.com:~/dnmame.<timestamp>.sql.bz2 ./ $ sudo su postgres $ createdb dbname $ bzcat dnmame.<timestamp>.sql.bz2 | psql dbname $ psql dbname postgres=# GRANT ALL ON DATABASE dbname TO myproject; postgres=# \q $ exit
Installing dependencies
I then created my virtualenv and installed my dependencies.
$ sudo apt-get install build-essential python-dev libpq-dev $ sudo apt-get install python-setuptools $ sudo easy_install -U pip $ sudo pip install virtualenv $ sudo pip install virtualenvwrapper $ mkdir /sites/.virtualenvs $ emacs ~/.bashrc
At the bottom of .bashrc:
export WORKON_HOME=/sites/.virtualenvs source /usr/local/bin/virtualenvwrapper.sh
Then, back in the shell:
$ . ~/.bashrc $ mkvirtualenv projectname $ workon projectname $ pip install -r /sites/mydomain.com/code/requirements.txt $ deactivate
PIL is a tough one. I usually just install it globally. Since I install my virtualenv's with --no-site-packages, I need to symlink PIL into the site-packages.
$ sudo apt-get install python-imaging $ workon projectname $ cdsitepackages $ ln -s /usr/lib/python2.6/dist-packages/PIL $ ln -s /usr/lib/python2.6/dist-packages/PIL.pth
Then, I edited my project's settings.py to reflect the new environment setup.
Finally, I added ntpdate to keep my server-s clock in sync.
$ sudo dpkg-reconfigure tzdata $ sudo apt-get install ntpdate $ sudo crontab -e 30 23 * * * /usr/sbin/ntpdate ntp.ubuntu.com > /dev/null
Nginx
I added an nginx.conf to my source control.
$ emacs /sites/mydomain.com/code/deploy/nginx.conf
The Nginx conf:
server { listen 80; server_name www.mydomain.com; rewrite ^/(.*) http://mydomain.com/$1 permanent; } server { listen 80; server_name mydomain.com; access_log /sites/mydomain.com/logs/access.log; error_log /sites/mydomain.com/logs/error.log; location /media { root /sites/mydomain.com/public; } location / { proxy_pass http://127.0.0.1:29000; } }
I then symlinked it into /etc/nginx/sites-available/ and sites-enabled/.
$ sudo ln -s /sites/mydomain.com/code/deploy/nginx.conf /etc/nginx/sites-available/mydomain.com $ sudo ln -s /etc/nginx/sites-available/mydomain.com /etc/nginx/sites-enabled/mydomain.com $ sudo /etc/init.d/nginx restart
Gunicorn
Gunicorn should be installed in your virtualenv as part of your requirements.txt. If you're not using virtualenv and pip, or zc.buildout, then you should definitely read up on them. They are a vital part of any serious Django stack.
$ emacs /sites/mydomain.com/code/deploy/gunicorn.conf.py
I already had a simple Gunicorn conf in my source control, which I simply modified for the new environment.
bind = "127.0.0.1:29000" logfile = "/sites/mydomain.com/logs/gunicorn.log" workers = 3
Supervisord
$ sudo pip install supervisor $ sudo emacs /etc/supervisord.conf
Here's a basic config file:
[unix_http_server] file=/tmp/supervisor.sock ; (the path to the socket file) [supervisord] logfile=/var/log/supervisord.log ; (main log file;default $CWD/supervisord.log) logfile_maxbytes=50MB ; (max main logfile bytes b4 rotation;default 50MB) logfile_backups=10 ; (num of main logfile rotation backups;default 10) loglevel=info ; (log level;default info; others: debug,warn,trace) pidfile=/var/run/supervisord.pid ; (supervisord pidfile;default supervisord.pid) nodaemon=false ; (start in foreground if true;default false) minfds=1024 ; (min. avail startup file descriptors;default 1024) minprocs=200 ; (min. avail process descriptors;default 200) [rpcinterface:supervisor] supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface [supervisorctl] serverurl=unix:///tmp/supervisor.sock ; use a unix:// URL for a unix socket [program:myproject] command=/sites/.virtualenvs/myproject/bin/gunicorn_django -c deploy/gunicorn.conf.py directory=/sites/mydomain.com/code user=www-data autostart=true autorestart=true stdout_logfile=/sites/mydomain.com/logs/supervisord.log redirect_stderr=true
Then I created an init.d script:
$ sudo emacs /etc/init.d/supervisord
Here's what I used:
# Supervisord auto-start # # description: Auto-starts supervisord # processname: supervisord # pidfile: /var/run/supervisord.pid SUPERVISORD=/usr/local/bin/supervisord SUPERVISORCTL=/usr/local/bin/supervisorctl case $1 in start) echo -n "Starting supervisord: " $SUPERVISORD echo ;; stop) echo -n "Stopping supervisord: " $SUPERVISORCTL shutdown echo ;; restart) echo -n "Stopping supervisord: " $SUPERVISORCTL shutdown echo echo -n "Starting supervisord: " $SUPERVISORD echo ;; esac
Then I finished up the init.d script.
$ sudo chmod +x /etc/init.d/supervisord $ sudo update-rc.d supervisord defaults $ sudo /etc/init.d/supervisord start
If all goes well, you can check the status on your site.
$ sudo supervisorctl status
myproject RUNNING pid 11616, uptime 0:00:03
You can also grep for the gunicorn processes.
$ ps -ef | grep gunicorn
Testing
To test this new setup, you can add domain name overrides to the /etc/hosts file on your local machine.
# Do this on your local machine - not your server $ sudo emacs /etc/hosts 123.45.67.890 mydomain.com
Now you can go to your browser and access mydomain.com. You should see your site.
To restart your site, use:
$ sudo supervisorctl restart myproject
Make sure to change your /etc/hosts back!
Comments
Brandon Konkle
I've been creating websites for over 10 years, and I've been using Django since early 2008. I focus on high quality, well-tested, maintainable code and reliable high-performance deployments. Web development is something that I am very excited about, and I love finding elegant and innovative ways to push web applications further.
