Foreword
I was frustrated recently. I was frustrated of all the half-baked tutorials in regard to Python/Apache/WSGI deployment. Most tutorials or examples miss a huge amount of information i.e. describing the context. The usual approach seems to be to write such a tutorial just for a 15-minutes-of-fame-gig after one mastered those steps to deploy once.
The pity is, most of these tutorial writers seem to have exactly no clue what they did and why they did it. They do not explain why something is done this way and not the other way and they never mention how things might differ in your setup. They describe just their own story like in a „my little diary“-story and simply claim: „works for me.“
Installing an Open Source issue tracker
I wanted to install Roundup an open source issue tracker I would like to use to keep track of bugs and improvement ideas in my projects. Roundup is a python web application working with templates and I like Python. I chose Roundup after I checked out several lists like e.g. „15 Most Popular Bug Tracking Software to Ease Your Defect Management Process“. Other softwares which were in my focus were e.g. „Trac“ or „Mantis“. I thought, well, if this thing is Python, I might be able to easily customize it later on and since I deployed already one Python app successfully I thought this one might be quite easy.
Why apt-get install is a trap
Somewhere in the Debian Wiki I found that apt-get install roundup would just do an installation for me. At least that was what I thought…
Yeah the installation went fine, but this started a python app as a standalone server on some weird port. Oh and it started that server without asking back if I actually wanted it to start at all. I wanted this thing to run smoothly under a nice subdomain like issues.mydomainname.com in my existing apache setup. The need to explicitly put the port number in the url to reach the issue tracker like issues.mydomainname.com:8484/issues/ was not exactly what I had in mind. But the Debian wiki seems to be completely fine with the usual „works for me.“-claim I guess.
I think the person who wrote that tutorial missed the point completely that you usually want to deploy a bug tracking system. Let me tell you something Debian wiki writer no one wants a bug tracker which runs on localhost only for personal use and only reachable through it’s own explicit port. But hey, why not drop such a tutorial there anyway to make things funny for people who want to solve a problem, right?
Why explain it the right way if there is some half-right way?
Another thing most of those nice tutorial writers completely forget about is the most common way of $something. I recognized this when I read one of the better tutorials on the web.
When I am deploying something on some UNIX box, they forget to mention what kind of UNIX that box is running, they also forget to mention the version of the Apache they use and many other things & details which are important to actually reproduce the success they had. They forget to mention they run their Python application using the global system-wide python installation with all it’s installed modules. You can be happy if someone tells you upfront that his setup uses Python 3.x or Python 2.x but most of the time you recognize that painfully later on.
What does it need to succeed?
For me, I just wanted to know how to bring a bug tracking solution to life on a production server. So these are my needs:
- Give me screenshots of what the possible solution would look like
because if it is an ugly piece of crap, I won’t ever spend hours installing it. Bonus, have a running demo of your solution on the web (without fancy registrations). - Give me an easy to find downloadlink of the software
most solutions hide the download of their package for installation somewhere on the page. - Next to the downloadlink put the link to the installation manual. Keep that manual short and precise and provide it for the three most common use-cases. If you need some kind of rocket science preconditions which would require some neurosurgery on my running production system, then tell me here in big red letters and put a warning sign next to them.
- If you ever opt-in to let your software be redistributed through e.g.
apt-getmake damn sure that you distribute the most recent version in a meaningful way for the most common use-case. - If there are preconditions then talk about those preconditions.
E.g. tell me that in the process of installation I will need some mySQL server, so I can install that before I start with your tool. - If something runs out of the box like magic, someone kept the very important details about environment variables, searchpaths & preconditions (like e.g. preinstalled global packages e.g. for Python, PHP, etc.) to himself and you will never be able to reproduce this success on a different machine.
So please: tell me the exact environment variables, searchpaths, versions of packages needed, etc. to reproduce your success. - Actually explain why something is done exactly this way so I can learn from that. Things usually have a reason why they are done this way and not the other way. But especially on UNIX there often are a thousand ways to solve a problem. Tell me why you chose exactly your approach.
How I did install Python app roundup
First things first. So here I will show you all files relevant to us right now.
Apache
Apache2 site-configuration file tasks.meinserver.de which resides in my folder /etc/apache2/sites-available and which was enabled in Apache typing in the command line
$ a2ensite tasks.meinserver.de
After activating that site you need to do a
$ service apache2 restart
which will automatically reload and apply all config changes.
# apache config for tasks.meinserver.de config
<VirtualHost *:80>
# configure admin email and name of this domain
ServerAdmin admin@somehost.com
ServerName tasks.meinserver.de
# directory where usually all the static html files go
DocumentRoot /home/www/tasks.meinserver.de/htdocs/
# configure WSGI adapter
WSGIScriptAlias / /home/www/tasks.meinserver.de/app/tasksapp.wsgi
WSGIDaemonProcess tasksapp user=www-data group=www-data processes=1 threads=5
<Directory /home/www/tasks.meinserver.de/app/>
WSGIProcessGroup tasksapp
WSGIApplicationGroup %{GLOBAL}
WSGIScriptReloading On
Order deny,allow
Allow from all
</Directory>
# directory where to store log files i.e. errors
ErrorLog /home/www/tasks.meinserver.de/logs/error.log
CustomLog /home/www/tasks.meinserver.de/logs/access.log combined
# Possible values include: debug, info, notice, warn, error, crit, alert, emerg.
LogLevel warn
</VirtualHost>
As you can see, in line 08 we refer to the DocumentRoot of this particular website which is served by Apache. Usually you will not need that because your website will be created by the python app we will deploy in a minute.
But, it is nice to have this DocumentRoot and also some simple index.html inside that folder, to easily check if the Apache works like expected. You can check if Apache works by deactivating lines 10 to 19 (all related to WSGI) and what you should then see is what resides in the DocumentRoot.
WSGI
The next file is the .wsgi-file which is passed into the Apache. That file was referenced in line 11 of the siteconfig. It is basically also a python file, but it is actually only a kind of wrapper to kickstart your real Python app.
There are two important lines of code in it. On line 01 we tell Apache which Python-command should be used to execute this WSGI-file. This is important, if we do not define this, some random global python may be used. You never know which version that python is and which modules it may have installed.
So in line 01 we explicitly define that this file should be executed with a python residing in the folder /home/www/tasks.meinserver.de/app/bin we also append a searchpath in line 05 to point to the folder /home/www/tasks.meinserver.de/app. This will tell Python where to search for our .py-file when it tries to import from that file.
Both of these path statements are very important. They tell Apache how this app should be launched and they manifest which Python environment will be used for execution.
#! /home/www/tasks.meinserver.de/app/bin/python
# -*- coding: utf-8 -*-
import sys
sys.path.append('/home/www/tasks.meinserver.de/app')
from tasksapp import app as application
Python app
So tasksapp.wsgi is used to launch tasksapp.py the real application which will then produce an answer to requests coming from the web via Apache:
#! /home/www/tasks.meinserver.de/app/bin/python
# -*- coding: utf-8 -*-
import sys
sys.path.append("/home/www/tasks.meinserver.de/app/lib/python2.7/site-packages")
from flask import Flask
app = Flask( __name__ )
@app.route("/")
def hello():
return "<html><body><h1>Hellow world! Python WSGI app running.</h1></body></html>"
if __name__ == "__main__":
app.run()
This simple app uses a very common module called Flask which provides a lightweight webapplication framework to easily produce dynamic websites or other responses delivered through the webserver.
But Flask does not belong to the standard python installation it has to be installed separately and someone has to tell our application where to find this module. This is why we again need to define which Python should be used to execute this file (see line 01) and where to search for the modules (see line 05).
Installation
Before we install anything that is a python script, it is best to use virtualenv to build an encapsulated environment for this python application to run in. virtualenv does exactly this, it creates a new folder by creating a full copy of your default/global python installation without all global modules you might have installed. I used following command to create my app folder.
virtualenv --always-copy /home/www/tasks.meinserver.de/app
Check out the help page for virtualenv to adjust the created folder to your needs. Maybe you want a different python version installed.
Read this again
Okay, now read this text again in backward order. I know that still several pieces are missing also in this post but I had the urge to share what I wrote down already. To be continued…
Why do I blog this? I found it not that easy to deploy python in a stable and reproducable way. So this is why I want to document a way that worked for me. I would not recommend deploying anything webserver related through apt-get installation, never do this. This is far to risky and it does not communicate the important things you need to know for mission critical deployments. Stay away from magic you do not understand.