Installing & deploying Python web application using Debian / Apache2 / WSGI

Foreword

I was frustrated recently. I was frustrated of all the half-baked tutorials in regard to Python/Apache/WSGI deployment. Most tutorials or examples miss a huge amount of information i.e. describing the context. The usual approach seems to be to write such a tutorial just for a 15-minutes-of-fame-gig after one mastered those steps to deploy once.

The pity is, most of these tutorial writers seem to have exactly no clue what they did and why they did it. They do not explain why something is done this way and not the other way and they never mention how things might differ in your setup. They describe just their own story like in a „my little diary“-story and simply claim: „works for me.“

Installing an Open Source issue tracker

I wanted to install Roundup an open source issue tracker I would like to use to keep track of bugs and improvement ideas in my projects. Roundup is a python web application working with templates and I like Python. I chose Roundup after I checked out several lists like e.g. „15 Most Popular Bug Tracking Software to Ease Your Defect Management Process“. Other softwares which were in my focus were e.g. „Trac“ or „Mantis“. I thought, well, if this thing is Python, I might be able to easily customize it later on and since I deployed already one Python app successfully I thought this one might be quite easy.

Why apt-get install is a trap

Somewhere in the Debian Wiki I found that apt-get install roundup would just do an installation for me. At least that was what I thought…

Yeah the installation went fine, but this started a python app as a standalone server on some weird port. Oh and it started that server without asking back if I actually wanted it to start at all. I wanted this thing to run smoothly under a nice subdomain like issues.mydomainname.com in my existing apache setup. The need to explicitly put the port number in the url to reach the issue tracker like issues.mydomainname.com:8484/issues/ was not exactly what I had in mind. But the Debian wiki seems to be completely fine with the usual „works for me.“-claim I guess.

I think the person who wrote that tutorial missed the point completely that you usually want to deploy a bug tracking system. Let me tell you something Debian wiki writer no one wants a bug tracker which runs on localhost only for personal use and only reachable through it’s own explicit port. But hey, why not drop such a tutorial there anyway to make things funny for people who want to solve a problem, right?

Why explain it the right way if there is some half-right way?

Another thing most of those nice tutorial writers completely forget about is the most common way of $something. I recognized this when I read one of the better tutorials on the web.

When I am deploying something on some UNIX box, they forget to mention what kind of UNIX that box is running, they also forget to mention the version of the Apache they use and many other things & details which are important to actually reproduce the success they had. They forget to mention they run their Python application using the global system-wide python installation with all it’s installed modules. You can be happy if someone tells you upfront that his setup uses Python 3.x or Python 2.x but most of the time you recognize that painfully later on.

What does it need to succeed?

For me, I just wanted to know how to bring a bug tracking solution to life on a production server. So these are my needs:

  1. Give me screenshots of what the possible solution would look like
    because if it is an ugly piece of crap, I won’t ever spend hours installing it. Bonus, have a running demo of your solution on the web (without fancy registrations).
  2. Give me an easy to find downloadlink of the software
    most solutions hide the download of their package for installation somewhere on the page.
  3. Next to the downloadlink put the link to the installation manual. Keep that manual short and precise and provide it for the three most common use-cases. If you need some kind of rocket science preconditions which would require some neurosurgery on my running production system, then tell me here in big red letters and put a warning sign next to them.
  4. If you ever opt-in to let your software be redistributed through e.g. apt-get make damn sure that you distribute the most recent version in a meaningful way for the most common use-case.
  5. If there are preconditions then talk about those preconditions.
    E.g. tell me that in the process of installation I will need some mySQL server, so I can install that before I start with your tool.
  6. If something runs out of the box like magic, someone kept the very important details about environment variables, searchpaths & preconditions (like e.g. preinstalled global packages e.g. for Python, PHP, etc.) to himself and you will never be able to reproduce this success on a different machine.
    So please: tell me the exact environment variables, searchpaths, versions of packages needed, etc. to reproduce your success.
  7. Actually explain why something is done exactly this way so I can learn from that. Things usually have a reason why they are done this way and not the other way. But especially on UNIX there often are a thousand ways to solve a problem. Tell me why you chose exactly your approach.

How I did install Python app roundup

First things first. So here I will show you all files relevant to us right now.

Apache

Apache2 site-configuration file tasks.meinserver.de which resides in my folder /etc/apache2/sites-available and which was enabled in Apache typing in the command line
$ a2ensite tasks.meinserver.de
After activating that site you need to do a
$ service apache2 restart
which will automatically reload and apply all config changes.

# apache config for tasks.meinserver.de config
<VirtualHost *:80>
  # configure admin email and name of this domain
  ServerAdmin admin@somehost.com
  ServerName  tasks.meinserver.de
  
  # directory where usually all the static html files go
  DocumentRoot /home/www/tasks.meinserver.de/htdocs/
  
  # configure WSGI adapter
  WSGIScriptAlias / /home/www/tasks.meinserver.de/app/tasksapp.wsgi
  WSGIDaemonProcess tasksapp user=www-data group=www-data processes=1 threads=5 
  <Directory /home/www/tasks.meinserver.de/app/>
    WSGIProcessGroup tasksapp
    WSGIApplicationGroup %{GLOBAL}
    WSGIScriptReloading On
    Order deny,allow
    Allow from all
  </Directory>

  # directory where to store log files i.e. errors
  ErrorLog  /home/www/tasks.meinserver.de/logs/error.log
  CustomLog /home/www/tasks.meinserver.de/logs/access.log combined

  # Possible values include: debug, info, notice, warn, error, crit, alert, emerg.
  LogLevel warn
</VirtualHost>

As you can see, in line 08 we refer to the DocumentRoot of this particular website which is served by Apache. Usually you will not need that because your website will be created by the python app we will deploy in a minute.

But, it is nice to have this DocumentRoot and also some simple index.html inside that folder, to easily check if the Apache works like expected. You can check if Apache works by deactivating lines 10 to 19 (all related to WSGI) and what you should then see is what resides in the DocumentRoot.

WSGI

The next file is the .wsgi-file which is passed into the Apache. That file was referenced in line 11 of the siteconfig. It is basically also a python file, but it is actually only a kind of wrapper to kickstart your real Python app.

There are two important lines of code in it. On line 01 we tell Apache which Python-command should be used to execute this WSGI-file. This is important, if we do not define this, some random global python may be used. You never know which version that python is and which modules it may have installed.

So in line 01 we explicitly define that this file should be executed with a python residing in the folder /home/www/tasks.meinserver.de/app/bin we also append a searchpath in line 05 to point to the folder /home/www/tasks.meinserver.de/app. This will tell Python where to search for our .py-file when it tries to import from that file.

Both of these path statements are very important. They tell Apache how this app should be launched and they manifest which Python environment will be used for execution.

#! /home/www/tasks.meinserver.de/app/bin/python
# -*- coding: utf-8 -*-

import sys
sys.path.append('/home/www/tasks.meinserver.de/app')

from tasksapp import app as application

Python app

So tasksapp.wsgi is used to launch tasksapp.py the real application which will then produce an answer to requests coming from the web via Apache:

#! /home/www/tasks.meinserver.de/app/bin/python
# -*- coding: utf-8 -*-

import sys 
sys.path.append("/home/www/tasks.meinserver.de/app/lib/python2.7/site-packages")

from flask import Flask
 
app = Flask( __name__ )
  
@app.route("/")
def hello():
        return "<html><body><h1>Hellow world! Python WSGI app running.</h1></body></html>"

if __name__ == "__main__":
  app.run()

This simple app uses a very common module called Flask which provides a lightweight webapplication framework to easily produce dynamic websites or other responses delivered through the webserver.

But Flask does not belong to the standard python installation it has to be installed separately and someone has to tell our application where to find this module. This is why we again need to define which Python should be used to execute this file (see line 01) and where to search for the modules (see line 05).

Installation

Before we install anything that is a python script, it is best to use virtualenv to build an encapsulated environment for this python application to run in. virtualenv does exactly this, it creates a new folder by creating a full copy of your default/global python installation without all global modules you might have installed. I used following command to create my app folder.

virtualenv --always-copy /home/www/tasks.meinserver.de/app

Check out the help page for virtualenv to adjust the created folder to your needs. Maybe you want a different python version installed.

Read this again

Okay, now read this text again in backward order. I know that still several pieces are missing also in this post but I had the urge to share what I wrote down already. To be continued…

Why do I blog this? I found it not that easy to deploy python in a stable and reproducable way. So this is why I want to document a way that worked for me. I would not recommend deploying anything webserver related through apt-get installation, never do this. This is far to risky and it does not communicate the important things you need to know for mission critical deployments. Stay away from magic you do not understand.

The Virtual Typewriter Museum

The The Virtual Typewriter Museum has the history and pictures of amazing machines.

Like e.g. these:


The Writing Ball

typewriter is one of the great inventions of 19th Century communications technology. Between the 1860s and 1920s engineers, inventors and even carpenters invested all their creativity in the development of the ultimate writing machine. This virtual museum, that is based on private collections of antique typewriters from around the world, is a tribute to their ingenuity.

A nice craft of typewriting machines also resonates emotionally with me. It’s sexy. And so it did not surprise me to find the book Sexy Legs and Typewriters – „Women in office-related advertising, humor, glamour and erotica.“ on the webpage there.

Women and typewriters entered into a sensual partnership of skin and steel the moment at which the first writing machine was operated by female fingers. That was back in the 1870s. Ever since, office girls have fed the male imagination for many different reasons.

Pretty women and typewriters appeared together in advertising; humorous photo series and postcards; glamour photos; and erotica. This richly illustrated book covers all these areas, from the first woman to appear in a typewriter ad to a large collection of nudes from all ages.

Why do I blog this? I like the love in detail and the variety of typewriters built.

Adding Emoji to your LaTeX document

I found a rather quick way to get emoji in my document. But this is the brute force quick way. The russian way. It always works but is not that elegant. PDF-images.

How to

  1. Read this blog post „Emoji in LaTeX documents“
  2. Download the hires emoji-images in this github-repo
  3. Search the emoji-PDF files you need for your document
  4. Copy them to the folder where your .tex-document resides
  5. Define a shortcut/macro to include those emoji-PDF files you need
  6. Include the following code in your .tex-document
\usepackage{scalerel} % needed package to scale the pdf-images perfectly
\def\emojithumbup{\scalerel*{\includegraphics{emoji_thumbup_1F44D.pdf}}{O}}
\def\emojistar{\scalerel*{\includegraphics{emoji_star_2B50.pdf}}{0}}

In your document

This my five star rating
\emojistar\emojistar\emojistar\emojistar\emojistar
I would say \emojithumbup for this easy integration.

Why do I blog this? Because when I tried to integrate the coloremoji.sty class in my LaTeX document, I got a series of issues with conflicting textinput encodings not matching the one the coloremoji expects. This way your avoid any new encoding issues by directly integrating PDF-images which is what will happen anyway with that class. maybe you need to include \usepackage[pdftex]{graphicx} too.