citizen science by the Bundesregierung

Es gibt ein neues Portal für Bürger namens http://www.buergerschaffenwissen.de/ von unserer Regierung.

Was ist das und von wem?

Die Plattform Bürger schaffen Wissen ist ein gemeinsames Projekt von Wissenschaft im Dialog gGmbH und dem Museum für Naturkunde Berlin – Leibniz-Institut für Evolutions- und Biodiversitätsforschung.

Herausgeber:
Wissenschaft im Dialog gGmbH
Charlottenstrasse 80
D-10117 Berlin

Was passiert da so?

Zitat:

Die Citizen Science-Plattform Bürger schaffen Wissen präsentiert Projekte, in denen Bürger gemeinsam mit Wissenschaftlern zu verschiedenen Themen forschen können und lädt zum Mitmachen ein. Initiatoren von Citizen Science-Projekten können Projekte einstellen, Erfahrungen austauschen und sich vernetzen. Die Plattform informiert über die Aktivitäten und Ergebnisse der bundesweiten Initiative zur Förderung von Citizen Science in Deutschland.

Leseempfehlung unabhängig von dem Angebot

Why do I blog this? Wenn es von der Regierung kommt bin ich skeptisch. Ich werde das mal näher angucken und dann den Post ergänzen.

DIY Gravitation-Vacuum-Dripping-Valve as a Plant Watering System

In a few days I will go on holiday. Someone has to either look after my plants on the balcony while I am away or some kind of automatism should ensure that my plants are still alive when I am back.

So I did some googling around and found nice hints, but nothing that really kicked me enough and often I did not have the required things in the household. So I went with something quite common, a 1,5L plastic bottle of water, an empty mini glass of jar, a short piece (about 10cm) of yarn, a hot glue pistol and an edding permanent marker.

Things required

  • 1,5L plastic glass bottle (could also be a glass bottle)
  • mini jar glass (which fits the neck of the bottle tightly)
  • 10cm of yarn
  • Hot glue pistol
  • Edding permanent marker

Mini Jar Glass

Empty a mini jar glass and make it clean and dry.

Piece of yarn

Get some yarn and cut off a piece of about 10cm or longer (not too short).

Like about this size…

1,5L PlasticGlass Bottle

Hot Glue Pistol

Prepare the piece of yarn like in the photo then fetch and turn on the hot glue pistol.

Glue the yarn to the bottom of the inside of the glass like in this photo.

The gravitation-vacuum-dripping-mechanism

The bottle should now fit perfectly into the glass by putting it in upside-down. The yarn should leave the edge between bottle and glass on both sides (this is where the water should slowly drip from). The yarn should actually only ensure that there is a small opening where water can escape from the bottle.

System Installation

First dig a small hole in the earth beneath your plants. The hole can be slightly bigger than the glass itself, so the bottle will have full contact with the surrounding earth to stand stable.

Apply the prepared glass on the top of the water filled bottle like so…

…then rotate the whole thing upside-down and slowly insert it into the digged hole. Like so…

…check that the bottle is standing in 90 degress from the ground exactly in upright position to not fall over. Press some earth/dirt against the bottom surroundings to make it stand stable in the earth. Be careful that you keep the pieces of yarn below the surface so the dripping will happen below surface. Otherwise too much water will be lost to immediate evaporation.

System Monitoring

Not fetch your permanent marker.

And apply a mark for the water level in the bottle. Put a date/time beneath the mark like so…

…you can even add the correct time to have exact data on your experiment.

Congratulations!

These plants can now self-service some 1,5L of water during your stay away. The final system looks a bit weird… but it works. Hopefully!

Update: Correction on bottle material

The plastic bottles I used were not fit for the purpose. After several hours of the watering system in operation the plastic bottles looked like this…

So plastic is not stable enough to stand the atmospheric pressure while the water is sucked out of the bottom opening. So as of today I replaced the bottles by 0,5L glass bottles to see if that helps. Until now it works.

To be continued…

Why do I blog this? I wanted a quick and failsafe solution. In the last really hot summer I went with a small clay jug to fit the bottle in and I hot-glued the hole at the bottom of the jug closed. But the clay jug broke with the time and it did allow too much water to be dispensed. So this time I will try this new approach.

Installing & deploying Python web application using Debian / Apache2 / WSGI

Foreword

I was frustrated recently. I was frustrated of all the half-baked tutorials in regard to Python/Apache/WSGI deployment. Most tutorials or examples miss a huge amount of information i.e. describing the context. The usual approach seems to be to write such a tutorial just for a 15-minutes-of-fame-gig after one mastered those steps to deploy once.

The pity is, most of these tutorial writers seem to have exactly no clue what they did and why they did it. They do not explain why something is done this way and not the other way and they never mention how things might differ in your setup. They describe just their own story like in a „my little diary“-story and simply claim: „works for me.“

Installing an Open Source issue tracker

I wanted to install Roundup an open source issue tracker I would like to use to keep track of bugs and improvement ideas in my projects. Roundup is a python web application working with templates and I like Python. I chose Roundup after I checked out several lists like e.g. „15 Most Popular Bug Tracking Software to Ease Your Defect Management Process“. Other softwares which were in my focus were e.g. „Trac“ or „Mantis“. I thought, well, if this thing is Python, I might be able to easily customize it later on and since I deployed already one Python app successfully I thought this one might be quite easy.

Why apt-get install is a trap

Somewhere in the Debian Wiki I found that apt-get install roundup would just do an installation for me. At least that was what I thought…

Yeah the installation went fine, but this started a python app as a standalone server on some weird port. Oh and it started that server without asking back if I actually wanted it to start at all. I wanted this thing to run smoothly under a nice subdomain like issues.mydomainname.com in my existing apache setup. The need to explicitly put the port number in the url to reach the issue tracker like issues.mydomainname.com:8484/issues/ was not exactly what I had in mind. But the Debian wiki seems to be completely fine with the usual „works for me.“-claim I guess.

I think the person who wrote that tutorial missed the point completely that you usually want to deploy a bug tracking system. Let me tell you something Debian wiki writer no one wants a bug tracker which runs on localhost only for personal use and only reachable through it’s own explicit port. But hey, why not drop such a tutorial there anyway to make things funny for people who want to solve a problem, right?

Why explain it the right way if there is some half-right way?

Another thing most of those nice tutorial writers completely forget about is the most common way of $something. I recognized this when I read one of the better tutorials on the web.

When I am deploying something on some UNIX box, they forget to mention what kind of UNIX that box is running, they also forget to mention the version of the Apache they use and many other things & details which are important to actually reproduce the success they had. They forget to mention they run their Python application using the global system-wide python installation with all it’s installed modules. You can be happy if someone tells you upfront that his setup uses Python 3.x or Python 2.x but most of the time you recognize that painfully later on.

What does it need to succeed?

For me, I just wanted to know how to bring a bug tracking solution to life on a production server. So these are my needs:

  1. Give me screenshots of what the possible solution would look like
    because if it is an ugly piece of crap, I won’t ever spend hours installing it. Bonus, have a running demo of your solution on the web (without fancy registrations).
  2. Give me an easy to find downloadlink of the software
    most solutions hide the download of their package for installation somewhere on the page.
  3. Next to the downloadlink put the link to the installation manual. Keep that manual short and precise and provide it for the three most common use-cases. If you need some kind of rocket science preconditions which would require some neurosurgery on my running production system, then tell me here in big red letters and put a warning sign next to them.
  4. If you ever opt-in to let your software be redistributed through e.g. apt-get make damn sure that you distribute the most recent version in a meaningful way for the most common use-case.
  5. If there are preconditions then talk about those preconditions.
    E.g. tell me that in the process of installation I will need some mySQL server, so I can install that before I start with your tool.
  6. If something runs out of the box like magic, someone kept the very important details about environment variables, searchpaths & preconditions (like e.g. preinstalled global packages e.g. for Python, PHP, etc.) to himself and you will never be able to reproduce this success on a different machine.
    So please: tell me the exact environment variables, searchpaths, versions of packages needed, etc. to reproduce your success.
  7. Actually explain why something is done exactly this way so I can learn from that. Things usually have a reason why they are done this way and not the other way. But especially on UNIX there often are a thousand ways to solve a problem. Tell me why you chose exactly your approach.

How I did install Python app roundup

First things first. So here I will show you all files relevant to us right now.

Apache

Apache2 site-configuration file tasks.meinserver.de which resides in my folder /etc/apache2/sites-available and which was enabled in Apache typing in the command line
$ a2ensite tasks.meinserver.de
After activating that site you need to do a
$ service apache2 restart
which will automatically reload and apply all config changes.

# apache config for tasks.meinserver.de config
<VirtualHost *:80>
  # configure admin email and name of this domain
  ServerAdmin admin@somehost.com
  ServerName  tasks.meinserver.de
  
  # directory where usually all the static html files go
  DocumentRoot /home/www/tasks.meinserver.de/htdocs/
  
  # configure WSGI adapter
  WSGIScriptAlias / /home/www/tasks.meinserver.de/app/tasksapp.wsgi
  WSGIDaemonProcess tasksapp user=www-data group=www-data processes=1 threads=5 
  <Directory /home/www/tasks.meinserver.de/app/>
    WSGIProcessGroup tasksapp
    WSGIApplicationGroup %{GLOBAL}
    WSGIScriptReloading On
    Order deny,allow
    Allow from all
  </Directory>

  # directory where to store log files i.e. errors
  ErrorLog  /home/www/tasks.meinserver.de/logs/error.log
  CustomLog /home/www/tasks.meinserver.de/logs/access.log combined

  # Possible values include: debug, info, notice, warn, error, crit, alert, emerg.
  LogLevel warn
</VirtualHost>

As you can see, in line 08 we refer to the DocumentRoot of this particular website which is served by Apache. Usually you will not need that because your website will be created by the python app we will deploy in a minute.

But, it is nice to have this DocumentRoot and also some simple index.html inside that folder, to easily check if the Apache works like expected. You can check if Apache works by deactivating lines 10 to 19 (all related to WSGI) and what you should then see is what resides in the DocumentRoot.

WSGI

The next file is the .wsgi-file which is passed into the Apache. That file was referenced in line 11 of the siteconfig. It is basically also a python file, but it is actually only a kind of wrapper to kickstart your real Python app.

There are two important lines of code in it. On line 01 we tell Apache which Python-command should be used to execute this WSGI-file. This is important, if we do not define this, some random global python may be used. You never know which version that python is and which modules it may have installed.

So in line 01 we explicitly define that this file should be executed with a python residing in the folder /home/www/tasks.meinserver.de/app/bin we also append a searchpath in line 05 to point to the folder /home/www/tasks.meinserver.de/app. This will tell Python where to search for our .py-file when it tries to import from that file.

Both of these path statements are very important. They tell Apache how this app should be launched and they manifest which Python environment will be used for execution.

#! /home/www/tasks.meinserver.de/app/bin/python
# -*- coding: utf-8 -*-

import sys
sys.path.append('/home/www/tasks.meinserver.de/app')

from tasksapp import app as application

Python app

So tasksapp.wsgi is used to launch tasksapp.py the real application which will then produce an answer to requests coming from the web via Apache:

#! /home/www/tasks.meinserver.de/app/bin/python
# -*- coding: utf-8 -*-

import sys 
sys.path.append("/home/www/tasks.meinserver.de/app/lib/python2.7/site-packages")

from flask import Flask
 
app = Flask( __name__ )
  
@app.route("/")
def hello():
        return "<html><body><h1>Hellow world! Python WSGI app running.</h1></body></html>"

if __name__ == "__main__":
  app.run()

This simple app uses a very common module called Flask which provides a lightweight webapplication framework to easily produce dynamic websites or other responses delivered through the webserver.

But Flask does not belong to the standard python installation it has to be installed separately and someone has to tell our application where to find this module. This is why we again need to define which Python should be used to execute this file (see line 01) and where to search for the modules (see line 05).

Installation

Before we install anything that is a python script, it is best to use virtualenv to build an encapsulated environment for this python application to run in. virtualenv does exactly this, it creates a new folder by creating a full copy of your default/global python installation without all global modules you might have installed. I used following command to create my app folder.

virtualenv --always-copy /home/www/tasks.meinserver.de/app

Check out the help page for virtualenv to adjust the created folder to your needs. Maybe you want a different python version installed.

Read this again

Okay, now read this text again in backward order. I know that still several pieces are missing also in this post but I had the urge to share what I wrote down already. To be continued…

Why do I blog this? I found it not that easy to deploy python in a stable and reproducable way. So this is why I want to document a way that worked for me. I would not recommend deploying anything webserver related through apt-get installation, never do this. This is far to risky and it does not communicate the important things you need to know for mission critical deployments. Stay away from magic you do not understand.