Django’ing with Bluemix

Here are some of my experiences getting Django working on Bluemix.  I got it working most of the way that is until you try to access the database.  These instructions might be helpful to someone who is also getting started getting Django going on Bluemix.  Leave some suggestions if you know how I might get around my problem at the end.  This post assumes you know how to use Django.  Django is a Web Application framework written in Python based on Model View Controller pattern.  Having previously deployed an existing Django Application I built on Heroku, I thought I would try to port the application to Bluemix to see if it would work.   I was surprised at how quickly I was able to move it to Bluemix.  The reason is was able to port it was because I am not using any proprietary or unique services in the application that only existed on Heroku.  I recommend that you read the “Getting Started with Django Tutorial” on Heroku.  While many of the concepts apply the command are obviously different.

Bluemix provide some great boiler plates for the most common platform as a service stack and continues to add new ones everyday.  Unfortunately none of those are for Python or for Django.  You can however still run Python apps like I showed in my Hello World Python Bluemix post.

Python utilizes a utility called PIP to install modules for your application.  You can install Django utilizing PIP.

Install pip

http://pypi.python.org/pypi/pip

Follow directions here:
https://pip.pypa.io/en/latest/installing.html

On Windows

$python get-pip.py

Determine your application requirements

Locally install python package requirements for your application.

$python pip install django

Locally install python package requirements using my requirements.txt file below.

$python pip install –r requirements.txt

Or get install requirements for your application.

$python pip freeze –r > requirements.txt

Here is the requirements.txt file for the Django application I deployed to Bluemix. It includes the module and version.

Django==1.6.2
amqp==1.4.5
anyjson==0.3.3
autoenv==1.0.0
billiard==3.3.0.17
celery==3.1.11
dj-database-url==0.3.0
dj-static==0.0.5
django-celery==3.1.10
django-pagination==1.0.7
django-registration==1.0
django-taggit==0.11.2
django-toolbelt==0.0.1
gunicorn==18.0
httplib2==0.8
kombu==3.0.15
oauth2==1.5.211
oauthlib==0.6.1
pika==0.9.13
psycopg2==2.5.2
pysolr==3.2.0
pystache==0.5.4
python-openid==2.2.5
python-social-auth==0.1.24
python-twitter==1.3.1
pytz==2014.3
requests==2.3.0
requests-oauthlib==0.4.0
simplejson==3.5.2
six==1.6.1
static==1.0.2

The bare minimum you need are:

  • Django – To install Django
  • dj-database-url – To read the environment variable with the database name host, port, user id and password
  • gunicorn – A WSGI server o serve HTTP content including your Django application
  • psycopg2 – To allow your Django application to make Python Database calls to your Postgres Database Community Edition service on Bluemix.
  • dj-static – To serve static content

Start a Django app locally

Create an empty top-level directory for our project:

$ mkdir hellodjango && cd hellodjango
$ django-admin.py startproject hellodjango .
$ manage.py startapp hello

Create a Profile file

Use a Procfile, a text file in the root directory of your application, to explicitly declare what command should be executed to start the Bluemix Instannces. Start the Gunicorn with a few arguments.

Here is the Procfile that I used for my app. It should be called Procfile and live at the root directory of our project:

Procfile

web: gunicorn hellodjango.wsgi --workers 3

Django settings

Next, configure the application for the Bluemix environment, including PostgreSQL database. The dj-database-url module will parse the value of the DATABASE_URL environment variable and convert them to something Django can understand.

Make sure ‘dj-database-url’ is in your requirements file, then add the following to the bottom of your settings.py file:

settings.py

# Parse database configuration from $DATABASE_URL
import dj_database_url
DATABASES['default'] = dj_database_url.config() 
# Honor the 'X-Forwarded-Proto' header for request.is_secure()
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO','https') 
# Allow all host headers
ALLOWED_HOSTS = ['*'] 
# Static asset configuration 
import os
BASE_DIR = os.path.dirname(os.path.abspath(__file__))
STATIC_ROOT ='staticfiles'
STATIC_URL ='/static/' 
STATICFILES_DIRS = (
   os.path.join(BASE_DIR,'static'),
)

With these settings available, you can add the following code to wsgi.py to serve static files in production:

wsgi.py

fromdjango.core.wsgiimportget_wsgi_applicationfromdj_staticimportCling application = Cling(get_wsgi_application())

Set database URL environment variable.

You can see whether or not you are getting your Database settings picked up by looking at the settings on error page on Django.

DATABASES
{'default': {'ATOMIC_REQUESTS': False,
'AUTOCOMMIT': True,
'CONN_MAX_AGE': 0,
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'HOST': '192.155.243.12',
'NAME': 'd4f5c4d18346a40a7a043cc5985fc2466',
'OPTIONS': {},
'PASSWORD': u'********************',
'PORT': 5433,
'TEST_CHARSET': None,
'TEST_COLLATION': None,
'TEST_MIRROR': None,
'TEST_NAME': None,
'TIME_ZONE': 'UTC',
'USER': 'ucba74fb890ea4d999c31a90dd2f84b03'}}

If you haven’t run python manage.py syncdb you will likely get this error

ProgrammingError at /yourURLContext/
relation "django_session" does not existLINE 1: SELECT (1) AS "a" FROM "django_session" WHERE "django_sessio...

Run test locally

Simple if it doesn’t work locally don’t expect it to work on Bluemix. Look out for python packages you installed locally using exe.  For example I wasn’t able to get LXML to install on Bluemix because of this reason.

Deploy to Bluemix

There are many buildpacks available out there. The challenge is finding one that works with Django and Bluemix. I was able to use the one below to get my Django application to deploy and run on Bluemix.

$cf push hellodjango -m 256M -b https://github.com/ephoning/heroku-buildpack-python.git

1. First create database service and push web app without the db setup command. It deploys successfully.
$cf push app_name -m 256M -b https://github.com/ephoning/heroku-buildpack-python.git

2. Interactively in the Bluemix Web UI bind your app to your DB Service.

3. Now restart the app with command to populate DB. Now that you have a connection to it. Instead stop the app and then redeploy using the cf push command and this time append the -c “setup_db.sh” option.

$cf push app_name -m 256M -b https://github.com/ephoning/heroku-buildpack-python.git -c "bash ./setup_db.sh"

Where setup_db.sh looks like:

#!/bin/bash
echo "Trying----------------------------- to synch"
python manage.py syncdb --noinput
echo "Finish manage.py synchdb ----------------------------"
echo "from django.contrib.auth.models import User; User.objects.create_superuser('carlos', 'carlos.ferreira@us.ibm.com', 'carlos')" | python manage.py shell
echo "Finish creating super user ----------------------------"

4. Now restart the app with command to run.sh without create users cf push command and this time append the option:

$cf push app_name -m 256M -b https://github.com/ephoning/heroku-buildpack-python.git -c “bash ./run.sh"

Where run.sh looks like:

#!/bin/bash
if [ -z "$VCAP_APP_PORT" ];
then SERVER_PORT=5000;
else SERVER_PORT="$VCAP_APP_PORT";
fi
echo port is------------------- $SERVER_PORT
python manage.py syncdb --noinput

Troubleshooting

Review the logs

$cf logs hellodjango

The above steps worked. However I initially was unable to synch my postgres database with the tables required by Django. Here are some of the options I tried to do to be able to get my database populated for Django to work:

Tried invoking a custom script like was done in this example.

Tried passing in the syncdb command using the – c option as part of my push.

$cf push hellodjango -c “python manage.py syncdb”

Review your logs to ensure your environment variables are set properly

$ cf files hellodjango /logs/env.log

Gotcha’s

If you install the wrong Psycopg2 like I did you likely will want to uninstall it. Make sure you get the right 32 bit version for your Operating System. Here is how remove python packages that were installed with easy_install which is another install utility useful for installing Pip but also exe packages like the one for psycopg2.

http://stackoverflow.com/questions/1231688/how-do-i-remove-packages-installed-with-pythons-easy-install

To uninstall an .egg you need to rm -rf the egg (it might be a directory) and remove the matching line from site-packages/easy-install.pth

Thanks for reading an appreciate comments or suggestions on my final hurdle getting this to work with Postgres database.

Aside

Hello World on Bluemix With Python

Getting Started

Get an account

Access bluemix Web site here: https://ace.ng.bluemix.net/
Click on the “Sign Up” for Bluemix button.
Read quick start  http://www.ng.bluemix.net/docs/QuickStart.jsp

Installing the cf tool

Before you begin, install the command line tool that is used to upload and manage your application. BlueMix uses a tool called cf command line interface.
Download cf command line interface from https://github.com/cloudfoundry/cli
The cf command is a program that you need to install. Select the appropriate installer for your operating system. After the cf command tool is installed, you can use cf -v command to verify its version as follows:

$cf -vcf version 6.0.0-90db382

Login using the cf tool

Login. Be sure to use the right endpoint. To see the end point type:

$cf –t
API endpoint: https://api.ng.bluemix.net (API version: 2.2.0)Not logged in. Use 'cf login' to log in.
$cf login

I had to reset mine to the beta site.

$ cf api https://api.stage1.ng.bluemix.net

Important files

Procfile is where the command is for launching your app

Web: python hello.py

Hello.py s the contents of your hello world app

import os
from flask import Flask
app = Flask(__name__)
@app.route('/')
def hello():
    return 'Hello World!'
    port = os.getenv('VCAP_APP_PORT', '5000')
if __name__ == "__main__":
    app.run(host='0.0.0.0', port=int(port))

Get help getting answers to your questions

Go to IBM.com forums

Prepare the environment for your chosen runtime

Python is my language of choice. Bluemix doesn’t have Python as a pre-existing runtime. You can however create your own runtime by using buildpacks that create the environment for you.   In this hello world example we will use Flask as our HTTP server.
requirements.txt file describes the application prequisites.

Flask==0.10.1

You can push your application to Bluemix along with a Python buildpack:
You can find Python buildpacks on Github. https://github.com/ephoning/heroku-buildpack-python From this page you can click on the Download Zip file on the right column.
Unzip the file into a local directory. Extract the zip into directory named hello-python-master in the folder where cf tool was installed in.

c:\CloudFoundry\hello-python-master

Push App to Bluemix

cf push <YOUR_APP_NAME> -m 128M -b https://github.com/ephoning/heroku-buildpack-python.git
$ c:\CloudFoundry\hello-python-master>cf push hello-python-master -m 128M -b https://github.com/ephoning/heroku-buildpack-python.git

Check the logs in the command window for errors. You likely might get some because you aren’t executing the commands in the right folder. Make sure you invoke the command where the requirements.txt file is for the helloworld.py file. If successful you should see the output below.

…-----> Uploading droplet (26M) 0 of 1 instances running, 1 starting1 of 1 instances running 
App started Showing health and status for app hello-python-master in org carlos.ferreira@ youremail.com / space dev as carlos.ferreira@youremail.com...
OK requested state: startedinstances: 1/1usage: 128M x 1 instancesurls: hello-python-master.stage1.ng.bluemix.net      
state     since                   cpu   memory           disk#0   
running   2014-05-08 09:51:17 PM   0.0%   109.4M of 128M   99M of 1G


Try your App in the browser

Open a browser and type the name of the app pre-pended to your Bluemix URL.
http://hello-python-master.stage1.ng.bluemix.net
My app is hello-python-master
My Bluemix URL is stage1.ng.bluemix.net
Figure 1 Hello World
hello
Check the logs in the command line window
Check your dashboard on bluemix that the app is running

Figure 2. 1 Bluemix Dashboard
helloRunning
Took me about 1.5 hours to get the app running.   This was much faster than my hello world experience with Heroku.

First Step in a Journey

Python is my programming language of choice.  I recently have deployed Python Apps using Django on Heroku and wanted to see what the experiences would be like to do the same with IBM Bluemix.   I plan on blogging my adventures as I go along.   I hope to share with you what I learn so that other will be able to quickly overcome any challenges that I encounter.  Hopefully you can leave comments and suggestions along the way too.  The views here are completely my own and not my employer. I do this on my own time because I enjoy problem solving and discovering new technology and enabling others on it!!

Learn more about Bluemix on there Docs

At the time of the post Bluemix doesn’t support Python runtime out of the box but you can get Python apps to run on Bluemix.  The first app I build will be based on a Hello World example using Python. Read my next post to learn how to do it.