Tuesday, October 11, 2016

Python and Mongo: Up and Running with docker-compose


This tutorial walks through
  • Usage of docker-compose (v2 syntax) with the data volume persisted on the host (e.g. your laptop). 
  •  A python project created using cookiecutter 
    • and the Python/Mongo driver specified in the requirements.txt file for controlled and repeatable installation
  • Inserting and Querying data to and from Mongo
    • also terminating and re-starting the docker instance to demonstrate persistence beyond the container lifecycle


At the time of this article, I'm using
  • OS X 10.11.6
  • Docker version 1.12.2-rc1-beta27 (build: 12496) 
  • Python 2.7.12 |Anaconda 4.1.1 (x86_64)| (default, Jul  2 2016, 17:43:17)

Mongo in Docker

Copy this into docker-compose.yml
version: '2'
    image: mongo
      - "27017:27017"
      - /Users/craigtrim/docker/volumes/mongobot:/data/db
Note that the use of docker-compose and a standard mongo image.  We do not have to create/edit an existing Dockerfile.  It's best to use trusted images.  Also note the path for /volumes. 

Copy this into run.sh
#!/usr/bin/env bash

# STEP 1: clean up past docker images
docker stop $(docker ps -a -q)
docker rm $(docker ps -a -q)
docker rmi $(docker images -q -f "dangling=true")
docker rm $(docker ps -q -f status=exited)

# STEP 2 (optional): deletes all docker images
if [ "$1" == "-c" ]; then
 docker rmi $(docker images -q)

# STEP 3: build and run
docker-compose build
docker-compose up --remove-orphans --force-recreate

We will use this script to gracefully launch docker-compose.

The Python Project

I prefer to use cookiecutter to create new Python projects.
cookiecutter https://github.com/wdm0006/cookiecutter-pipproject

For developers with a JEE background, this is similar to using the Maven Archetype Generator (shown here for reference):
mvn archetype:generate -DgroupId=$1 -DartifactId=$2 -DarchetypeArtifactId=maven-archetype-quickstart -DinteractiveMode=false

After stepping through the cookiecutter questionnaire, I replace the contents of "requirements.txt" with

I use this command to install the python project:
python setup.py install clean --all

The Python Code

 The python code looks like this:

if __name__ == "__main__":
    from pymongo import MongoClient

    client = MongoClient("mongodb://localhost:27017")

    db = client.primer

    from datetime import datetime

    result = db.restaurants.insert_one(
            "address": {
                "street": "2 Avenue",
                "zipcode": "10075",
                "building": "1480",
                "coord": [-73.9557413, 40.7720266]
            "borough": "Manhattan",
            "cuisine": "Italian",
            "grades": [
                    "date": datetime.strptime("2014-10-01", "%Y-%m-%d"),
                    "grade": "A",
                    "score": 11
                    "date": datetime.strptime("2014-01-16", "%Y-%m-%d"),
                    "grade": "B",
                    "score": 17
            "name": "Vella",
            "restaurant_id": "41704620"

    print (result.inserted_id)

    cursor = db.restaurants.find()
    for document in cursor:

The docker instance can be terminated.  When launched again, the data inserted in the first session will still be present.