Monday, September 1, 2014

Docker: A QPID Message Broker Container

OK I lied. I realized I can't just move on to working with Pulp in Kubernetes without building the other sub-service Pulp needs.

This one is merely going to be an exposition of the QPID container and it's actually simpler than the MongoDB container, so this will be a short one. A QPID service is even simpler than a MongoDB because (so long as you don't care about store-and-forward messages) you don't need persistent storage.

Like the MongoDB container, I need to define the package set that will be installed on top of the base image. I also need to declare a TCP port for the QPID service. Finally I need to define the primary process that will be started when the container starts.  This will be an invocation of the QPID service daemon.

QPID Dockerfile


Here's the Dockerfile for QPID on Fedora 20.


Let's walk through the Dockerfile directives quickly.

Line 1: FROM - Just as in the MongoDB image,  I'm using the stock Fedora 20 image as the base

Line 2: MAINTAINER - Indicate who to contact with problems (AND THANKS!)

Yeah, that's me.

Line 7: RUN - Install the QPID packages

I think there are several QPID servers.  I'm using the one written in C++, hence the package names: qpid-cpp-server and qpidd-cpp-server-store.  

Line 10: ADD - Create a location for the daemon to run.  If you specify a file to add but there is no matching file in the build context directory, then Docker will create the target in the container as an empty directory.

I'm creating /.qpidd for the daemon to run in.

Line 12: WORKDIR - Set the location where the initial process will run.  Here is where I tell Docker to run the daemon in the directory I created with the previous ADD directive.

Line 14: EXPOSE - QPID uses port 5672/TCP.  This line opens the firewall for that port and causes Docker to bind it to a host port.

Line 16: ENTRYPOINT - This indicates the binary or script that will be called when the container runs.

The ENTRYPOINT and CMD directives are used to craft the invocation of the primary process of the container.

Explaining ENTRYPOINT and CMD


I got some help for this from a Stackoverflow article: What is the difference between CMD and ENTRYPOINT

When I docker container is run, a single process is started inside the container.  This process may spawn others, but it remains as the anchor process for all of the others.

The invocation of the container primary process is created by combining the values of the ENTRYPOINT and CMD directives.  The ENTRYPOINT, if it is set, becomes the path of the binary to be executed.  The value of the CMD directive is used as the arguments to the primary process.

There are two twists on this.

If no ENTRYPOINT is provided, then the CMD directive is run using /bin/sh -c.
Also if the docker run command has any positional arguments following the regular docker arguments, these will replace the CMD value.

By setting the ENTRYPOINT to the QPID command, then the arguments to the daemon can be passed directly on the docker run line.

If an image has an ENTRYPOINT directive then it can be overridden with the --entrypoint option to docker run.

Building the Image


docker build -t markllama/qpid images/qpid
Sending build context to Docker daemon  2.56 kB
Sending build context to Docker daemon 
Step 0 : FROM fedora:20
 ---> 88b42ffd1f7c
Step 1 : MAINTAINER Mark Lamourine 
 ---> Using cache
 ---> 95516239225e
Step 2 : RUN yum install -y qpid-cpp-server qpid-cpp-server-store python-qpid-qmf python-qpid &&      yum clean all
 ---> Running in 7fc6b6ed2128
Resolving Dependencies
--> Running transaction check
---> Package python-qpid.noarch 0:0.26-2.fc20 will be installed
--> Processing Dependency: python-qpid-common = 0.26-2.fc20 for package: python-qpid-0.26-2.fc20.noarch
---> Package python-qpid-qmf.x86_64 0:0.26-2.fc20 will be installed
--> Processing Dependency: qpid-qmf(x86-64) = 0.26-2.fc20 for package: python-qpid-qmf-0.26-2.fc20.x86_64
--> Processing Dependency: libqmf2.so.1()(64bit) for package: python-qpid-qmf-0.26-2.fc20.x86_64
---> Package qpid-cpp-server.x86_64 0:0.26-11.fc20 will be installed
--> Processing Dependency: qpid(client)(x86-64) = 0.26 for package: qpid-cpp-server-0.26-11.fc20.x86_64
--> Processing Dependency: qpid-proton-c(x86-64) >= 0.5 for package: qpid-cpp-server-0.26-11.fc20.x86_64
...                                     
  python-qpid-common.noarch 0:0.26-2.fc20                                       
  qpid-cpp-client.x86_64 0:0.26-11.fc20                                         
  qpid-proton-c.x86_64 0:0.7-3.fc20                                             
  qpid-qmf.x86_64 0:0.26-2.fc20                                                 

Complete!
Cleaning repos: fedora updates
Cleaning up everything
 ---> d7e61654fb92
Removing intermediate container 7fc6b6ed2128
Step 3 : ADD . /.qpidd
 ---> 10c44a5719a5
Removing intermediate container a8a37c5986a5
Step 4 : WORKDIR /.qpidd
 ---> Running in 2833da1629d9
 ---> 1963a2551db8
Removing intermediate container 2833da1629d9
Step 5 : EXPOSE 5672
 ---> Running in d0d92a1e58ad
 ---> 425ba5994308
Removing intermediate container d0d92a1e58ad
Step 6 : ENTRYPOINT ["/usr/bin/qpidd", "-t", "--auth=no"]
 ---> Running in e678dc1a4b66
 ---> ae30e626e215
Removing intermediate container e678dc1a4b66
Successfully built ae30e626e215

Verifying the Image


With respect to docker, verifying the image is the same as it was for the MongoDB image.

docker run -d --name qpid1 --publish-all markllama/mongodb
1b513bee6d8d5d4328059a059f9520c469ff405228b88370b91bb85ef659b708


Process information


docker ps
CONTAINER ID        IMAGE                      COMMAND                CREATED             STATUS              PORTS                      NAMES
1b513bee6d8d        markllama/qpid:latest      /usr/sbin/qpidd -t -   7 seconds ago       Up 5 seconds        0.0.0.0:49157->5672/tcp    qpid1  

Docker logs


docker logs qpid1
2014-09-01 23:32:34 [Model] trace Mgmt create memory. id:amqp-broker
2014-09-01 23:32:34 [Broker] info Management enabled
2014-09-01 23:32:34 [Management] info ManagementAgent generated broker ID: dc7d2
473-58e4-4eea-a21b-46105345054e
...
2014-09-01 23:32:34 [Management] debug ManagementAgent added class org.apache.qp
id.broker:queueThresholdExceeded
2014-09-01 23:32:34 [Model] trace Mgmt create system. id:cfaf5a0f-1291-41e5-b0c0
-e5eb07c77c1e
2014-09-01 23:32:34 [Model] trace Mgmt create broker. id:amqp-broker
2014-09-01 23:32:34 [Model] trace Mgmt create vhost. id:org.apache.qpid.broker:b
roker:amqp-broker,/
2014-09-01 23:32:34 [Security] notice SSL plugin not enabled, you must set --ssl
-cert-db to enable it.
2014-09-01 23:32:34 [Broker] info Loaded protocol AMQP 1.0
2014-09-01 23:32:35 [Store] notice Journal "TplStore": Created
2014-09-01 23:32:35 [Store] notice Store module initialized; store-dir=//.qpidd
2014-09-01 23:32:35 [Store] info > Default files per journal: 8
2014-09-01 23:32:35 [Store] info > Default journal file size: 24 (wpgs)
2014-09-01 23:32:35 [Store] info > Default write cache page size: 32 (KiB)
2014-09-01 23:32:35 [Store] info > Default number of write cache pages: 32
2014-09-01 23:32:35 [Store] info > TPL files per journal: 8
2014-09-01 23:32:35 [Store] info > TPL journal file size: 24 (wpgs)
2014-09-01 23:32:35 [Store] info > TPL write cache page size: 4 (KiB)
2014-09-01 23:32:35 [Store] info > TPL number of write cache pages: 64
2014-09-01 23:32:35 [Model] trace Mgmt create exchange. id:
...
2014-09-01 23:32:36 [Model] trace Mgmt create exchange. id:qmf.default.direct
2014-09-01 23:32:36 [Broker] notice SASL disabled: No Authentication Performed
2014-09-01 23:32:36 [Security] info Policy file not specified. ACL Disabled, no 
ACL checking being done!
2014-09-01 23:32:36 [Security] trace Initialising SSL plugin
2014-09-01 23:32:36 [Network] info Listening to: 0.0.0.0:5672
2014-09-01 23:32:36 [Network] info Listening to: [::]:5672
2014-09-01 23:32:36 [Network] notice Listening on TCP/TCP6 port 5672
2014-09-01 23:32:36 [Store] info Enabling management instrumentation for the sto
re.
...
2014-09-01 23:32:36 [Model] trace Mgmt create store. id:org.apache.qpid.broker:b
roker:amqp-broker
2014-09-01 23:32:36 [Management] debug Management object (V1) added: org.apache.
qpid.legacystore:store:org.apache.qpid.broker:broker:amqp-broker
2014-09-01 23:32:36 [Broker] notice Broker running

The QPID logs will continue accumulating. With the default debug level it reports a lot of connection information.

Connectivity


To test connectivity to the QPID services I use the qpid-config command from the qpid-utils package on Fedora. Install that package to get the command.


qpid-config queues -b guest@127.0.0.1:49157
Queue Name                                Attributes
=================================================================
7783123e-9589-4814-8b7b-b976a576c853:0.0  auto-del excl 


This command lists the queues present on the broker.  It connects using the guest account and specifies the localhost IPv4 address and the port indicated by the output of the docker ps or docker ports commands.

This is a very simple connectivity test.  The single queue is the default for an unused AMQP server.  Once the Pulp components connect they will create additional queues.

Running A Shell in an image with an ENTRYPOINT


Using an ENTRYPOINT directive has a couple of effects that you want to be aware of.

On the plus side you can add arguments to the entrypoint binary just by adding them after the image name on the invocation.

One gotcha is that you can't just put /bin/sh after the image to get a shell as you otherwise would. It is very common and convenient to examine an image by running it with a shell, overriding the CMD. Docker provides the --entrypoint option to allow overriding when necessary.

docker run -it --entrypoint /bin/sh markllama/qpid 
sh-4.2# pwd
/.qpidd
sh-4.2# ls
Dockerfile
sh-4.2# exit
exit

Now I have images for both of the secondary services that Pulp needs.

Time to start playing with Kubernetes a bit.

References:


No comments:

Post a Comment