Verifying the DNS Plugin using Rails Console

Each of the OpenShift broker plugins provides an interface implementation class for the  plugin's abstract behavior.   In practical terms this means that I can fire up the rails console, create an instance of the plugin class and then use it to manipulate the service behind the plugin.

Since the DNS plugin has the simplest interface and Bind has the cleanest service logs, I'm going to demonstrate with that.  The technique is applicable to the other back-end plugin services.

Preparing Logging

To make life easy I'm going to configure logging on the DNS server host so that the logs from the named service are written to their own file.

A one line file in /etc/rsyslog.d will do the trick:

if $programname == 'named' then /var/log/named.log

Write that into /etc/rsyslog.d/00_named.conf and restart the rsyslog service.  Then restart the named service and check that the logs are appearing in the right place.

If I didn't filter out the named logs, I could still use grep on /var/log/messages to extract them.

Configuring the Bind DNS Plugin

As indicated in previous posts, the OpenShift DNS plugin is enabled by placing a file in /etc/openshift/plugins.d with the configuration information for the plugin.  The name of the file must be the name of the rubygem which implements the plugin with the suffix .conf. The Bind plugin is configured like this:

/etc/openshift/plugins.d/openshift-origin-dns-bind.conf



BIND_SERVER="192.168.5.11"
BIND_PORT=53
BIND_KEYNAME="app.example.com"
BIND_KEYVALUE="put-your-hmac-md5-key-here"
BIND_ZONE="app.example.com"

When the Rails application starts, it will import a plugin module for each .conf file and will set the config file values.

The Rails Console

Ruby on Rails has an interactive testing environment.  It it started by invoking rails console from the root directory of the application.  If I start the rails console at the top of the broker application I should be able to instantiate and work with the plugin objects.

The rails console command runs irb to offer a means of manual testing.  In addition to the ordinary ruby script environment it imports the Rails application environment which resides in the current working directory. Among other things, it processes the Gemfile which, in the case of the OpenShift broker, will load any plugin gems and initialize them. I'm going to use the Rails console to directly poke at the back end service objects.

I'm going to go to the broker application directory.  Then I'll check that bundler confirms the presence of all of the required gems.  Then I'll start the Rails console and check the plugin objects manually.

cd /var/www/openshift/broker
bundle --local
....
Your bundle is complete! Use `bundle show [gemname]` to see where a bundled gem is installed.
rails console
Loading production environment (Rails 3.0.13)
irb(main):001:0> 

The last line above is the Rails console prompt.

Creating a DnsService Object

The OpenShift::DnsService class is a factory class for the DNS plugin modules. It also contains an interface definition for the plugin, though Ruby and Rails don't seem to be much into formal interface specification and implementation.  The plugin interface definitions reside in the openshift-origin-controller rubygem:

https://github.com/openshift/origin-server/tree/master/controller/lib/openshift

The factory classes provide two methods by convention: provider=() sets the actual class which implements the required interface and instance() is the factory method, returning an instance of the implementing class.  They also have a private instance variable which will contain a reference to the instantiating class.  When the plugins are loaded, a reference to the instantiating class is set into the factory class.

Once the broker application is loaded using the rails console I should be able to create and work with instances of the DnsService implementation.

The first step is to check that the factory class is indeed loaded and has the right provider set. Since I can just type at the irb prompt it's easy to see what's there.

irb(main):001:0> OpenShift::DnsService
=> OpenShift::DnsService
irb(main):002:0> d = OpenShift::DnsService.instance
=> #<OpenShift::BindPlugin:0x7f540dfb9ee8 @zone="app.example.com",
 @src_port=0, @server="192.168.5.2",
 @keyvalue="GwhJNLZPghbpTya2M6N+lvcLmBQx6TYbuH7j6TPyetE=",
 @port=53, @keyname="app.example.com",
 @domain_suffix="app.example.com">

Note that the class is OpenShift::BindPlugin and the instance variables match the values I set in the plugin configuration file. I now have a variable d which refers to an instance of the DNS plugin class.

The DnsService Interface

The DNS plugin interface is the simplest of the plugins.  It contains just four methods:

• register_application(app_name, namespace, public_hostname)
• deregister_application(app_name, namespace)
• modify_application(app_name, namespace, public_hostname)
• publish()

All but the last will have a side-effect which I can check by observing the named service logs and by querying the DNS service itself.

Note that the publish() method is not included in the list with side-effects.  publish() is always called at the end of a set of change calls.  It is there to accommodate batch update processing.  Third party DNS services which use use web interfaces may require batch processing. The OpenShift::BindPlugin submits changes instantly.

Change and Check

The process of testing now will consist of three repeated steps:

1. Make a change
2. Check the DNS server logs
3. Check the DNS server response

I will repeat the steps once for each method. (though I'll only show a couple of samples here)

The logs are time-stamped.  To make it easier to find the right log entry, I'll check the time sync of the broker and DNS server hosts, and then check the time just before issuing each update command.

First I check the date and add an application record.  An application record is a DNS CNAME record which is an alias for the node which contains the application. Here goes:

irb(main):003:0> `date`
=> "Wed Dec  5 15:25:59 GMT 2012\n"
irb(main):002:0> d.register_application "testapp1", "testns1", "node1.example.com"
=> ;; Answer received from 192.168.5.11 (129 bytes)
;;
;; Security Level : UNCHECKED
;; HEADER SECTION
;; id = 25286
;; qr = true    opcode = Update    rcode = NOERROR
;; zocount = 1  prcount = 0  upcount = 0  adcount = 1

OPT pseudo-record : payloadsize 4096, xrcode 0, version 0, flags 32768

;; ZONE SECTION (1  record)
;; app.example.com. IN SOA

The register_application() method returns the Dnsruby::Message returned from the DNS server.  A little digging should indicate that the update was successful.

Next I'll examine the named service log on the DNS server host.

tail /var/log/named.log
...
Dec  5 15:26:41 ns1 named[11178]: client 10.16.137.216#54040/key app.example.com: signer "app.example.com" approved
Dec  5 15:26:41 ns1 named[11178]: client 10.16.137.216#54040/key app.example.com: updating zone 'app.example.com/IN': adding an RR at 'testapp1-testns1.app.example.com' CNAME

Finally, I'll check that the server is answering queries for that name:

dig @ns1.example.com testapp1-testns1.app.example.com CNAME

; <<>> DiG 9.9.2-rl.028.23-P1-RedHat-9.9.2-8.P1.fc18 <<>> @ns1.example.com testapp1-testns1.example.com CNAME
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 10884
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;testapp1-testns1.example.com. IN CNAME

;; AUTHORITY SECTION:
example.com.  10 IN SOA ns1.example.com. hostmaster.example.com. 2011112904 60 15 1800 10

;; Query time: 3 msec
;; SERVER: 192.168.1.11#53(192.168.1.11)
;; WHEN: Thu Dec  5 15:28:41
;; MSG SIZE  rcvd: 108

That's sufficient to confirm that the DNS Bind plugin configuration is correct and that updates are working. In a real case I'd go on and check each of the operations. for Now I'll just delete the test record and go on.

d.deregister_application "testapp1", "testns1"
=> ;; Answer received from 192.168.5.11 (129 bytes)
;;
;; Security Level : UNCHECKED
;; HEADER SECTION
;; id = 26362
;; qr = true    opcode = Update    rcode = NOERROR
;; zocount = 1  prcount = 0  upcount = 0  adcount = 1

OPT pseudo-record : payloadsize 4096, xrcode 0, version 0, flags 32768

;; ZONE SECTION (1  record)
;; app.example.com. IN SOA

dig @ns1.example.com testapp1-testns1.app.example.com CNAME
; <<>> DiG 9.9.2-rl.028.23-P1-RedHat-9.9.2-8.P1.fc18 <<>> @ns1.example.com testapp1-testns1.example.com CNAME
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 50598
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;testapp1-testns1.example.com. IN CNAME

;; AUTHORITY SECTION:
example.com.  10 IN SOA ns1.example.com. hostmaster.example.com. 2011112904 60 15 1800 10

;; Query time: 3 msec
;; SERVER: 192.168.5.11#53(192.168.5.11)
;; WHEN: Thu Dec  5 15:31:20
;; MSG SIZE  rcvd: 108

This is what a negative response looks like. There's a question section but no answer section.
Things are back where I started and I can move on to the next test.

Resources

• Rails console
• irb - Interactive Ruby

• OpenShift DNS plugin class/interface
• OpenShift Bind DNS plugin class/implementation

• dig(1) man page
• rsyslog home page
• rsyslogd man page