Tuesday, November 27, 2012

Openshift Back End Services: DNS - An Authoritative Server

OpenShift Origin dynamic DNS requires an authoritative zone for the application records.  So the first step to creating the DNS back end service is to establish that authoritative zone.  When I'm sure it will serve records then I'll add the dynamic update feature to it.

There are a number of good resources on setting up DNS on Linux.  I'll post a few links to them in the Resources.  I've even written part of one, but each time I do this I see it a little differently and (I think) improve my understanding.  So I'm going to do it again.


I'm going to try to create a proper (ready to be delegated) zone.

Here's the list of information I'll need to set it up:

DNS Service Configuration Variables
Primary Nameserver
IP Address192.168.5.2
Hostnamens1.example.comNot in the app domain
Secondary Nameserver
IP Address192.168.5.3
Hostnamens2.example.comNot in the app domain
Update Configuration
Application Zoneapp.example.comNot the top level
Application Zone Key Nameapp.example.comArbitrary name
Application Zone Key TypeHMAC-MD5
Application Zone Key Size64 bits512 bits in real life
Application Zone Key ValueA Base64 encoded stringGenerated by dnssec-keygen

I'll be setting up both a primary and secondary server for the zone.  The update configuration information won't be needed here except to establish a static zone that will be made dynamic later.

The operations listed below will all be performed on ns1.example.com.

Bind on Linux

Installing ISC Bind 9 on most modern Linux distributions is pretty easy. It's an old and well worn tool.  For RPM based distributions you can just use yum. I generally install the bind-utils package as well just so they're handy.
yum install bind bind-utils

The DNS service binary is called named.  When I refer to Bind, I'll be talking about the software in general. When I refer to named I'll be referring to the daemon and its configuration files.

Once the package is installed there are a number of small configuration changes that are needed to enable and verify the caching service.

  • Enable listener ports
  • Enable rndc
  • Enable start on boot
  • Start the named service
  • Confirm caching service
  • Confirm rndc controls

Named Configuration and Management

The named service follows a traditional layout model on Linux (It's actually probably one of the canonical services).  Before I start making changes I want to

Named Configuration Files

The primary service configuration file is /etc/named.conf.  Additional configuration information and service data reside in the /var/named directory.  The named daemon logs to /var/log/messages through syslog.
  • /etc/named.conf - primary configuration file
  • /var/named - additional configuration and data
  • /var/log/messages - service logging

You can filter for named related log messages in /var/log/messages by grepping for (wait for it!) "named".

ISC provides a complete reference of the named configuration options on their site.

The named service adds one non-traditional feature.  It has a tool called rndc which might stand for "remote name daemon controller".  The rndc tool use used locally by the service command to provide status for the named service.  It can also be used to adjust the logging level, to dump the current zones to a file, or to force a zone reload  without restarting the daemon.

rndc does require some additional setup.  It is not enabled by default as it requires an authentication key.  A default key would present a security risk.  The bind package provides rndc-confgen to help set up the rndc access to the local named. This command produces a required key file which will reside at /etc/rndc.key.

  • /etc/rndc.key - Remote name daemon control access key

rndc also requires an addition to the /etc/named.conf file. I'll do that just before I try starting the service.

Before making any change to a configuration file I copy the original and name the new file <filename>.orig. For example, /etc/named.conf would become /etc/named.conf.orig. This way I can track my changes and revert them to the initial values if I need to.

Enable Listeners

The initial configuration of /etc/named.conf restricts queries and updates to the local IP interfaces (IPv4 and IPv6 ::1).  Since I want to allow basically anyone to find out about my zone, I have to open this up.  There are three configuration lines that control listeners and query access:

  • listen-on
  • listen-on-v6
  • allow-query

Each of these options takes a semi-colon (;)delimited list of listeners.  The list is encapsulated in a paired set of curly-braces ( {} ). Since this will be a public service, I just have to replace the appropriate localhost address entries with the keyword "any".


options {
 listen-on port 53 { any ; };
 listen-on-v6 port 53 { any; };
 directory  "/var/named";
 dump-file  "/var/named/data/cache_dump.db";
        statistics-file "/var/named/data/named_stats.txt";
        memstatistics-file "/var/named/data/named_mem_stats.txt";
 allow-query     { any; };
 recursion yes;

If you want to be tricksy a sed one-liner (and a safety copy) will do the trick if you're starting from the default:

cp /etc/named.conf /etc/named.conf.orig
sed -i -e 's/\|::1\|localhost/any/' /etc/named.conf

Enable rndc

rndc is the Remote Name Daemon Control program. It's now used for all communications and control to the named on a bind server host. rndc does allow you to securely control a name server daemon remotely, but we won't be using it that way. rndc is also the program that gets and reports the status information when you run service named status so it's nice to have it configured.

rndc comes with a nice little configuration tool to help: rndc-confgen. rndc-confgen creates a unique rndc access key and places the rncd configuration in /etc/rndc.key for you.

rndc-keygen -a

This creates a new file named /etc/rndc.key which contains the access key for the named process. Both the named and the rndc command must have access to this key. rndc uses the /etc/rndc.key file by default. named must be configured for it.

If the rndc-keygen command hangs it is because there is not enough entropy (randomness) on the system. I could Log onto another window and type random commands for a bit and it would complete. If I'm impatient I could run it with -r /dev/urandom which will always complete, but which may be less secure because it will not block waiting for enough randomness to generate a good key. I often do that for lab systems.

rndc-keygen does not set the SELinux context for the key file.  It also does not set the ownership and permissions so that the named can read it. restorecon will do it for me though.

restorecon -v /etc/rndc.key
chown root:named /etc/rndc.key
chmod 640 /etc/rndc.key

Now that we have a key, we have to tell the named to use it. Append the section below to the bottom of the /etc/named.conf file.

// enable service controls via rndc
// use the default rndc key
include "/etc/rndc.key";

controls {
        inet port 953
        allow {; } keys { "rndc-key"; };

Verifying a caching DNS server

With this configuration I have a caching DNS server.  I need to check the operation now before going ahead to add a new zone.

I use the expected tools to start the named service and get status:

service named start
Starting named:                                            [  OK  ]

service named status
version: 9.8.2rc1-RedHat-9.8.2-0.10.rc1.el6_3.5
CPUs found: 16
worker threads: 16
number of zones: 19
debug level: 0
xfers running: 0
xfers deferred: 0
soa queries in progress: 0
query logging is OFF
recursive clients: 0/0/1000
tcp clients: 0/100
server is up and running
named (pid  31010) is running...

This indicates that the server started and that it is responding to rndc queries. If the daemon does not start or if I got errors from the status query, I'd check /var/log/messages for error messages.

Next we want to verify that the server is actually answering queries. I use dig or host to check. host has a simpler interface and output but for this I want the verbosity of dig to help diagnose if there are any problems. I have to try from two different locations: localhost and "somewhere else". I want first to verify that the service is running and answering, and second that it is accessable from outside itself. I'll only show the localhost queries, but the remote ones are identical.

dig @ www.example.com

; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.10.rc1.el6_3.5 <<>> @ www.example.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 29408
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 4

;www.example.com.  IN A

www.example.com. 172789 IN A

example.com.  172788 IN NS a.iana-servers.net.
example.com.  172788 IN NS b.iana-servers.net.

a.iana-servers.net. 172788 IN A
a.iana-servers.net. 172788 IN AAAA 2001:500:8c::53
b.iana-servers.net. 172788 IN A
b.iana-servers.net. 172788 IN AAAA 2001:500:8d::53

;; Query time: 0 msec
;; WHEN: Wed Nov 21 20:42:21 2012
;; MSG SIZE  rcvd: 185

This is actually the real right answer.  Since RFC 2606 reserves the example.com domain, the IANA also  serves that domain.  When I build my test server I'll override that.  If I were building a real OpenShift Origin service I'd get a properly delegated sub-domain of my organization or get my own domain from a registrar.

Take a moment to look at that output. It's meaningful. The ANSWER SECTION contains the only response, an A record. The AUTHORITY SECTION lists the servers which are the designated sources for the content in the example.com zone. It contains two NS records. NS record values are fully qualified domain names (FQDN). That means those names don't have some implied suffix. They end with a dot (.) which anchors them to the root of the DNS. The ADDITIONAL SECTION provides IP address resolution for the NS record FQDNs. The last section indicates that the answer came from the IPv4 localhost address and gives the date/time stamp.

So this answer tells you not only that the IP address for www.example.com is but where the answer came from.

I'll do that again from some other host and set the server address to the public IP address of my nameserver host.

Zone Configuration

Now that I have a caching server running, it's time to start adding some content. The /etc/named.conf file syntax has a directive to include another file in line. Rather than putting the entire configuration section in the master configuration file, I'll put my configuration information in another file and include it. That just requires appending one line to /etc/named.conf

echo 'include "app.example.com.conf" ;' >> /etc/named.conf

Other than the /etc/named.conf file, all of the named configuration files reside in /var/named/. That's the default location for relative path names in the /etc/named.conf as well. I'll create a configuration file fragment there. Since I'm creating the app.example.com domain I'll call the file /var/named/app.example.com.conf.

zone "app.example.com" IN {
    type master;
    file "dynamic/app.example.com.db";

The directory option in the /etc/named.conf file determines the location of any files listed with relative path names. (see the fragment in the Enabling Listeners section)  The file directive above means that the absolute path to the zone file will be /var/named/dynamic/app.example.com.db

This will eventually be a dynamic zone.  That's why I'm putting it in "dynamic/app.example.com.db".  If it were to be a static zone I'd probably put it right at the top of the /var/named tree.

The Zone File

The last file I need to create is the zone file. This file defines the initial contents of the application zone. It also defines the NS (nameserver) records for the zone and the default TTL (time to live).


$TTL 1800 ; Default TTL: 30 Minutes
app.example.com. IN SOA ns1.example.com. hostmaster.example.com. (
                         2011112904 ; serial
                         60         ; refresh (1 minute)
                         15         ; retry (15 seconds)
                         1800       ; expire (30 minutes)
                         10         ; minimum (10 seconds)
                     NS ns1.example.com.
                     NS ns2.example.com.
;; prime the nameserver IP addresses for the app zone.
ns1.example.com.               A
ns2.example.com.               A

Verifying The App Zone

Once the configuration file and the zone database file are in place it's time to try restarting the named service. I use service named restart. and observe the results.

service named restart
Stopping named: .                     [ OK ]
Starting named:                       [ OK ]

Then I use grep named /var/log/messages to observe the typical start up messages. I look for a line indicating that the app.example.com zone has been loaded.

grep named /var/named/messages | grep loaded
Nov 22 17:40:10 ns1 named[3888]: zone 0.in-addr.arpa/IN: loaded serial 0
Nov 22 17:40:10 ns1 named[3888]: zone loaded serial 0
Nov 22 17:40:10 ns1 named[3888]: zone loaded serial 0
Nov 22 17:40:10 ns1 named[3888]: zone example.com/IN: loaded serial 2011112904
Nov 22 17:40:10 ns1 named[3888]: zone app.example.com/IN: loaded serial 2011112906
Nov 22 17:40:10 ns1 named[3888]: zone localhost.localdomain/IN: loaded serial 0
Nov 22 17:40:10 ns1 named[3888]: zone localhost/IN: loaded serial 0
Nov 22 17:40:10 ns1 named[3888]: managed-keys-zone ./IN: loaded serial 53

Now that I know that the zone has been loaded successfully, I'll check that it's served properly. I first request the SOA (Start of Authority) record, and then the full zone dump.

dig @localhost app.example.com soa
; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.10.rc1.el6_3.5 <<>> @localhost app.example.com soa
; (2 servers found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 3502
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 2

;app.example.com.  IN SOA

app.example.com. 30 IN SOA ns1.example.com. hostmaster.example.com. 2011112906 60 15 1800 10

app.example.com. 30 IN NS ns2.example.com.
app.example.com. 30 IN NS ns1.example.com.

ns1.example.com. 600 IN A
ns2.example.com. 600 IN A

;; Query time: 0 msec
;; SERVER: ::1#53(::1)
;; WHEN: Thu Nov 22 17:29:32 2012
;; MSG SIZE  rcvd: 148

Now test a complete zone dump, (and save it for comparison)

dig @ app.example.com axfr
; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.10.rc1.el6_3.5 <<>> @localhost app.example.com axfr
; (2 servers found)
;; global options: +cmd
app.example.com. 30 IN SOA ns1.example.com. hostmaster.example.com. 2011112906 60 15 1800 10
app.example.com. 30 IN NS ns1.example.com.
app.example.com. 30 IN NS ns2.example.com.
app.example.com. 30 IN SOA ns1.example.com. hostmaster.example.com. 2011112906 60 15 1800 10
;; Query time: 1 msec
;; SERVER: ::1#53(::1)
;; WHEN: Thu Nov 22 17:18:28 2012
;; XFR size: 4 records (messages 1, bytes 152)


At this point I have a working authoritative server for the app.example.com zone.  To get it properly delegated I need to create a secondary server and configure zone transfers.  Then I can provide the nameserver NS and A records and a contact to my IT department and they can complete the delegation.

For now I'm going to skip delegation. The next post will describe the configuration of dynamic updates


No comments:

Post a Comment