Hostess, a Tool for Local DNS

Published 2016-04-14

Hostess is a tool for tweaking local DNS by editing your /etc/hosts file. It's idempotent, which means it's great for scripting. And it's statically compiled, so it's really easy to install. Just download a binary and it works!

But first, let's talk a bit about why you need Hostess. I'm going to assume you already know generally what DNS is and have maybe looked at your hosts file before.

The Hosts File

The format of /etc/hosts (or C:\Windows\System32\drivers\etc\hosts on Windows) is relatively simple:

# This is a comment
192.168.0.3 this.is.an.alias

# 192.168.0.2 this.is.a.hostname.for.another.ip
192.168.0.1 this.is.a.basic.hostname this.is.an.alias
192.168.0.1 this.is.a.second.hostname # This is another comment

You can have comments at the start of a line or in the middle of a line. You can have blank lines. Each hosts entry starts with an IP address in either IPv4 or IPv6 format, and is followed by one or more hostnames. Technically each hostname after the first is called an "alias" but this is a semantic difference that Hostess ignores.

You may also notice that some of the hostnames are duplicated. This is not allowed and when the system parses the hosts file to lookup an IP, it ignores the duplicate entries. Which one ends up being used is not specified, so it's best to avoid dupes. Also, the specification (on Linux, at least) says that you should not have more than one entry for the same IP, but in practice you can usually get away with this.

Though the format is simple, reading and writing the hosts file is not trivial. For a human, it's hard to recognize duplicates (when a hostname is specified more than once), or conflicts (when a hostname points to two different IPs). And debugging it is not very easy either because errors in the hosts file are silently ignored and DNS tools you might use for debugging bypass the hosts file completely.

From a scripting perspective you have to figure out a way to deal with duplicates. Essentially, you need a human to resolve these or a set of rules for resolving conflicts automatically (e.g. first-in wins). Also, you need to deal with the multiple potential formats that you might encounter (1 hostname per line, 1 IP with multiple hostnames, etc.).

Formatting the Hosts File

The first thing Hostess does is format your hosts file deterministically. All of the entries are parsed, duplicates are reported or discarded (at your option), and the entire file is rewritten to conform to the spec. Entries are also sorted so they are easier for a human to find when looking at the file directly.

Let's start with our example hosts file above and see what it like Hostess formats it (the -n flag means "preview" or "no-op", similar to other tools):

$ hostess fix -n
Conflicting hostname entries for this.is.an.alias -> 192.168.0.1 and -> 192.168.0.3
192.168.0.1 this.is.a.basic.hostname this.is.a.second.hostname this.is.an.alias
# 192.168.0.2 this.is.a.hostname.for.another.ip

This tells us that this.is.an.alias was duplicated, and if we look at the output we can see the entry pointing to 192.168.0.3 was removed. If we're OK with this change we can apply it by running

$ hostess fix

From here, we can run hostess ls to see what we have in our hosts file:

$ hostess ls
this.is.a.basic.hostname          -> 192.168.0.1      (On)
this.is.a.second.hostname         -> 192.168.0.1      (On)
this.is.an.alias                  -> 192.168.0.1      (On)
this.is.a.hostname.for.another.ip -> 192.168.0.2      (Off)

Editing Entries

Hostess provides a handful of ways to change state in the hosts file. You can add, del, on (enable) and off (disable) entries. In addition to these commands there are a handful of other variants that are intended to facilitate scripting. Run hostess to see the full list.

add and del are the most immediately useful options. For instance, if you want to black hole traffic destined for ads.doubleclick.net you can do this:

$ hostess add ads.doubleclick.net 0.0.0.0
Added ads.doubleclick.net -> 0.0.0.0 (On)

If you change your mind and want to remove this entry, simply

$ hostess del ads.doubleclick.net
Deleted ads.doubleclick.net

The on and off commands will write the entry into a comment, but not completely remove it from your hosts file. This allows you to easily toggle DNS changes without having to repeat the IP address. This is useful for debugging, for example when you want DNS to resolve to a specific machine, or if you want to simulate an external service going offline.

Scripting

Hostess is idempotent. This means you can run the same command multiple times and it will always succeed (or fail) in the same way. For example, hostess add ads.doubleclick.net 0.0.0.0 will only add the entry once. If ads.doubleclick.net already pointing to 0.0.0.0 it does nothing! If it's pointing somewhere else Hostess just changes the IP. So you can drop Hostess commands into hooks, Makefiles, install scripts, etc. and they will just work.

If you need more sophisticated logic, check out the has, dump, and apply commands, which can help you automate in more ways. Check out jq for some combo ideas with JSON data sources.

Use Cases

Here are a few things you can do with Hostess. Be creative!

Hostnames for Virtual Machines or Containers

My most common use of Hostess is to point hostnames at VMs that I'm working on. For example, if you have a web application inside a VM you can add a line to your Makefile or Vagrantfile to add a hostname pointing to the VM when you start it up. For example:

hostess add mysite.com `vagrant ssh-config | grep HostName | awk '{print $NF}'`

Jess Fraz uses a similar technique to automatically add hostnames for containers on her laptop.

With this setup you can reference mysite.com in your browser, or in other scripts. If you combine this with open (on OS X) you can even open the site in your browser automatically, which makes for a very slick onboarding experience for teammates or contributors.

Debugging Production Systems

Hostess is also great for debugging problems in running systems. When you have a problem in prodution you can use Hostess to override the production DNS name (only on your computer!) and point it to a local VM or to one server that you have removed from the load balancer.

This is especially useful for debugging HTTPS problems (which require the hostname to match in your browser), redirects, and other issues that trigger same origin policy (cookies, AJAX calls, etc.).

Forcing DNS Resolution

I have sometimes used Hostess in production systems to ensure that certain hostnames always resolve to specific IPs when DNS is flaky or not available. This is somewhat brittle, but much less brittle than hard-coding IP addresses in configuration files! A real DNS resolver like BIND or Consul would be better, but sometimes these aren't available.

Failure Testing

Applications fail in all sorts of spectacular ways when the database goes down, or when an external service goes offline (youtube.com, github.com, etc.). Testing these scenarios can be hard. You certainly don't want to take your production databases offline just to test things, and you (hopefully) can't take youtube.com or github.com offline on a whim. But you can simulate these scenarios:

hostess add youtube.com 0.0.0.0
hostess add www.youtube.com 0.0.0.0

Or do something even more subtle by pointing your application to a secondary (read-only) database and see what happens.

hostess add primary.db.mysite.com 10.40.6.20  (the secondary database IP)

You can use iptables (or possibly configuration changes) to simulate these things but it's very easy to just change the DNS lookup. In fact, for robustness, I'd encourage you to test using all of these techniques. A DNS lookup failure is a different kind of error/exception than an HTTP timeout, so make sure your application is resilient and can handle all types of errors!

IPv6 Testing

Hostess supports manipulating IPv6 entries. If you are curious to see what happens when you try to interact with your hosts over IPv6, Hostess is a great way to try it out!

Limitations

Using Hostess has two (possibly significant) limitations.

  1. Hostess does not preserve comments in your hosts file, aside from hosts entries that are in the "off" state. Preserving comments is tricky. When Hostess merges entries together on the same line the context for the comments is lost, so I opted for simplicity and punted on this. My expectation is that being able to write scripts with Hostess obviates the need for comments.

  2. /etc/hosts does not support recursion or DNS zone lookups. For example, if you add an entry for example.com to your hosts file, foo.bar.baz.example.com will not resolve to that entry. This is actually a limitation of /etc/hosts and not a limitation in Hostess itself. If you want this behavior you will need to add both entries or run a real DNS server, like BIND, which supports the full DNS protocol.


Related

hostess tools go dns