At the end of my last entry, I hinted at some network restructuring I'd done in the fallout of Yahoo deciding to take its network and go home. To be more specific I was just putting the last finishing touches on what was already (I think) a cool use (or abuse, I'm sure some would claim) of the DNS network. While my setup has evolved a bit over time, I am going to explain its current, theoretically final incarnation in the hopes that it proves useful to others who wish to go nuts with DNS in order to run fancy servers on a dynamic IP address.
DNS in a Nutshell
A brief introduction to DNS is, I suppose, useful for those not familiar with it. If you are already familiar with DNS you can skip this section, or else read it so that you can poke fun at my attempts at describing it.
DNS stands for Domain Name System, and it is the network of servers around the world that is responsible for turning human-readable names such as "www.garfieldtech.com" into computer-readable IP addresses like such as 126.96.36.199. Like most of the core protocols of the Internet, it is standardized by the Internet Engineering Task Force (IETF) through a series of RFCs (or "Request for Comments", a rather silly name since by the time an RFC is published the time for comments has long-since passed). The earliest reference I could find to DNS (although it is not referred to by that name explicitly) is RFC 0799 dating from 1981. It's not a very normative document, and in fact ends with the the amusing phrase "This memorandum is intended to stimulate discussion, not simulate it. " Many RFCs then established various editions of the DNS concept, although I believe RFC 1034 (description of the system) and RFC 1035 (description of the protocol), both dating from 1987, are the ones currenty in use (mostly). Dozens of later RFCs edited and tweaked and supplemented the system. (It seems the policy of the RFC editor to never fully document anything when you can just publish an add-on or supplement or extension RFC instead and let implementers figure out how it all works together.)
In its modern incarnation DNS is a centrally-controlled hierarchical recursive system. At the "top" of the network is the Internet Corporation for Assigned Names and Numbers (ICANN), a theoretically non-profit organization run by major technology companies. ICANN runs a large network of "root servers" around the world that are the authoritative source for the "." root domain. That is, those servers can tell you the authoritative servers that know all about top-level-domains (TLDs) such as "com.", "edu.", "org.", and so on. Each of those TLDs is controlled by some other organization that can tell you the authoritative servers that know about "yahoo.com.", "mit.edu.", or "fsf.org.". Each of those servers... you get the idea. At any point you can also specify the actual IP address of a system, that is, say that "yahoo.com." translates to some IP address. (There really is a period at the end of a Fully Qualified Domain Name (FQDN), although in virtually all programs it's safe to omit it as the software will silently assume it's there.)
Each authoritative server can have a number of records associated with it of various types. We're only interested in a few:
Every time a program (web browser, ssh client, instant messaging program, anything) tries to connect to a given server by name, it first asks the operating system to convert it into an IP address. The OS, in turn, is configured with one or more DNS servers to use (defined in /etc/resolv.conf in most *nix systems) that it will ask to turn the name, www.garfieldtech.com, for instance, into an IP address. The DNS server will check to see if it knows what IP address that refers to already. If not, it will ask the root servers if they know what server(s) are responsible for "com.". The root servers will direct the DNS server to one of the many authoritative servers for that TLD. The DNS server will then ask one of those servers if it knows the IP address of www.garfieldtech.com, and if not what server is authoritiative for "garfieldtech.com.". The "com." server will respond with the server(s) responsible for "garfieldtech.com.", and the DNS server will then ask that server if it know the IP address of "www.garfieldtech.com.". That server does, and will return an IP address. The DNS server will return that IP address to the computer that asked it in the first place, which will then return that IP address to the program so that it can go ahead and contact the remote server it asked for in the first place, blissfully ignorant of the complicated multi-step process involved.
Of course, it would be disgustingly inefficient to go through that complicated multi-step process every time you refreshed Slashdot.org. Therefore, DNS servers will cache every record they see for some period, 12 hours by default. That keeps the entire network reasonably clippy, and in general you will never see it in action except for the brief "looking up host www.garfieldtech.com" message that flashes by in your web browser's status bar when viewing a new web site.
There is, of course, much more to DNS but that's enough for our needs and I suspect you're already bored, so I'll get back to how to twist DNS into weird shapes that scare small children and make end-runs around greedy ISPs.
The caching of domain names speeds the system up considerably (it would be functionally useless without it), but does make one rather large assumption: Domain Name to IP address mappings change very rarely, and when they do the few days it takes to percolate through the entire worldwide network is acceptable. (Two! Two rather large assumptions!) That's fine if you have a static IP address, but most ISPs today don't give out static IP addresses to home users unless they pay some obscene amount of money, and even then only if you're on broadband. What about people who want to run a server at home that people can actually access?
The most common solution is something called "Dynamic DNS". Many organizations have sprung up offering dynamic DNS service. That is, you install some small program on your computer. Every time your computer connects to the Internet, it phones home to the provider's DNS server and gives it your computer's new IP address. The DNS server is already configured to have a very short cache time (and to tell other servers not to cache data it provides for very long), so changes percolate through the system in an hour or so instead of days.
The other challenge, of course, is that home users with multiple systems (such as yours truly) generally run a NAT firewall (Network Address Translation) in order to allow them to mask multiple computers as having only one IP address visible to the outside world. Besides making it possible to run multiple computers on a single broadband account, it adds a layer of security since remote systems can't connect to systems behind the NAT box without the NAT box being specially configured for it. Of course, that also makes accessing them when you actually want to more difficult.
So, with all of that background out of the way, here's what I set out to accomplish:
The main catch being that my home address is semi-dynamic and changes sometimes when my cable modem is restarted, and all my systems are behind a NAT firewall. (TWO! The Two main catches are...) At first blush it sounds easy. Just use one of them new-fangled dynamic DNS services. Ah, but it's not quite that easy. That works for points 1, 2, and 3, but not 4, 5, or 6. Something trickier is needed.
OK, so we need to run a DNS server. I can't run it at home, however, because DNS servers have to be on really-static IP addresses, and if other servers can't look up my IP address because it's dynamic then how are they going to look up my IP address to get my DNS server in order to look up my IP address?
Instead we get a full DNS host, one that will offer a real DNS server that I can configure. There are many many DNS providers out there. Personally, I use Pair Networks. Pair offers web hosting as well as DNS registration and hosting via its sister company PairNIC. They're not the cheapest host, but they're very reliable, offer good service, are all FreeBSD-based, and host free (and fast!) mirrors for Debian and various BSDs as good corporate citizens, so I'm willing to support them. They also offer almost everything I need to setup my inside-out network.
Here, then, is the magical incantation to run any server on a static domain name on a dynamic IP address without anyone knowing, in 10 easy steps. (Yes, we're finally getting to the point of this whole article. Yay!)
Note that the algorithm to delete irrelevant RRs breaks if LOCAL has a alias and the alias is listed in the MX records for REMOTE. (E.g. REMOTE has an MX of ALIAS, where ALIAS has a CNAME of LOCAL). This can be avoided if aliases are never used in the data section of MX RRs.
Since I'm not playing fancy games with LOCAL and REMOTE, only with CNAME, it doesn't seem to cause a problem. If, however, this step offends your sensibilities you are free to not run your own mail server on a dynamic IP address, and then renumber the following steps accordingly. (The latter is left as an exercise to the reader.)
So what exactly does this buy us? It allows me to have access to my home network via SSH or a web browser as well as have a "real" site hosted by a professional hosting company, and still have email@example.com as an e-mail address and as a Jabber address. And all without paying extra for a static IP address. That's just cool. In geek parlance, it's a "cleaver hack", from which most cool ideas are born (as well as many stupid recursive acronyms, but we won't get into that). If you feel like asking for more details, you can already reach me by Jabber at firstname.lastname@example.org, which I'm happy to say is now fully functional.
What? You don't know what Jabber is? Well, I guess that's a topic for another time.