Learning how to dig made me a better programmer
“You never know what your next dig is going to find.”
This week at work we had a massive DNS resolution failure with one of our
datacenters. The issue was quickly resolved, but in the process, they assigned
our services new IPs which took forever to propagate. While it is known to most
people that DNS propagation can take up to 72 hours, this meant total
downtime for our customers in the meantime. Also, because our team could not do
anything about it, I simply updated all my tickets, and then took the time to
finally sit down and understand how
dig query looks like
dig @server name type. But if you simply
dig and do nothing else, you end up getting the 13 root nameservers. They
play a fundamental role in the operation of DNS infrastructure.
The root nameservers do not directly handle queries for specific domain names but rather provide information about the authoritative nameservers for each top-level domain. These authoritative nameservers, in turn, handle queries for their respective domains.
dig ; <<>> DiG 9.10.6 <<>> ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 11663 ;; flags: qr rd ra; QUERY: 1, ANSWER: 13, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;. IN NS ;; ANSWER SECTION: . 1800 IN NS d.root-servers.net. . 1800 IN NS h.root-servers.net. . 1800 IN NS j.root-servers.net. . 1800 IN NS a.root-servers.net. . 1800 IN NS k.root-servers.net. . 1800 IN NS i.root-servers.net. . 1800 IN NS e.root-servers.net. . 1800 IN NS g.root-servers.net. . 1800 IN NS l.root-servers.net. . 1800 IN NS m.root-servers.net. . 1800 IN NS f.root-servers.net. . 1800 IN NS b.root-servers.net. . 1800 IN NS c.root-servers.net. ;; Query time: 127 msec ;; SERVER: 100.64.0.2#53(100.64.0.2) ;; WHEN: Sun Jun 25 18:21:17 IST 2023 ;; MSG SIZE rcvd: 228
That… is a lot of noise. Perhaps that’s because I don’t fully know what every line means. Today, I’m going to change that.
The first thing that caught my attention was that
dig was using (at the time)
an unknown IP for resolving DNS queries. I know I set my primary DNS resolver
184.108.40.206, so where was this IP coming from?
man dig to the rescue:
If no server argument is provided, dig consults
/etc/resolv.conf; if an address is found there, it queries the nameserver at that address.
Let’s check what my
/etc/resolve.conf looks like:
# # macOS Notice # # This file is not consulted for DNS hostname resolution, address # resolution, or the DNS query routing mechanism used by most # processes on this system. # # To view the DNS configuration used by this system, use: # scutil --dns # # SEE ALSO # dns-sd(1), scutil(8) # # This file is automatically generated. # nameserver 100.64.0.2
Ah, macOS, you never fail to
amaze piss me (off). Let me check what
scutil --dns | grep 'nameserver\[[0-9]*\]' | sort | uniq nameserver : 220.127.116.11 nameserver : 100.64.0.2 nameserver : 127.0.0.1 nameserver : 18.104.22.168 nameserver : 2606:4700:4700::1111 nameserver : 2606:4700:4700::1001
My machine is using
22.214.171.124 for scoped queries, and
100.64.0.2 is used as
;; SERVER: 100.64.0.2#53(100.64.0.2)
Now, what is the
#53 above? That’s the port number the Internet Assigned
Numbers Authority (IANA) assigned to be used for DNS.
Going back to the original query, because I passed no arguments/flags/options to
dig, it performed a nameserver (
NS) query to the root
., which is at the
top of the DNS hierarchy. Basically, if you did a
tree of the DNS hierarchy,
this is what it would look like:
. (root) ├── com (TLDs) │ ├── example │ │ ├── www (SLDs) │ │ └── mail ├── ca (CC-TLDs) │ ├── example
The DNS hierarchy is a lot more dense, and I have intentionally left out those
bits. You’ll also notice that each domain in the
ANSWER section ends with a
.. That’s (
a.root-servers.net.) what you call a Fully Qualified Domain Name
FQDN). I won’t go any further into this topic, but
you should know that that is not an error.
. 1800 IN NS a.root-servers.net.
1800 above is the time-to-live (TTL) for the entry, or how long the DNS
resolver will cache the particular entry to optimize subsequent queries.
. 1800 IN NS a.root-servers.net.
IN stands for the Internet class. Other classes include
CH (Chaos net),
HS (Hesoid), and
NONE for placeholder records. I’ll use
CH in an upcoming
section for debugging purposes. The chaos class is a special class used for
querying server-related information.
Personally, I would have liked it if Chaosnet took off instead of the Internet. I like the name more.
Being able to query the root nameservers is great, but that’s rarely why you
dig in day-to-day life. You’re reading this post on
kimchiii.space; but what nameservers does my domain use?
dig +short @126.96.36.199 NS kimchiii.space. ns1.vercel-dns.com. ns2.vercel-dns.com.
I’m sure Vercel informs its users what DNS service they use (it’s NS1, but I think in the past they use AWS’ Route 53), but it’s cool that you can find out this information from the comfort of your command-line.
Nice. It looks like
ns1.vercel-dns.com. is the primary DNS server for my
ns2.vercel-dns.com. is the secondary DNS server.
I want to take it a step further.
ns2.vercel-dns.com. are authoritative nameservers for Vercel, but what IPs do
they resolve to?
dig +short @188.8.131.52 A ns1.vercel-dns.com. 184.108.40.206
Who does this IP address belong to? Well if I haven’t fallen victim to DNS
whois 220.127.116.11 should return info on NS1.
whois 18.104.22.168 ... NetRange: 22.214.171.124 - 126.96.36.199 CIDR: 188.8.131.52/23 NetName: NSONE-DNS NetHandle: NET-198-51-44-0-1 Parent: NET198 (NET-198-0-0-0-0) NetType: Direct Allocation OriginAS: AS62597 Organization: NSONE Inc (NSONE) RegDate: 2013-08-07 Updated: 2021-12-14 Comment: http://nsone.net Ref: https://rdap.arin.net/registry/ip/184.108.40.206 ...
Phew. Not seeing NSONE Inc as the
Organization would have me tripping right
now. Looks like I can move on with my experiment.
I’m having way too much fun. I don’t want to stop now. I have already confirmed
that they have moved away from AWS for their DNS needs, but what about their
hosting? I want to find out more about the IPs used to resolve my domain
dig +short @220.127.116.11 A kimchiii.space. 18.104.22.168 22.214.171.124
Who do these IPs belong to? As of me writing this post, I think Vercel still uses AWS. Let’s confirm that:
... OrgName: Vercel, Inc OrgId: ZEITI Address: 340 S LEMON AVE #4133 City: Walnut StateProv: CA PostalCode: 91789 Country: US RegDate: 2020-03-26 Updated: 2020-06-05 Comment: https://vercel.com Ref: https://rdap.arin.net/registry/entity/ZEITI ...
Hmm… this is not alarming, but it is also not what I expected. Does this
mean Vercel uses its own infrastructure for hosting? I’m not so certain, because
ipinfo.io tells me that the ASN
(AS16509) for the IP
126.96.36.199 is Amazon. It
also confirms that it is allocated for hosting purposes. If you further drill
down information on the allocated IP address ranges, you will find that
188.8.131.52/24 is assigned to Vercel,
which to me, confirms that Vercel still uses AWS for hosting.
If you were to run the same query, you’d most likely get a different set of IPs each time. This is most likely because Vercel (or rather, NS1) is implementing DNS load balancing or DNS round-robin techniques. This is a good thing.
Going back to the tree structure of the DNS hierarchy, I should be able to trace the path to my domain’s nameserver from the root nameserver by iteratively querying the authoritative nameservers for each level of the DNS hierarchy. Let’s try that:
dig +short @184.108.40.206 NS . a.root-servers.net. b.root-servers.net. c.root-servers.net. d.root-servers.net. e.root-servers.net. f.root-servers.net. g.root-servers.net. h.root-servers.net. i.root-servers.net. j.root-servers.net. k.root-servers.net. l.root-servers.net. m.root-servers.net.
220.127.116.11 with either one of the 13 root nameservers:
dig @a.root-servers.net. NS kimchiii.space. ... ;; AUTHORITY SECTION: space. 172800 IN NS b.nic.space. space. 172800 IN NS e.nic.space. space. 172800 IN NS f.nic.space. space. 172800 IN NS a.nic.space. ...
Then, replacing one of the root nameservers with the authoritative nameservers above:
dig @a.nic.space. NS kimchiii.space. ... ;; AUTHORITY SECTION: kimchiii.space. 3600 IN NS ns2.vercel-dns.com. kimchiii.space. 3600 IN NS ns1.vercel-dns.com. ...
Bingo. We have arrived at the same nameservers from the previous section.
FWIW, you can find out this information using
dig +all +trace @server <fqdn>, albeit the output is a bit too verbose
for this use-case.
What if I want to find out which nameserver node responds to my queries
dig? As I’m writing this post, I’m connected to a NordVPN server hosted
somewhere in London. Because my site is hosted at the edge, I should be
hitting an edge datacenter hosted in London, or closest to it. Let’s confirm
dig +norec +short @ns1.vercel-dns.com. CH TXT hostname.bind "ns1dns-lhr04-11184-5310"
FeelsGoodMan to be right.
lhr04 gives it away.
LHR is the airport code for
Heathrow Airport. Now I’m going to connect to a server in Canada (Montréal):
dig +norec +short @ns1.vercel-dns.com. CH TXT hostname.bind "ns1dns-yyz03-11199-5323"
Interesting. Looks like NS1 does not have any datacenters in Montréal because YYZ is the airport code for Toronto Pearson International Airport. Or I just don’t happen to be hitting it. This opens up an avenue for another experiment. 😏
There are several DNS record types that I need to check from time-to-time, so I thought I’d mention them here.
This is slowly being
phased out, but a
lot of public DNS servers still respond to this record with results.
18.104.22.168 already returns nothing:
dig +short @22.214.171.124 ANY kimchiii.space.
AdGuard and Quad9 DNS return an
HINFO in the answer section set to
dig @126.96.36.199 ANY kimchiii.space. ... ;; ANSWER SECTION: kimchiii.space. 60 IN HINFO "RFC8482" "" ...
dig @188.8.131.52 ANY kimchiii.space. ... ;; ANSWER SECTION: kimchiii.space. 60 IN HINFO "RFC8482" "" ...
You can try other DNS servers. I’ll update this post once I find one that still returns results. But you should probably stop using this record type.
MX record directs email to a mail server. In my case, I use
ImprovMX for email forwarding. One can confirm this by
dig +short @184.108.40.206 MX kimchiii.space. 10 mx1.improvmx.com. 20 mx2.improvmx.com.
The highlighted parts above tell you the priority (or preference) with which a mail server, or rather, the Message Transfer Agent (MTA) will try to send emails. Lower number is higher priority. But one can also set the same value for the priority to enable load-balancing.
PTR records serve the opposite purpose of
A records, which provide the
IP address associated with a domain name.
PTR records are used in reverse DNS
lookups. The most common use-case I can think of is a mail server using the
reverse lookup to confirm that an email came from the source it claims to have
dig +short @220.127.116.11 NS kimchiii.space. ns1.vercel-dns.com. ns2.vercel-dns.com.
dig +short @18.104.22.168 ns1.vercel-dns.com. 22.214.171.124
dig -x 126.96.36.199 ... ;; QUESTION SECTION: ;188.8.131.52.in-addr.arpa. IN PTR ;; ANSWER SECTION: 184.108.40.206.in-addr.arpa. 1800 IN PTR dns1.p13.nsone.net. ...
This is all well and good, but this is the crucial part: looking up the
dns1.p13.nsone.net. should return the IP address
dig +short @220.127.116.11 A dns1.p13.nsone.net. 18.104.22.168
dig +all +multiline @22.214.171.124 SOA kimchiii.space. ... ;; ANSWER SECTION: kimchiii.space. 3600 IN SOA ns1.vercel-dns.com. hostmaster.nsone.net. ( 1655751660 ; serial 43200 ; refresh (12 hours) 7200 ; retry (2 hours) 1209600 ; expire (2 weeks) 60 ; minimum (1 minute) ) ...
You can also use the
SOA query to find out the primary nameserver for a
domain. Here’s another example:
dig +all +multiline @126.96.36.199 SOA example.com. ... ;; ANSWER SECTION: example.com. 3600 IN SOA ns.icann.org. noc.dns.icann.org. ( 2022091303 ; serial 7200 ; refresh (2 hours) 3600 ; retry (1 hour) 1209600 ; expire (2 weeks) 3600 ; minimum (1 hour) ) ...
The FQDN that follows the highlighted ones above are not actually domains, but
rather an email address. For example, in the case of NS1, you would contact
email@example.com for any concerns. This email is different from the email
address used to contact for abuse. That can be found using
The Sender Policy Framework (SPF) record type doesn’t
actually exist any
more, but it can be set using the
TXT record. It exists because the Simple
Mail Transfer Protocol (SMTP), by default, performs no authentication on the
“From” address in an email. Here’s what the SPF record for my domain looks like:
dig +short @188.8.131.52 TXT kimchiii.space. "v=spf1 include:spf.improvmx.com ~all"
Because there is no dedicated
SPF record, your
TXT record must begin
v=spf1, or the server querying your domain won’t know of its existence.
You cannot have more than one SPF record per domain, either.
I use an
include tag, which tells the server what third-party organizations
are authorized to send emails on behalf of the domain. The domain must be a
valid one, and in my case, it is set to ImprovMX’s domain.
I also made it so that unlisted emails will be marked as insecure or spam but
still accepted, using the
~all option. Other options are
-all (reject any
and all emails not listed in the SPF record) and
+all (any server can send
emails on your behalf). You probably don’t want to set the last option. If you
don’t set either of the
all options, then you must set a
redirect: tag which
tells the MTA that the SPF record is hosted by another domain.
I had a lot of fun with this experiment. I started working on this post at ~0100h EST, and it is now ~0800h EST. Now, I’m going to go make me some coffee.
I left out any and all record types pertaining to DNSSEC because otherwise this post would get too long. I think that DNSSEC deserves its own post, so I’m working towards it as you read this.
Another thing I’d like to mention is that I used
dig that came pre-installed
with macOS (
9.10.6), which has no support for DoH or DoT. If you have that
requirement, you may want to upgrade
9.16 or higher. You may also
wish to use
If you’ve read this far: wow, thank you! Did I get anything wrong? Anything you think I should mention? Feel free to email me.