Nebula can now set your system's DNS on macOS and Windows

Nebula, our open source mesh VPN, can now set your system’s DNS resolvers on macOS and Windows as of Nebula 1.12.0 (for Defined Networking Managed Nebula users, DNClient 1.0.0). Linux isn’t supported yet, for reasons this post will get into. The DNS code can be found on GitHub as dnsconfig, a standalone MIT-licensed Go library.

Until now, many Nebula users have reached their hosts by manually typing IP addresses. One of Nebula’s top-voted issues on GitHub for years has been “Control over client DNS servers” - meaning, the ability for Nebula to configure the DNS resolvers that applications on your computer query, when your computer is connected to the mesh.

This is something nearly every production VPN offers - from AnyConnect to OpenVPN. But while Nebula has long had experimental lighthouse DNS, which enables lighthouses to act as network DNS resolvers by resolving names in Nebula certificates to their associated IP addresses, Nebula offered no built-in way to point the computer it’s running on at that resolver once connected to the Nebula network.

This year we decided it was time for Nebula to support configuring the system’s DNS resolvers.

Our friends at Tailscale - who’ve written about the challenges of DNS in the past - released their client software under the permissive BSD 3-clause license.

So, when we started down this path ourselves, I wondered: can’t we just import Tailscale’s DNS package? They’ve had years to iron out the kinks, and sharing a codebase means we can contribute back bugfixes and improvements of our own.

But when I dug into the code I was disappointed to find that it was tightly coupled to Tailscale’s build and flag system and that many of the macOS pieces were in proprietary packages.

As a jumping-off point, I gave Claude an interface modeled on Tailscale’s: check whether split DNS is supported, read the original config, set the DNS, and close/cleanup. Knowing that the devil was going to be in platform-specific details, I added “Include a smoke test framework to verify each platform.” Within a week Claude had created a fully-formed repo claiming support for Windows, macOS and Linux. “Uh… Now what?” I thought to myself.

The smoke tests were passing, which was a great start, but I still didn’t have any confidence in the code. In the first version of the smoke tests, we only checked that the expected config had been written - not that DNS queries on the machine actually flowed through it. When Claude updated the tests to send real queries, two bugs were caught immediately. If the tests couldn’t even be trusted, would the code work on a real machine? What invisible tradeoffs had Claude made, and what were the failure modes?

I decided to take it for a spin. I installed it on a Windows VM and it appeared to work great! Then, I ran it on my MacBook and… nothing. It was time to stop and take a real look at the code Claude had produced, with human eyes.

The immediate bug was simple: the library was passing the name of the VPN device (e.g. utun4) to networksetup, but networksetup only configures network services like “Wi-Fi” and “Thunderbolt Ethernet.” The VPN device isn’t one, so nothing got set.

But the larger problem was that we were managing DNS on macOS through networksetup at all. This tool is the command-line interface into the System Settings app on macOS. It’s designed for admins to set their own config, not for apps that want to temporarily override those settings while a VPN is active. To make it reliable, we would have to back up the original settings, overwrite them with our own, and then run a watcher to react to any changes made while the VPN was still online. So I built exactly that, and it worked. And that’s when I realized this approach was entirely wrong.

My code had surpassed my own understanding of the problem space. Clearly, I was going to have to actually learn how each OS handles DNS under the hood.

I had a goal for how I wanted the library to behave. What I didn’t have was a deep understanding of the systems the library needed to interact with.

I wasn’t going to ship code I didn’t understand, so I embarked on a multi-month effort to learn everything I possibly could about Windows NRPT (Name Resolution Policy Table), Apple’s configd (the system configuration server), and the vast DNS ecosystem on Linux.

macOS and Windows both turned out to have a better solution than the one I’d started with. macOS’s configd stores two layers of network config: Setup:, the persistent settings admins configure, and State:, where DHCP and VPNs publish runtime facts like “while the user is connected to my network, this resolver is available.” Writing a State: key never touches the admin’s settings, and a reboot clears it, so a crash doesn’t leave a Mac stuck with our DNS forever. On Windows, NRPT does the same job, minus the crash safety. Each rule says “send this domain to that resolver,” sits alongside the admin’s own configuration, and is removed when we’re done. If the process crashes, the NRPT rules live on in the registry - when Nebula starts up next, it cleans up anything a previous run left over, before doing anything else.

$ scutil <<< 'show State:/Network/Service/dnsconfig-dnclient.host-<host-id>-<suffix>-0/DNS'
<dictionary> {
  ServerAddresses : <array> {
    0 : 192.168.100.1
  }
  SupplementalMatchDomains : <array> {
    0 : mesh.example.com
  }
  SupplementalMatchDomainsNoSearch : 0
}

Linux was a different story, and as I dug in, I gained a deeper appreciation for Tailscale’s blog post on configuring DNS on Linux. What I had not understood when originally reading their article was that while they had documented the most reliable method of detecting and configuring the DNS owner on the system, there were a number of cases where it was actually impossible to tell who was in charge of DNS, much less restore it correctly on shutdown.

Take for example a system where /etc/resolv.conf is written directly, with no manager. Your job is to back up the file, overwrite it, and restore the backup when you’re done. But if another VPN started before you, the “original” you backed up is actually its resolvers. And if it started after you, it may have saved your settings as its backup. Either way, whichever of you shuts down second restores a resolver that no longer exists, and the machine is left without DNS. In tests against Tailscale’s own backend code, we could reproduce these orderings, and their bug tracker contains a related case where a file left behind by a crash leaves the system without DNS. Even the team that’s spent years on this problem still runs into these edge cases.

After weeks spent trying to sort through every edge case on Linux, we decided to hit pause on Linux support. We had demonstrated that we could reliably configure DNS on Windows and macOS and we’d rather ship macOS and Windows now than wait for Linux support. That said, systemd-resolved looks to offer nearly the same affordances as NRPT and configd, so I am optimistic we can support it in the future.

Once macOS and Windows had been ironed out, it was time to share it with a wider audience. When I showed my colleague Matt Richardson what I had built, and a proof of concept integrating it into both Nebula and DNClient, he immediately noticed a problem: I had tried to defer some decisions to the application, like how often to check for drift, and where to store state. While I had hoped it would keep the library simpler and more flexible, the end result was that it was harder to use, and that each client ended up implementing the same lifecycle code.

The simplifications didn’t stop there. At this point, dnsconfig had grown two retry loops: one per OS backend, which repaired our settings whenever they were overwritten, and another that re-applied the config whenever a change failed. This also meant the Go structs had grown a pile of state fields, which had to be kept in sync, just to track which repairs were in progress and which had been given up on.

I asked Matt whether we should just retry forever, which would vastly simplify how much state we had to manage. He agreed: it’s a background service, and it should always be driving towards the desired state. A little spelunking showed that the bounded retries had only been added as part of our previous attempt at Linux support. On Linux, it’s not uncommon to get stuck in an endless fight with NetworkManager or DHCP clients. On Windows and macOS, these types of fights are rare.

That left one difference between the two loops: repair was gentler. It only rewrote what had been dropped. In contrast, setting new DNS rewrote everything. Claude proposed keeping repair as its own operation for exactly that reason, and Matt asked: why doesn’t setting DNS work that way too? Once it did, repairing just meant setting DNS again. The two loops became one, and all the state tracking them went away.

Claude would have happily kept that repair operation. It took Matt asking “why?” to simplify the design.

mgr, err := dnsconfig.New(ctx, dnsconfig.Options{
    Identity:      "nebula",
    InterfaceName: "nebula1", // Windows only: the tunnel adapter
})
if err != nil {
    return err
}
defer mgr.Close() // stops the worker and removes our DNS config

// Remove anything a crashed previous run left behind.
mgr.Clear()

// Declare the goal. A background worker applies it, retries, and
// repairs it if something else overwrites it.
mgr.Set(dnsconfig.Config{
    Nameservers:  []netip.Addr{netip.MustParseAddr("192.168.100.1")},
    MatchDomains: []string{"mesh.example.com"},
})

You can configure Nebula to set these settings by adding the following to your config.yml.

dns:
  enabled: true
  nameservers:
    - 192.168.100.1
  match_domains:
    - mesh.example.com
  search_domains:
    - mesh.example.com
  identity: nebula

If you have a resolver listening at 192.168.100.1, you will now be able to resolve mesh names.

# before
$ ping db.mesh.example.com
ping: cannot resolve db.mesh.example.com: Unknown host

# after
$ ping db.mesh.example.com
PING db.mesh.example.com (192.168.100.7): 56 data bytes
64 bytes from 192.168.100.7: icmp_seq=0 ttl=64 time=12.3 ms

These settings can be applied in Defined Networking’s Managed Nebula directly to hosts or through tags.

In the end, I’m very happy with where dnsconfig landed. There were a lot of technical challenges to bringing it to fruition, and I hope to cover more in a future blog post (hello, captive portals!) We’ve released it as its own MIT-licensed library that others in the Go ecosystem can import. It’s already shipped in Nebula 1.12.0 and DNClient 1.0.0.

Check out the library on GitHub or sign up for Managed Nebula to take it for a spin.