Underlay network

The underlay network is the physical network an overlay network runs on: home Wi-Fi, an office LAN, a cloud VPC, a mobile carrier, and the internet in between. A host gets underlay addresses from whatever network it’s on, and keeps the same overlay address wherever it goes.

A laptop with the overlay address 100.100.0.5 might be 192.168.1.23 at home, behind a router with a public IP, and 10.20.4.17 at the office. On a phone hotspot it’s something else again. Its overlay address never changes, so nothing that connects to it has to know where it is.

How Managed Nebula uses the underlay

Each host tells the network’s lighthouses its local underlay addresses, and the lighthouse also sees the public address its traffic comes from. When two hosts want to talk, the lighthouse shares those addresses and the hosts try them to find a direct path (NAT traversal).

Overlay traffic then crosses the underlay as encrypted UDP. Routers and anyone else on the underlay can see which addresses are exchanging packets, but not what’s in them.

What the overlay needs from the underlay

Hosts need to be able to exchange UDP packets, directly or through a relay. The overlay’s range can’t overlap any underlay range your hosts use, or traffic meant for the overlay can end up on the local network instead (see Choosing a CIDR and the CGNAT range). And because overlay packets get bigger when they’re wrapped for the underlay, the overlay uses a smaller MTU.

Controlling which underlay addresses are used

A few config overrides change how hosts use the underlay:

• lighthouse.local_allow_list limits which local addresses a host reports, for example to leave out Docker networks no other host can reach.
• lighthouse.remote_allow_list limits which addresses a host tries when connecting to peers, for example only private ranges so traffic never crosses the internet.
• preferred_ranges tells a host which underlay ranges to prefer when a peer has several addresses.

Configuring hosts with tags has examples.