Handshake

A handshake is the exchange two hosts go through before they can send each other traffic. In Managed Nebula, each host proves who it is with its certificate, and the two agree on fresh encryption keys for the tunnel between them.

What happens during a handshake

Nebula’s handshake follows the IX pattern from the Noise Protocol Framework and takes two messages. The initiating host sends a new ephemeral public key along with its certificate. The receiving host checks that the certificate was signed by a certificate authority it trusts, hasn’t expired, and doesn’t belong to a blocked host. If it passes, the receiver replies with its own ephemeral key and certificate, and the initiator checks those the same way.

Both hosts then derive the same session keys from the ephemeral keys. Key agreement uses Curve25519 or NIST P-256, and traffic from then on is encrypted with AES-256-GCM (see Cipher).

Because each certificate includes the host’s name, overlay IP, role and tags, the firewall rules applied afterward act on an identity the CA vouched for. Each handshake also uses new ephemeral keys, so stealing a host’s long-term private key later doesn’t let anyone decrypt traffic they recorded earlier.

A lighthouse may let a host know that a peer is trying to reach it, as part of NAT traversal, but it isn’t part of the handshake and never holds the keys. A relay can forward handshake packets without being able to read or change them. And since certificates are exchanged during the handshake itself, a newly enrolled host can connect to existing hosts without any of them hearing about it first. PKI covers why that matters.

When handshakes fail

If two hosts can’t connect, DNClient’s logs show how far the handshake got. The usual causes are UDP being blocked between the hosts with no relay available, or one of the hosts having been blocked in the admin panel, in which case every other host rejects its certificate. See Viewing DNClient logs.