Self-Hosted DERP Relay: Stable 20ms Domestic Fallback for Headscale

Background: I self-hosted a private network with Headscale. Most devices can connect directly through WireGuard P2P, but several machines behind symmetric NAT or carrier-grade NAT always failed NAT traversal and had to use a relay. Official Tailscale DERP relays are overseas for me, often starting at 100ms+. So I built a domestic DERP relay and brought latency down to around 20ms. This post records the full process and three pitfalls.

What Does DERP Do?

The ideal WireGuard mesh is peer-to-peer direct connection. In reality, many devices sit behind NAT, and NAT traversal does not always work. When direct connection cannot be established, traffic needs a public relay. That relay is DERP: Designated Encrypted Relay for Packets.

The key point: DERP is only a fallback when direct connection fails. If P2P works, DERP does not forward traffic. A self-hosted DERP is not meant to replace direct connection; it makes the fallback path less painful.

flowchart LR
    A["Device A
behind NAT"]
    B["Device B
behind NAT"]
    D["Self-hosted DERP
public relay"]

    A -. "1. try P2P first" .-> B
    A == "2. NAT traversal fails → relay" ==> D
    D == "2. encrypted forwarding" ==> B

    style D fill:#dbe9ff,stroke:#2563eb,stroke-width:2px

Flow Overview

flowchart TD
    S1["1. Install Go"] --> S2["2. go install derper"]
    S2 --> S3["3. Modify cert.go
disable hostname check"]
    S3 --> S4["4. Build + self-signed SSL cert"]
    S4 --> S5["5. systemd service
auto start"]
    S5 --> S6["6. Register DERP region
in Headscale"]
    S6 --> S7["7. Verify with netcheck
latency and hit"]
    S7 --> S8["8. Harden security
--verify-clients"]

    style S3 fill:#ffe9d5,stroke:#b71d18
    style S8 fill:#ffe9d5,stroke:#b71d18

The two orange steps are the easiest to miss: modifying source for self-signed certs, and preventing free riders.

1. Install Go

derper needs to be compiled with Go:

# Download; replace version if needed
wget https://golang.google.cn/dl/go1.23.2.linux-amd64.tar.gz
# Install
rm -rf /usr/local/go && tar -C /usr/local -xzf go1.23.2.linux-amd64.tar.gz
# Environment variable
export PATH=$PATH:/usr/local/go/bin
echo 'export PATH=$PATH:/usr/local/go/bin' >> /etc/profile && source /etc/profile
# Enable modules + domestic proxy
go env -w GO111MODULE=on
go env -w GOPROXY=https://goproxy.cn,direct
# Verify
go version

2. Install derper

go install tailscale.com/cmd/derper@latest

The source code is placed under ~/go/pkg/mod/tailscale.com@<version>/. The version changes, so check with ls before entering the directory.

3. Modify cert.go

By default, derper in manual certificate mode checks that the TLS handshake ServerName equals the startup hostname. With a self-signed certificate, this can cause clients to fail. Comment out that check.

# Replace version with your actual directory
cd ~/go/pkg/mod/tailscale.com@v1.76.3/cmd/derper/
sudo chmod 777 ./cert.go
sudo vim cert.go

Find getCertificate and comment out the hostname check:

func (m *manualCertManager) getCertificate(hi *tls.ClientHelloInfo) (*tls.Certificate, error) {
	// ↓↓↓ Comment out these lines for self-signed certs ↓↓↓
	// if hi.ServerName != m.hostname {
	// 	return nil, fmt.Errorf("cert mismatch with hostname: %q", hi.ServerName)
	// }

	certCopy := new(tls.Certificate)
	*certCopy = *m.cert
	certCopy.Certificate = certCopy.Certificate[:len(certCopy.Certificate):len(certCopy.Certificate)]
	return certCopy, nil
}

This edits the Go module cache. If you upgrade with go install, this change will be lost and must be reapplied.

4. Build and Create Self-Signed Certificates

# Build to a fixed path
go build -o /etc/derp/derper

# Self-signed SSL certificate; replace derp.e7coding.com
openssl req -x509 -newkey rsa:4096 -sha256 -days 3650 -nodes \
  -keyout /etc/derp/derp.e7coding.com.key \
  -out /etc/derp/derp.e7coding.com.crt \
  -subj "/CN=derp.e7coding.com" \
  -addext "subjectAltName=DNS:derp.e7coding.com"

5. systemd Service

sudo vim /etc/systemd/system/derp.service

Example:

[Unit]
Description=HW Derper
After=network.target
Wants=network.target

[Service]
User=root
Restart=always
ExecStart=/etc/derp/derper -hostname derp.e7coding.com -a :33445 -http-port 33446 -certmode manual -certdir /etc/derp
RestartPreventExitStatus=1

[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl start derp
sudo systemctl status derp

Parameter notes: -a :33445 is DERP HTTPS port, -http-port 33446 is the health-check HTTP port, and -certmode manual -certdir /etc/derp points to the self-signed cert.

6. Register the DERP in Headscale

On the Headscale server, edit or create DERP config:

vim /etc/headscale/derp.yaml

regions:
  900:
    regionid: 900
    regioncode: custom
    regionname: custom shanghai
    nodes:
      - name: 900a
        regionid: 900
        hostname: <DERP server public IP>
        ipv4: <DERP server public IP>
        ipv6: <DERP server IPv6; remove if none>
        derpport: 33445
        insecurefortests: true

insecurefortests: true is needed because of the self-signed certificate. Make sure headscale config.yaml references this file under derp.paths, then restart:

systemctl restart headscale
systemctl status headscale

7. Verify Latency

Run netcheck on any joined client. On macOS:

/Applications/Tailscale.app/Contents/MacOS/Tailscale netcheck

If the self-hosted node appears and latency is lower than overseas nodes, it works:

Report:
	* UDP: true
	* Nearest DERP: HW shanghai
	* DERP latency:
		-  hw: 23.8ms  (custom shanghai)   ← self-hosted, around 20ms
		-  hb: 43.3ms  (Huawei Beijing)
		-  hk: 46.7ms  (Hong Kong)

At this point it works, but do not stop yet. The default setup is open.

8. Harden Security with –verify-clients

By default, anyone who knows your DERP address and port can use it as a free relay. If the address leaks, you become someone else’s traffic relay.

The fix is to make DERP serve only nodes in your own network. Install a Tailscale client on the DERP server, log it into your Headscale, then add --verify-clients to derper. It will use local tailscaled to verify whether visitors belong to your tailnet.

flowchart TD
    X["Stranger has
DERP address + port"] -->|"--verify-clients enabled"| Y{"Is this a tailnet node?"}
    Y -->|"No"| Z["Reject relay"]
    Y -->|"Yes"| W["Forward normally"]

    style Z fill:#ffe9d5,stroke:#b71d18
    style W fill:#dbe9ff,stroke:#2563eb

Install Tailscale on the DERP server and log into your Headscale:

# Install
curl -fsSL https://tailscale.com/install.sh | sh
# Log into self-hosted Headscale
tailscale up --login-server=<your Headscale URL>
# The terminal prints a register link; approve the node in browser or headscale-ui

After approving the DERP node, add --verify-clients to the service:

ExecStart=/etc/derp/derper -hostname derp.e7coding.com -a :33445 -http-port 33446 -certmode manual -certdir /etc/derp --verify-clients

sudo systemctl daemon-reload
sudo systemctl restart derp
sudo systemctl status derp

Now this DERP only serves your own nodes.

Pitfalls

• Source changes are lost on upgrade: the cert.go change is in the Go module cache. Reapply it after upgrading derper.
• Do not copy the version path blindly: tailscale.com@v1.76.3 may differ. Check with ls.
• --verify-clients depends on local tailscaled: the DERP machine must successfully join Headscale first.
• Do not forget insecurefortests: true for self-signed certs.
• Open ports: cloud firewall / security groups must allow 33445 and your health-check port.

With control plane, web UI, and domestic relay ready, the final post covers daily device onboarding: Joining Devices to Self-Hosted Headscale.