If you are running a homelab or production environment, exposing services to the open web is a massive risk. Port forwarding 80 and 443 might seem harmless until your firewall logs instantly fill up with automated bots scanning for vulnerabilities.
Zero-trust networking is the best approach. Instead of poking holes in your firewall to let traffic in, your devices dial out to a central coordinator. That server introduces your devices to each other, and they build a direct, encrypted WireGuard tunnel between themselves.
For this situation, Tailscale is the most popular tool, but it relies on their proprietary control servers. If you want to own your infrastructure end-to-end, you want Headscale.
Headscale is an open-source clone of the Tailscale control server. Here is why it belongs in your homelab or production environment:
- No open inbound ports: Your router stays completely locked down.
- True self-hosting: Your network topology and encryption keys stay on your hardware.
- No device limits: Tailscale's free tier restricts your device count. Headscale lets you connect your entire Proxmox cluster, all your VMs, and your personal devices without hitting a paywall.
- WireGuard speed: You get the performance of WireGuard without the headache of manually managing routing tables and key exchanges.
Here is how to get it running using Docker Compose.
Prepare the Host
First, create the folders for your config and data. We also need to manually create an empty SQLite database file. If we don't do this, Docker will mistakenly create a directory named db.sqlite when we bring up the container.
sudo mkdir -p /opt/headscale/config /opt/headscale/data sudo touch /opt/headscale/data/db.sqlite cd /opt/headscale
The Docker Compose File
Create your docker-compose.yml file. Note that we are mapping the metrics port to 9091. If you run monitoring tools like Prometheus in your lab, port 9090 is usually already taken.
sudo nano docker-compose.yml
Paste this whole block.
services:
headscale:
image: headscale/headscale:latest
container_name: headscale
volumes:
- ./config:/etc/headscale
- ./data:/var/lib/headscale
ports:
- "8080:8080"
- "9091:9090"
command: serve
restart: unless-stopped
Configure Headscale
Headscale needs a configuration file to start. Download the official example file into your config folder.
sudo wget https://raw.githubusercontent.com/juanfont/headscale/main/config-example.yaml -O /opt/headscale/config/config.yaml
Open config/config.yaml and modify a few lines. Replace YOUR_SERVER_IP with your host's actual IP address. You also need to change listen_addr to 0.0.0.0:8080 so the application actually accepts traffic from outside the Docker container.
server_url: http://YOUR_SERVER_IP:8080 listen_addr: 0.0.0.0:8080 metrics_listen_addr: 0.0.0.0:9090
Double-check that db_type is set to sqlite3 and the db_path points to /var/lib/headscale/db.sqlite.
Start the Container and Create a User
Boot up the stack.
sudo docker compose up -d
Now, create a new user for this setup. Give it the name homelab. If you want to use another name, replace homelab below with your desired username.
sudo docker compose exec headscale headscale users create homelab
Connect a Device
Go to a different machine, such as a laptop or another VM, and install the standard Tailscale client. Instead of logging into Tailscale normally, tell the client to use your new Headscale server.
sudo tailscale up --login-server=http://YOUR_SERVER_IP:8080
The client will spit out a URL or a machine key. Copy the ID string that starts with hskey-authreq-.
Go back to your Headscale server terminal to authorize the device. Headscale recently updated their CLI syntax, so make sure you use the new auth register command:
sudo docker compose exec headscale headscale auth register --auth-id hskey-authreq-xxxxxxxxxxxx --user homelab
You are done. Your device is now connected to your private WireGuard mesh, managed entirely by your own server, without touching your router's firewall rules.
