How I plan to expose the services running in my homelab to IPv4 users despite not having a publicly routable IPv4 address/subnet.
Table of Contents
Motivation
The problem I have is one that a lot of people in the self-hosting community will be familiar with. My ISP does not give me a public IPv4 address, to expose my self-hosted services to the broader internet.
Luckily my ISP provides me with a publicly routable /48 IPv6 subnet.
So I natively expose my services to the internet using IPv6 and then only have the problem of allowing native IPv4 only clients to use my IPv6 services as well.
Old Setup
Currently I rent a VPS from DigitalOcean, which then naively forwards any TCP connections to my IPv6 services.
For this I have created ipv-proxy, which loads registered services from Consul and looks for specific tags in their configuration. These allow me to easily configure a service to be exposed and how that should be done, like what external port etc. For each such service, it then starts a TCP-listener on it's public IPv4 address and for all incoming requests connects to the IPv6 service being exposed, forwarding all data in both directions.
However because this is centered around having a connection, connectionless things like UDP do not work with this setup. And obviously it is not ideal to essentially split the one "logical" tcp connection from the client to my service, into two underlying tcp connections from the client to the vps and then the vps to the service.
Idea 1 - SIIT-EAM
The Idea
SIIT-EAM (Stateless IP/ICMP Translation) allows one to have a "translator" in the network, which receives the incoming IPv4 packets and translates them into their corresponding IPv6 destinations. The translator needs to have one more public IPv4 and an IPv6 subnet with at least "free" 32 bits. One can then configure how the public IPv4 addresses should map to IPv6 addresses and the translator will act as a middle-man between IPv4 and IPv6.
For more details I would recommend going through the jool1 documentation.
The Plan
I can rent a relatively cheap VPS on some cloud provider, with at least 1 public IPv4 and then it should ideally have a routable /64 IPv6 subnet as well.
Then I configure SIIT-EAM in jool to forward the IPv4 traffic to one of my servers IPv6 addresses.
Setup
Detailed instructions on how to setup the VPS
- Get a server that supports Dual-Stack networking and in the best case a /64 ipv6 subnet (I choose Scaleway as a European cloud provider, with cheap servers) Scaleway IPv6 Docs Scaleway Check neighbor discovery
- apt-get update and apt-get upgrade
- Install Jool
- Based on the jool documentation
sudo apt install jool-dkms jool-tools- Enable IP forwarding
/sbin/sysctl -w net.ipv4.conf.all.forwarding=1/sbin/sysctl -w net.ipv6.conf.all.forwarding=1
- Install NDP Proxy Daemon ndppd
sudo apt-get install ndppd/sbin/sysctl -w net.ipv6.conf.all.proxy_ndp=1
- Configure Jool
/sbin/modprobe jool_siitjool_siit instance add "example" --netfilter --pool6 2001:0bc8:1640:6554:0:0:0:0/96jool_siit -i "example" eamt add 2001:4dd5:b276:1:f652:14ff:fe94:dc00/128 51.158.177.228/32- (Optional for debugging)
jool_siit -i "example" global update logging-debug true
- ndppd for neighbor discovery
- In
/etc/ndppd.confproxy ens2 { rule 2001:0bc8:1640:6554:0:0:0:0/96 { static } }
- In
Troubles encountered
This is a small list of issues that I have encountered, while setting all of this up and you might also run into if you are doing this yourself.
Jool always requires a pool6 to be configured for SIIT, even if you are only configuring/using EAM entries.
There will be a lot of debugging of the network traffic going on. I recommend getting comfortable with tools like tcpdump or wireshark on both the "target" machine, the client and the VPS, to hopefully get a full picture of how packets are flowing.
Depending on your provider and specific configuration, Neighbor Solicitation might not work out of the box for you.
I first saw logs like 13:31:35.919212 IP6 _gateway > ff02::1:ff52:2f24: ICMP6, neighbor solicitation, who has 2001:bc8:1640:6554::4a52:2f24, length 32 in my tcpdumps on the VPS, which the server should have replied to, as it has the specific subnet assigned.
To fix this I added the last step in the setup, to configure ndppd, as that fixed my issue.
The biggest drawback of this approach is that everything ariving on the IPv4 address gets forwarded to the specific IPv6 address, with no fine grained control.
Idea 2 - NAT64 with static BIB
The Idea
The idea with this is to basically perform some static NAT64, to map ports on the IPv4 side to specific addresses and ports on the IPv6 side. This would allow me to have one entry for every port that I want to expose, regardless of the IPv6 address or port of the service.
Compared to Idea 1
The main difference between this and Idea 1 (SIIT-EAM), is that NAT64 allows me to map single ports and basically use a single IPv4 address to host a variety of services, that might be running on different hosts.
Setup
Detailed instructions on how to setup the VPS
- Get a server that supports Dual-Stack networking and in the best case a /64 ipv6 subnet (I choose Scaleway as a European cloud provider, with cheap servers) Scaleway IPv6 Docs Scaleway Check neighbor discovery
- apt-get update and apt-get upgrade
- Install Jool
- Based on the jool documentation
sudo apt install jool-dkms jool-tools- Enable IP forwarding
/sbin/sysctl -w net.ipv4.conf.all.forwarding=1/sbin/sysctl -w net.ipv6.conf.all.forwarding=1
- Install NDP Proxy Daemon ndppd
sudo apt-get install ndppd/sbin/sysctl -w net.ipv6.conf.all.proxy_ndp=1
- Configure Jool
/sbin/modprobe jool- Setup NAT64
jool instance add "example" --netfilter --pool6 2001:0bc8:1640:6554:0:0:0:0/96 - Setup pool4 for UDP
jool -i "example" pool4 add --udp 51.158.177.228 1-65535 - Setup pool4 for TCP
jool -i "example" pool4 add --tcp 51.158.177.228 1-65535
- ndppd for neighbor discovery
- In
/etc/ndppd.confproxy ens2 { rule 2001:0bc8:1640:6554:0:0:0:0/96 { static } }
- In
- Example Setup of bib[^bib_explained] entries (for teamspeak3 in this case)
jool -i "example" bib add 2001:4dd5:b276:1:f652:14ff:fe94:dc00#9987 51.158.177.228#9987 --udp[^bib_add_command]jool -i "example" bib add 2001:4dd5:b276:1:f652:14ff:fe94:dc00#30033 51.158.177.228#30033 --tcp[^bib_add_command]
Future Work
I need to automate the setup of jool and the VPS. My first thought would be to use ansible, as with most of my other services, but I might actually think that using something like terraform + cloud-init could be a better fit. Using terraform together with cloud-init, I could avoid having to log into the VPS at all and the configuration will be done automatically upon creation/startup.
I also want to integrate the configuration of jool, like the NAT64 entries, directly into ipv-proxy. This way I can easily migrate from the old setup the new one (related MR) and do not have to change anything else.
References
We need to configure the pool4 used by jool using the given pool4 commands