Implementing the ideas and conclusions from my initial post about exposing my IPv6 Homelab to IPv4 clients (Homelab IPv4 Expose) in the real world and deploying the entire system.

🔗Background

This will cover some of the background needed for this setup, if you have already read the previous post or are familiar with jool, NAT64 and ipv-proxy, you can skip this section.

🔗Jool

Jool is a tool that implements a bunch of IPv6 and IPv4 translation mechanisms. This does the heavy lifting, when it comes to actually processing the packets and performing the low-level translations.

🔗Stateful NAT64

Stateful NAT64 is one of the mechanisms to translate between IPv6 and IPv4 networks. It works similarly to normal NAT, with the addition of not only mapping ports and addresses, but mapping between an IPv6 and IPv4 address. So one side of the translation has a unique IPv6 address + port combination and the other side is a unique IPv4 address + port combination.

🔗ipv-proxy

ipv-proxy is a custom tool, I developed, which dynamically reads configuration from consul to expose IPv6 services to IPv4 clients. This usually runs on some external server, which can do both IPv4 and IPv6. Services in consul can then add specific tags, which are found by the ipv-proxy, which then exposes the IPv6 only service from the IPv4 address of the server, by forwarding all the traffic.

Previously this only worked with TCP and was doing the forwarding naively in user-space software, but as part of this I added support for using jool to do the actual forwarding.

🔗Plan

Okay so there are a couple of goals/requirements I have for this deployment:

1. The entire setup should be automated and portable between vendors
2. Extend ipv-proxy to natively use jool for forwarding
3. Use wireguard for connecting to my internal consul cluster

🔗1. Automated and portable setup

To make sure my setup is repeatable and portable between different providers, I decided to fully rely on cloud-init.

#cloud-config
ssh_authorized_keys:
  - ${macbook_ssh_key}
  - ${ubuntu_ssh_key}

package_update: true
package_upgrade: true

packages:
  - jool-dkms
  - jool-tools
  - ndppd

write_files:
  - path: /etc/sysctl.d/99-forwarding.conf
    permissions: '0644'
    content: |
      net.ipv4.conf.all.forwarding=1
      net.ipv6.conf.all.forwarding=1
      net.ipv6.conf.all.proxy_ndp=1
  - path: /etc/ndppd.conf
    permissions: '0666'
    content: |
      proxy ens2 {
          rule ${ipv6_prefix} {
              static
          }
      }
  - path: /etc/modules-load.d/jool
    permissions: '0666'
    content: |
      jool
  - path: /usr/local/bin/ipv-proxy
    permissions: '0777'
    source:
      uri: https://github.com/Lol3rrr/ipv-proxy/releases/download/${ipv_proxy_version}/ipv-proxy
  - path: /etc/systemd/system/ipv-proxy.service
    content: |
      [Unit]
      Description=Proxy IPv4 traffic to IPv6
      After=cloud-init.target
      Wants=cloud-init.target

      [Service]
      Type=simple
      ExecStart=/usr/local/bin/ipv-proxy %{ for addr in consul_addrs ~}--consul-addr ${addr} %{ endfor ~} --backend jool --public-ip ${proxy_ipv4_public} --jool-pool6-subnet ${proxy_ipv6_subnet}

      [Install]
      WantedBy=multi-user.target

wireguard:
  interfaces:
    - name: wg0
      config_path: /etc/wireguard/wg0.conf
      content: |
        [Interface]
        Address = 10.200.0.200/32
        MTU = 1280
        PrivateKey = ${wg_private_key}

%{ for peer in wg_peers ~}
        [Peer]
        # Name ${peer.name}
        PublicKey = ${peer.public_key}
        AllowedIPs = ${peer.ip}/32
        Endpoint = ${peer.endpoint}
        PersistentKeepalive = 25
%{ endfor ~}

runcmd:
  - "sysctl -w net.ipv4.conf.all.forwarding=1"
  - "sysctl -w net.ipv6.conf.all.forwarding=1"
  - "sysctl -w net.ipv6.conf.all.proxy_ndp=1"
  - "sysctl --system"
  - "modprobe jool"
  - "systemctl daemon-reload"
  - "systemctl enable ipv-proxy.service"
  - "systemctl start --no-block ipv-proxy.service"

The configuration essentially does everything mentioned in the first post of this mini-series + setting up wireguard.

🔗2. Extend ipv-proxy

In its original state, the ipv-proxy could only do "manual" forwarding and connect to a single consul address, which both needs to be reworked.

The first change, was to add support for different backends that perform the actual forwarding. Previously it was pretty hard-coded how the forwarding worked, but now all the behaviour for the forwarding is abstracted away and the core logic just gets something that knows how to do all the forwarding specific stuff. This allowed me to easily add a jool backend, which just needs to implement the corresponding trait and integrate it into CLI, but the rest of the logic is left untouched. Another benefit, that I hope to utilize soon, is that I can also pass in dummy/mock/test implementations for forwarding, to test the core logic and its interaction with the forwarding.

The second change, was more subtle and in the pursuit of better reliability, as it now supports having multiple consul addresses. Previously one could only pass a single address and if that had issues, then the ipv-proxy was basically not working anymore. However now, I can pass the addresses of all my consul servers and it will try all of them until one responds, which means I no longer have a single point of failure in this regard.

🔗3. Wireguard setup

This is required for my use-case, because ipv-proxy needs access to my consul cluster regardless of any proxy status or issues and to ensure this, I simply setup a wireguard network. Specifically in my case, all of my internal servers act as wireguard servers listening on their public IPv6 address for connections and then the external server connects to all of them for redundancy.

🔗Deployment

🔗Hosting Provider

Hard Requirements:

• Provides at least 1 public routable IPv4 address
• Provides at least a /33 IPv6 subnet
  • /32 for the NAT64
  • 1 address for the server itself
• Supports automation using terraform and cloud-init
• Reliable

Soft Requirements:

• Competitive prices
• European provider

I ultimately decided to go with Scaleway, because they tick all the boxes for me.

🔗Automation

The entire deployment is completly automated using terraform and cloud-init, as mentioned earlier. Using this combination allows me to completly automatically and easily spin up a new instance, and I am making use of that by never touching the existing instance for changes, but instead just recreating/replacing the entire thing.

🔗Final Thoughts

With everything combined, I now have a completly automated system for setting up my external proxy server and the proxying my services. All I need to do in my day-to-day to expose something to IPv4 clients, is to add a corresponding tag to the service definition.