This guide assumes the following:
As a systems engineer, I’m a big proponent of hybrid infrastructure with an emphasis on self-hosting whenever possible. The problem is that my ISP wants an arm and a leg for a single static IPv4 address while providing zero IPv6 support. It could be worse — I could be sitting behind more layers of CGNAT than a Russian nesting doll (looking at you, T-mobile).
My ISP charges $25/month for a static address, does not allow PTR control, and this comes on top of the $300/month I already pay for a business SLA. So what’s a systems engineer to do? Deploy a remote gateway, of course.
For those new to the concept, a remote gateway is fairly simple: we move our network’s egress to the internet somewhere else. We achieve this by tunneling our traffic to another PoP, which solves the issue of not having affordable, static, globally routable addresses. The trade-off is added latency and possibly less stability, though this depends mostly on our ISP and geographic location. In most cases, the latency is a negligible. In my setup, the VyOS core router establishes a Wireguard tunnel with the remote gateway and sends all of my lab traffic through the remote peer.
To build a remote gateway, we need a peer in the physical location where we want our egress to be. Since my network is IPv6-native, I also need an upstream provider with solid IPv6 support. At the time of writing, the only IaaS provider I know of that properly supports IPv6 is Linode. What do I mean by “proper support”? IPv6 best practice dictates that the smallest subnet should be a /64. If a provider won’t allocate a larger subnet, network segmentation through subnetting becomes limited and less efficient. Namely, we loose SLAAC and see double TCAM entries for prefix mapping. Most IaaS providers will only allocate a single /64 per instance. Of those that will give you a larger prefix, Linode is the only one that routes it to a virtual instance. AWS, for example, offers /56 prefixes but attaches them to their VPC product, which means we cannot advertise the prefix to our core network.
For this guide we’ll be using the following parameters:
2001:db8:1234:5600::/56 198.51.100.77 With our location chosen, it’s time to get our remote gateway up and running. To choose the best deployment location for our remote peer, we’ll run some ping and traceroute tests. Linode provides public endpoints for performance checks. In short, we want the datacenter with the best stability and throughput, and the lowest latency relative to our location. For example, to check the average round-trip latency of their Atlanta DC, we can ping speedtest.atlanta.linode.com. Here’s a one-liner to calculate average latency and jitter:
ping -c 20 speedtest.atlanta.linode.com | tail -n 1 | awk '{split($4,val,"/"); print "Avg: " val[2] "\nJit: " val[4]}'
It’s also wise to run a traceroute to see the tunnel’s path. We should repeat these tests at different times of day to confirm consistent performance. In my case, the closest DC has ~5ms latency, very low jitter, and no packet loss. That’s pretty darn good.
Wireguard is mostly CPU-bound because the protocol aims for broad device compatibility. Since many devices lack dedicated cryptographic APUs (like AES-NI), Wireguard employs ChaCha20 as its stream cipher. While AES performs well with hardware offload, ChaCha20 is more efficient when no APU is available.
Generally, a single core instance can achieve around 350 Mbps after kernel tuning. However, the WAN connection for our core network is a symmetrical gigabit connection, so I opted for a two-core instance. This allows me to reach fairly close to line speed at around 900 Mbps. If we wanted to get closer to our full 1 Gbps speed we could create two tunnels and use routing policy to separate upload and download traffic. However, the scope of this guide will not cover that.
As far as distro flavor goes, in this guide we'll be using RockyLinux. Why? It's simply where my muscle memory lies when deploying servers. If you prefer a different distro, this guide can be easily ported with some tweaks. (I love Debian too)
After provisioning our instance we need to properly harden it and configure the EPEL repo before proceeding. The reader is assumed to have the requisite skills needed to perform these tasks unguided. I highly recommend you enforce SElinux.
Once we have a hardened instance, the first thing we’ll want to do is allocate a /56 block. If you’re unfamiliar with Linode, here’s how this is done:
Once complete you should see your new /56 prefix listed under the instance’s IP addresses. For reference, the default, SLAAC derived IPv6 address associated with your instance will be used as the transit address for the /56 you allocated.
Next we need to install the requisite packages along with some diagnostic tools:
sudo dnf install -y frr wireguard-tools tcpdump traceroute iperf3
Wireguard uses PKI for authentication, so each peer will need to generate a secure key-pair.
Remote gateway instance:
sudo mkdir /etc/wireguard/keys
sudo umask 077
sudo wg genkey | tee privatekey | wg pubkey > publickey
VyOS Router:
generate pki wireguard key-pair
Remote gateway instance:
sudo vi /etc/wireguard/wg0.conf
And add the following:
[Interface]
Address = 2001:db8:1234:5600::1/56
ListenPort = 51820
PrivateKey = <Remote gateway private key>
[Peer]
PublicKey = <VyOS public key>
AllowedIPs = 2001:db8:1234:5600::/56
PersistentKeepalive = 25
Attributes explained:
AllowedIPs These are the IPs our remote gateway will accept traffic from. In our case, we only want to accept traffic from our core network.PersistentKeepalive = 25 This tells the remote gateway that the client (our VyOS router) will initiate the connection. This is necessary because our VyOS router is behind a dynamic IP.To take care of routing our /56 back to the core network we'll be using Free Range Router. On the surface this may look like overkill. However if we decide later we want to add geographic redundancy, we'll be ready to handle more advanced routing such as BGP (RFC 6996). Hint hint on what we'll be covering in the next post ;-)
Open our FRR config file for editing
sudo vi /etc/frr/frr.conf
Paste the following:
frr defaults traditional
hostname <fqdn>
service integrated-vtysh-config
interface wg0
ipv6 2001:db8:1234:5600::1/64
ipv6 route 2001:db8:1234:5600::/56 wg0
line vty
What's happening:
frr defaults traditional provides backwards compatibility.
service integrated-vtysh-config FRR has it's own CLI interface. This is simply tells it to load from a single config file so the CLI can read and modify configs.
interface wg0 This is where we declare the wg interface and it's IP to FRR
ipv6 route 2001:db8:1234:5600::/56 wg0 Creates a static interface route for our /56 to the wg tunnel.
Now we need to start wireguard and frr
sudo systemctl enable wg-quick@wg0 frr
sudo systemctl start wg-quick@wg0 frr
# Verify our services are in-fact running:
sudo systemctl status wg-quick@wg0
sudo systemctl status frr
For this guide we’ll use firewalld to set up zones and policies. I know some of you are die-hard raw nftables admins — and I get it. For complex or highly customized rule sets, I reach for raw nftables as well. That said, firewalld has matured a lot over the years. For straightforward policies, I find it quick and less prone to fat-fingered mistakes. Nowadays it's perfectly suitable even for fairly complex rule sets.
At the end of the day, one of great things about Linux is the freedom to choose the tool that fits your workflow. If direct nftables is your style, more power to you. For this setup, though, we’ll keep things simple with firewalld.
First we'll create a zone for our wireguard tunnel and any future tunnels we might add. This will allow us to easily apply rules to all our tunnels at once:
# create new zone
sudo firewall-cmd --permanent --new-zone=client-tunnels
# Add our wg0 interface to the new zone
sudo firewall-cmd --permanent --zone=client-tunnels --add-interface=wg0
Now we'll configure our intrazone forwarding policy:
# Create a new policy
sudo firewall-cmd --permanent --new-policy allowClientTun
# set our ingress and egress rules
sudo firewall-cmd --permanent --policy allowClientTun --add-ingress-zone client_tunnels
sudo firewall-cmd --permanent --policy allowClientTun --add-egress-zone client_tunnels
sudo firewall-cmd --permanent --policy allowClientTun --add-ingress-zone public
# allow wireguard traffic on our wg0 interface
sudo firewall-cmd --zone=client-tunnels --permanent --add-port=51820/udp
# reload the firewall and apply the configs
sudo firewall-cmd --reload
At this point our remote gateway peer is ready to accept Wireguard connections.
Now we need to configure our VyOS core router
config
# Set the address for the wg interface
set interfaces wireguard wg0 address '2001:db8:1234:5600::2/64'
# Configure the IPv4 address of the remote gateway
set interfaces wireguard wg0 peer <peer name> address '45.79.162.105'
# Allow connections to any IP. Obviously this is needed for internet access
set interfaces wireguard wg0 peer <peer name> allowed-ips '::/0'
# Initiate and maintain the tunnel connection starting on the VyOS router
set interfaces wireguard wg0 peer <peer name> persistent-keepalive '25'
# Configure wg0 to use the default wireguard port
set interfaces wireguard wg0 peer <peer name> port '51820'
# Set the remote gateways public key. This will be the public key we generated on the remote gateway earlier
set interfaces wireguard wg0 peer <peer name> public-key <public-key>
# Set the the private key for the router
set interfaces wireguard wg0 private-key
# Finally we need to create a static route to direct all our IPv6 traffic over the tunnel
set protocols static route6 ::/0 next-hop 2001:db8:1234:5600::2 interface 'wg0'
# Commit our progress
commit
Important! We can't forget that we're using IPv6 GUA, therefore we need firewall rules only allowing established/related traffic. Otherwise all clients will be fully exposed to the public internet.
# Set the default global policy
set firewall global-options state-policy invalid action 'drop'
set firewall global-options state-policy established action 'accept'
set firewall global-options state-policy related action 'accept'
# Create an interface group for remote gateways called RWAN (Remote WAN)
set firewall group interface-group RWAN interface 'wg0'
# Next we'll create a specific firewall ruleset for our remote tunnels
set firewall ipv6 name RWAN-IN default-action 'drop'
set firewall ipv6 name RWAN-IN rule 10 action 'accept'
set firewall ipv6 name RWAN-IN rule 10 state 'established'
set firewall ipv6 name RWAN-IN rule 10 state 'related'
# Finally we'll create a 'jump' rule. The jump in this case is traffic moving (jumping) between zones/interfaces. RWAN-IN is the firewall ruleset we want the jumping traffic to be filtered through.
set firewall ipv6 forward filter rule 100 action 'jump'
set firewall ipv6 forward filter rule 100 inbound-interface group 'RWAN'
set firewall ipv6 forward filter rule 100 jump-target 'RWAN-IN'
commit
save && exit
We should now have a working IPv6 tunnel where our IPv6 network is tunneled over the underlying IPv4 network. We can verify this by pinging hosts from either side of the tunnel. To benchmark our tunnels performance a simple iperf3 test will give us a good idea of our maximum throughput.
You'll likely notice that despite having internet, a number of sites, like Github for instance, don't work. As you're probably aware, a large slice of the internet has yet to embrace IPv6. So for part 2 we'll setup DNS64 and NAT64 on the remote gateway to ensure we still have connectivity to the IPv4 only internet. We'll also implement NAT46 service so that self hosted services can be reached via networks that are IPv4 only.
If you'd like to get $100 worth of Linode for free you can use my referral link below. Please note that I am not endorsing or promoting Linode, I simply feel they are the best provider for this particular project.
Referral LInk: https://www.linode.com/lp/refer/?r=a49dce640b79d4ca4ced13c0bbd9fddf9d896ab9