Today’s topic: the overlapping IPs
The topic is huge. You might hit this problem everywhere. What are the major reasons why organizations have this problem? I would say:
- Lack of proper IPAM to begin with
- Merges and acquisitions
- Rarely, but still, the company might be so large that, even with proper IPAM in place, there will be not enough usable private IP CIDRs at some point.
Solution: NAT in the Public Cloud
The solution is the same as in On-Prem/DC/Branch environment. Use NAT. But how to set up NAT in Public Cloud (regardless which one, AWS, Azure, GCP, OCI, Alibaba)? Or how to do it across the Public Clouds? The answer is obvious: leverage Aviatrix feature-set.
As you might know, the Aviatrix Network and Security Multi-Cloud Architecture brings many different benefits. One of them is possibility of doing NAT of course.
Aviatrix NAT in Action
LAB Scenario
The customer has a presence in Azure (let’s say Region#1). Additionally, it uses Region#2 as a DR-Region (in case of an outage in Region#1). The customer has acquired a company, let’s call it “EnvB” that owns some VNETs in Region#3.
The scenario might be also that Region#1 and Region#2 would be in one Public Cloud (e.g. AWS) and the newly acquired company Region#3 in GCP. It would not change anything when it comes to our discussion because Aviatrix works the same way in each Public Cloud. Remember: Aviatrix is Cloud-Agnostic.
Lab Details
Let’s focus on the CIDRs and VNETs deployed in the environment:
- Region#1 (“Primary”:
- Shared Services Spoke VNET (10.40/16)
- Private Spoke VNET (10.50/16)
- Region#2 (“DR”):
- Shared Services DR VNET (10.40/16)
- Private DR Spoke VNET (10.50/16)
- Region#3 (“EnvB”)
- Spoke-1 VNET (10.40/16)
- Spoke-2 VNET (10.151/16)
- Spoke-3 VNET (10.152/16)
There is a Shared Services VNET in Region#1. The goal is to have the connectivity from “EnvB” (there are three Spoke VNETs there) to Shared Services VNET. But wait… one of the EnvB Spoke-VNETs (Spoke-1) and Shared Services VNET use the same CIDR of 10.40/16. We need a NAT.
The following VMs have been deployed for verification purposes:
- Shared-VM in Shared Spoke VNET
- Shared-DR-VM in Shared DR Spoke VNET
- Test-VM in Spoke-1 VNET
Guess what, all of them are assigned the same Private IP of 10.40.16.4.
Part 1 – Solving Overlapping IPs Issue
Let’s focus on the connectivity from Spoke-1 VNET to Shared Services VNET. Two things must be done here:
- SNAT on Spoke-1 VNET side – to hide the real source IP address of Test-Spoke1-VM
- Spoke-1 VNET will use 10.240/16 prefix for NAT.
- Test-Spoke-1 VM will be hidden behind 10.240.16.4.
- DNAT on Shared Services VNET
- Shared Services Spoke VNET will use 10.41/16 prefix for NAT.
- Shared-VM will be known to Spoke-1 VNET as 10.41.16.4.
Configuration on Spoke-1 Gateways
Aviatrix Spoke Gateways are the components where NAT will be configured.
We will leverage the “Custom NAT” Aviatrix feature. The configuration is done under the GATEWAY menu. Please select the proper Gateway and choose the “Edit” option.
Where:
1 – Is a real source IP address of our Test-Spoke1-VM
2 – Is a NATed destination IP address of Shared-VM. It is not a real IP address of the Shared-VM
3 – Is the connection between Spoke1-Gateways and Transit-EnvB-Gateways
4 – Is a SNAT IP that will be used as a source of the packet outside of the Spoke1-VNET
Please notice the same translation must be configured on HA Spoke-1 Gateway.
Configuration on Shared-Services Spoke Gateways
Now, let’s configure Destination NAT on Shared Services Spoke Gateways:
Where:
1 – Is the source IP address of the packet after SNAT
2 – Is the destination IP address of Shared-VM. It is NOT a real IP address but it is a NAT-IP.
3 – Is the connection between Shared-Spoke-Gateways and Transit-Gateways
4 – Is a real IP address of Shared-VM
Spoke-1 Gateways (Region#3) advertise the NAT CIDR 10.240/16.
At the moment the NAT CIDR 10.41/16 is only advertised by Shared-Services Spoke Gateways in “primary” Region#1.
Such advertisement is not done by “Shared-Services-DR”.
All looks good. But there is one gotcha here. Aviatrix Spoke Gateways and Transit Gateways form an ActiveMesh. It is a full mesh of connections between them. This is the reason why we might face an issue with asymmetric routing. The way is required to ensure the symmetry of the initial flow and the response flow. The solution: Source NAT on Shared-Services Gateways. The SNAT is going to look like below:
Where:
1 – The real prefix inside Shared Services VNET where the destination Shared-VM resides
2 – eth0 interface, the interface attached to subnet in Spoke VNET. Let’s call it an “internal” interface.
3 – None
4 – The IP address of that (Primary) Shared-Services Gateway eth0 interface
The SNAT rule is also required on HA-Shared-Services Gateway. The only difference is the “SNAT IPS” address. In this case, it is going to be the IP address of the HA-Shared-Services Gateway eth0 interface:
What do those 2 SNAT entries (one on Primary Shared-Services Spoke Gateway and the other on HA Shared-Services Spoke Gateway) perform? They ensure that the return traffic will be routed to the Spoke Gateway where the initial traffic landed.
Testing the connectivity from Spoke1-VM to Shared Services-VM
Let’s test whether the connectivity works. We must use NATed 10.41.16.4 IP address as a destination of course. All looks good. The connectivity is there.
Part 2 – Disaster Recovery Setup
As mentioned at the beginning, the customer leverages Region#2 as a Disaster Recovery. What is even more important, it is a “Hot DR”. Why “Hot”? It means the customer has all the VNETs and Aviatrix Gateways already deployed in DR-Region#2. What is more, the Transit Gateways of DR Region#2 peer with Transit Gateways in “primary” Region#1
Depending on the application, in theory, we could also have the mirror VMs deployed in Region #2, which will be ready to take the load in case Region#1 fails. I have deployed Shared-DR-VM in Shared Services DR Spoke VNET. The IP address of the VNET (10.40/16) and the DR-VM (10.40.16.4) are the same as in Primary Region #1.
Additionally, I have created the same DNAT and SNAT rules on “Shared Services DR” Spoke Gateways as the ones present on (Region#1) “Shared Services” Spoke Gateways. There is also one more thing to be considered. We want to have a Disaster Recovery environment in Region#2. In normal conditions, Region#2 must not get any traffic. Though as soon as Region#1 fails, Region#2 must take all the traffic. How to ensure the traffic goes to Region#1 and not Region#2? The answer is simple. Aviatrix Gateways leverage BGP, so we can configure AS Path Prepend in the desired way.
AS Path Prepend on Aviatrix Gateways
Let’s see how AS Path Prepend could be done on Transit DR Gateways (go to ULTI-CLOUD TRANSIT -> Advanced Config -> choose Transit Gateway). We will put two additional ASN = 65000 when advertising prefixes to BGP peers. Please notice that AS Path Prepend can be done per Connection basis as well.
Now we can start advertising the NAT CIDR 10.41/16 from “Shared-Services-DR” Spoke Gateways.
Please remember that Aviatrix provides the possibility not only to check the routing table on the Gateway but also all the routing information (“Route Info DB Details”) received by the Gateway. We will verify now how the routing information looks like for prefix 10.41/16 (the one used by NAT on Shared-Services Spoke Gateways and Shared-Services-DR Spoke Gateways).
If we look at the “Route Info DB Details” on Transit-EnvB Gateways, we can see that there are two paths: one via Transit Gateways in “primary” Region#1 and the other towards Transit-DR Gateways in Region#2. Region#1-one is chosen as the best route because there is AS Path Prepend done by Transit-DR Gateways as shown below:
Region#1 “outage”
Let’s see what happens if “Shared-Services” VNET in the primary Region#1 is no longer available.
The best path towards 10.41.0.0/16 has been dynamically updated. Keep in mind, the backup route (with AS Path Prepend) was already present in “Route Info DB” before an outage.
The constant PING was executed from Test-Spoke1-VM towards 10.41.16.4 to check how many PINGs are lost during the switchover from Region#1 to Region#2. One PING was lost.
Summary
Aviatrix can be used for many purposes. In this blog article, I have presented the scenario of a customer having environments in different regions (might be different Clouds as well). The problem was, those regions utilized the same IP CIDR range. The following has been used:
- SNAT/DNAT on Aviatrix Spoke Gateways to provide connectivity between VNETs with Overlapping IP CIDRs
- BGP AS Path Prepend on Aviatrix Transit Gateways to influence the routing decisions and to set up the “Hot” Disaster Recovery environment (ready for primary Region failure)
I hope it was informative for you!