What is the underlay and Overlay network?
When it comes to Nuage-SDWAN there are two types of networks that we have to use. First one is called the underlay network and the second one is called the overlay network.
The underlay network is nothing but our traditional network which would help us to connect all the network equipment. Once the underlay is up and it’s reachable from point A to Point B then we can create a Network on top of it which is known as the Overlay network.
The Overlay network is not something new, VPN is an example of an Overlay network. On VPN you can use internet routable public IP space to connect the network, once the connection establishes the private IP can be exchanged on top of the public network that’s called the Overlay network.
When the underlay is down the overlay would also go down, however, when the overlay is down the underlay won’t go down.
For this reason, there would be multiple underlay path for the overlay network to traverse. I covered more on this topic here.
In this lab, we are going to create an underlay network with which the Nuage SD-WAN can function.
At last, we will create the Overlay network on top it using VXLAN tunnels.
- Underlay Network diagram
- How do we represent the Underlay Network for Nuage SD-WAN?
- Creation of VLAN’s on the switch
- Creation of VLAN’s on the Linux Server. (Linux sub interface configuration)
- Creation of Bridge interfaces on the Linux Server.
1. Underlay Network diagram
Let’s look at the topology that we are going to work for this lab.
As you can see, there are multiple devices connected with multiple underlay networks such as
- Management (10.1.5.0/24) for the network device management.
- Control/Data (10.1.6.0/24) for the VSC control and data traffic.
- and the Internet (192.168.0.0/16) to represent the WAN for NSG’s.
All the devices we would be installed as a VM on a KVM server and the server intern connected to the Layer 3 switch for routing traffic between the devices.
2. How do we represent the Underlay Network for Nuage SD-WAN?
In our lab setup, we are connecting the Server directly to the Switch, and we can represent the Underlay network for Nuage SD-WAN by creating multiple VLANs on the switch and do inter-VLAN routing.
You also need to create the sub interfaces that represent the same VLAN on the server and then bridge interfaces for the VM’s to connect to the host underlay network in Linux KVM.
Make sure the link between the switch and the server is a Trunk and the below VLANs are permitted.
List of VLANs that we are going to use.
The default gateway is the Layer3 VLAN IP known as SVI.
Note : In real world scenarios the WAN IP would be a public IP not private.
| VLAN Number | Description | Subnet | Default Gateway |
| 5 | Management | 10.1.5.0/24 | 10.1.5.1 |
| 6 | Control/Data | 10.1.6.0/24 | 10.1.6.1 |
| 11 | WAN1-NSG1 | 192.168.11.0/24 | 192.168.11.1 |
| 12 | WAN2-NSG1 | 192.168.12.0/24 | 192.168.12.1 |
| 13 | WAN3-NSG2 | 192.168.13.0/24 | 192.168.13.1 |
| 14 | WAN4-NSG2 | 192.168.14.0/24 | 192.168.14.1 |
First, we can create the layer 3 VLAN one by one on the switch. Once we finished the VLAN creation on the switch, we can go ahead and create the underlay network on the server using sub-interfaces, and then bridge interfaces.
Below are the Linux Server subinterfaces and its bridge name.
| Linux Sub interface | Bridge Name |
| ens1f1.5 | mgmt5 |
| ens1f1.6 | cntrl6 |
| ens1f1.11 | wan1 |
| ens1f1.12 | wan2 |
| ens1f1.13 | wan3 |
| ens1f1.14 | wan4 |
Note: usually, bridge interface name starts with br1, br2.etc, but for this lab we are using the name that represents its functions, which would provide more clarity.
3. Creation of VLAN’s on the switch
Here I am using HPE comware switch as the inter VLAN routing and gateway for the VM if you have a different vendor such as cisco, juniper then the config would be identical except for the differences in the commands.
Step.1 Create the VLAN that we are going to use.
<GETLABSDONE-SW1>sys System View: return to User View with Ctrl+Z. [GETLABSDONE-SW1]vlan 5 [GETLABSDONE-SW1-vlan5]desc [GETLABSDONE-SW1-vlan5]description Management
Step2. Creation of Vlan interface for inter VLAN routing.
[GETLABSDONE-SW1-vlan5]interface vlan 5 [GETLABSDONE-SW1-Vlan-interface5]de [GETLABSDONE-SW1-Vlan-interface5]description Management [GETLABSDONE-SW1-Vlan-interface5]ip add [GETLABSDONE-SW1-Vlan-interface5]ip address 10.1.5.1 24
- Save the config
[GETLABSDONE-SW1-Vlan-interface5]save force Validating file. Please wait... Saved the current configuration to mainboard device successfully.
Step3. Now you can go ahead and create the rest of the Layer 3 VLAN interfaces on the switch, after the VLAN creation, It would look like below.
Vlan5 up up 10.1.5.1 -- Management Vlan6 up up 10.1.6.1 -- Control/Data Vlan11 up up 192.168.11.1 -- WAN1-NSG1 Vlan12 up up 192.168.12.1 -- WAN2-NSG1 Vlan13 up up 192.168.13.1 -- WAN3-NSG2 Vlan14 up up 192.168.14.1 -- WAN4-NSG2
4. Creation of VLAN’s on the Linux Server. (Linux sub interface configuration)
We now need to create VLAN’s on the Linux Server (Redhat/Centos) to communicate to the VLANs on the switch.
Step1. Type “ifconfig” or “ip addr” to see the name of the physical interface connected to the switch.
In my case the name of the interface is ens1f1
Step2. To create sub interface that represents the vlan follow the below steps.
- Goto network configuration folder in linux.
cd /etc/sysconfig/network-scritps
- Type vi ifcfg-ens1f1.5 this will create the file called ifcfg-ens1f1.5
- Add the below config to it. We are also adding bridge interface that we are going to use with the subinterface. The bridge interface ‘mgmt5’ and others we are going to create on the further steps.
DEVICE=ifcfg-ens1f1.5 NAME=ifcfg-ens1f1.5 TYPE=Ethernet BOOTPROTO=none ONBOOT=yes VLAN=yes BRIDGE=mgmt5
Step3. Create the sub-interface for other VLAN’s as well.
The easy way you can do this insteady of copying and pasting the config file is by using the copy command on linux.
For example, to create the sub interface 6
cp ifcfg-ens1f1.5 ifcfg-ens1f1.6 vi ifcfg-ens1f1.6
- Modify the values according the sub interface 6, like below.
DEVICE=ifcfg-ens1f1.6 NAME=ifcfg-ens1f1.6 BOOTPROTO=static ONBOOT=yes VLAN=yes BRIDGE=cntrl6
That was easy, wasn’t it ?, now you may continue came steps for the other sub interfaces as well.
Step4. Once you finished the configuration of the sub interaces type ls to see the list of interfaces and you will be able to see the interface configuration as below.
ifcfg-ens1f1 ifcfg-ens1f1.11 ifcfg-ens1f1.12 ifcfg-ens1f1.13 ifcfg-ens1f1.14 ifcfg-ens1f1.5 ifcfg-ens1f1.6
5. Creation of Bridge interfaces on the Linux Server.
We have configured the subinterfaces on the server. Now you need to create bridge interfaces that would connect directly to the VM’s under the hypervisor
Step1. Verify the current bridge configuration on the server by typing brclt show
[root@getlabsdone ]# brctl show bridge name bridge id STP enabled interfaces virbr0 8000.525400b4df3a yes virbr0-nic [root@getlabsdone ]#
As you can see we don’t have any bridge interfaces configured, let’s go ahead and do that now.
Step 2. Creation of the bridge interface in linux.
In the terminal, create an interface configuration file called ifcfg-mgmt5 by typing the below for the management VLAN 5.
vi /etc/sysconfig/network-scripts/ifcfg-mgmt5
- Add the below config to it.
DEVICE=mgmt5 STP=yes BRIDGING_OPTS=priority=32768 TYPE=Bridge NAME=mgmt5 ONBOOT=yes
Step3. Create rest of the bridge interfaces, you may use the copy commands to copy the config from one file to another and modify the values according to the respective interfaces.
- Once finished, it would look like below.
ifcfg-cntrl6 ifcfg-mgmt5 ifcfg-wan1 ifcfg-wan2 ifcfg-wan3 ifcfg-wan4
Step4. Awesome! we are almost done.
You have now created the sub-interfaces, and the bridge interfaces now let’s go ahead and restart the network services on the Linux server by typing the below to take this effect.
[root@Getlabsdone network-scripts]# systemctl restart network
Step5. Verification of the Linux bridge configuration
To verify the bridge interface attached to the physical interface, you can type the below command. It would show the bridge interface properties.
[root@Getlabsdone network-scripts]# brctl show bridge name bridge id STP enabled interfaces cntrl6 8000.d06726bb1515 yes ens1f1.6 mgmt5 8000.d06726bb1515 yes ens1f1.5 virbr0 8000.525400b4df3a yes virbr0-nic wan1 8000.d06726bb1515 yes ens1f1.11 wan2 8000.d06726bb1515 yes ens1f1.12 wan3 8000.d06726bb1515 yes ens1f1.13 wan4 8000.d06726bb1515 yes ens1f1.14
Installation of Nuage SD-WAN underlay network is complete, next lets go ahead and install the VSD.