In this blog post, we continue the discovery of virtual networking in Microsoft Azure including in-depth coverage of VNet route tables, subnets, and the process of route selection.
Microsoft Azure Administrator Associate Exam (AZ-104) includes these topics in its blueprint:
- configure private and public IP addresses, network routes, network interface, subnets, and virtual network
- create and configure VNet peering
We covered basic concepts listed in the first bullet point in the previous article.
This article starts with discussing VNet peering and then dives into additional topics not covered in the first part of the article, such as Azure route tables, network routes, and best route selection process.
VNet Peering
VNet peering connects two VNets together, so resources, such as virtual machines deployed in different networks can communicate with each other. VNet peering provides high-speed, private connectivity using Microsoft backbone.
The peering VNets can be in the same or different regions. If VNets are in the same region, the communication performance between VMs via VNet peering matches the speed and latency of Intra-VNet connectivity.
VNet peering is configured by the provisioning of a peering link in each of the VNets. If VNets are under the same account, the second link can be automatically created. In all other cases, each side needs to be provisioned separately using Resource IDs of the peer VNet. In the lab section, we will demonstrate both options.
For the practical part of this blog post, we will use the lab topology shown in Figure 1.
Create VNet Peering Step-by-Step
Let’s start this example by creating 2 sets of VNets and subnets. Both VNets will be under the same subscription and in the same region. The detailed procedure on how to create a VNet is available here.
To create a peering open one of the created VNets and select Peerings in the side menu and click Add.
On the “Add peering” page we configured the following settings:
- Descriptive name of the peering link on the source VNet side. This is the VNet-A that we selected in the previous screenshot.
- Peering virtual network using a drop-down box to select VNET-B, which is accessible from our account.
- A descriptive name for the peering that will be automatically added to the target VNet.
To create a peering to VNet that is not in our account, select a checkbox called “I know my resource ID”. Then, instead of Steps 2 and 3, enter the resource ID of the partner VNet. The operation must be repeated in the peering VNet.
To get the Resource ID of a VNet, navigate to the Properties menu and press the copy to clipboard button next to Resource ID field.
Once peering links are created we can navigate to VNET-A and to VNET-B to validate settings on each side. The screenshot in Figure 5 shows the information on peering status and provisioned state and address space behind the peer. The page also has several configurable parameters.
Virtual network access setting controls if the IP range of a peer is classified as a VirtualNetwork service tag. Both inbound and outbound traffic is allowed between the networks with this tag.
Forwarded traffic, if enabled, permits entry of the traffic that is not originated in the peer VNet. Such traffic usually comes via VPN or ExpressRoute Gateways in the peer VNet. This setting is often used with combination with other two checkboxes:
- Allow gateway transit
- Use remote gateway
Allow gateway transit allows the peer to use the local VNet’s gateways. Such peering VNet must have “Use remote gateway” checkbox enabled on the peering toward the VNet containing a gateway.
Deploy VMs and Testing
Let’s deploy VMs into corresponding VNets and Subnets according to Figure 1. Use this article for information on how to do it.
Let’s validate IP address of VM-A by running Windows ipconfig command.
Let’s test that we can ping the VM-B from VM-A. To make this test work we had to disable Windows Firewall on VM-B. In the production environment, it is recommended to keep Windows Firewall enabled, and create a specific rule that allows ICMP through.
Notice that the traceroute doesn’t display any hops between two VMs. This is related to how Azure operates on the back-end. It is different from how packet forwarding works in the traditional data center, where traceroute displays a default gateway as the first hop.
Let’s check Azure VMs security rules and effective routes. Click on the Networking menu of the VM-A. The displayed window has tabs for Inbound and Outbound port rules. They are applied via the Network security group to the network interface, which we will cover in one of the future articles.
In the default configuration, both Inbound and Outbound security rules include a rule (#65000) that allows traffic with the source and destination tagged with VirtualNetwork.
As we discussed earlier, peering’s Access Control Setting checkbox, if enabled, classifies IPs in the peering VNet as part of VirtualNetwork. This effectively allows the traffic to be allowed on Azure level. Guest OS configuration is still required to allow the traffic in.
Click on the Network Interface name to proceed to the next step, red rectangle #2 in Figure 8.
Once the Network interfaces window opens, click on the Effective routes menu, as shown in Figure 9. The routing table has a route to 10.1.0.0/16 with the next hop type of VNet peering.
Routes and route tables
When you create a subnet, it has a default implicit route table associated with it. The Azure portal will not display such tables, and an administrator cannot add custom routes into them. The default route tables contain only system routes, which are automatically added by Azure.
To define a custom route an administrator must create a custom route table and associate it with the subnet. The custom route table will also have automatically added system routes, but these routes can be overridden.
To display routes, use the Effective routes tab of a network adapter, as described in the previous section, and shown in Figure 9. The system routes have Default as their source.
Figure 10 shows a conceptual diagram of relationships between subnets and route tables. Each subnet can have one route table associated with it. A route table, however, can be associated with multiple subnets at the same time.
(Mandatory) Default System Routes
Azure populates every route table, including the custom route tables, with the following routes.
| Route | Next hop type | Comments |
|---|---|---|
| Local VNet's address space | Virtual network | In our example, 10.0.0.0/16 for subnets in VNET-A and 10.1.0.0/16 in VNET-B. Note that no subnet-specific routes are created |
| 0.0.0.0/0 | Internet | Default route points to Internet. Traffic to Azure public services uses this route, however, it doesn't traverse Internet and stays within Microsoft Azure network |
| 10.0.0.0/8 | None | Private RFC 1918 range |
| 100.64.0.0/10 | None | Similar to RFC 1918 ranges (reserved for Shared Address Space, see RFC 6598) |
| 192.168.0.0/16 | None | Private RFC 1918 range |
Table 1. Mandatory Azure System Routes
The default system routes from Table 1 ensure that there is communication within VNet and all traffic is sent to the Internet, with the exception of private address space. Traffic to 3 private ranges is basically discarded.
Interestingly, 172.16.0.0/12 (also part of Private RFC 1918 range) is not among the default null routes. Traffic to this range matches 0.0.0.0/0 and sent to the Internet next hop. It is not routable on the Internet, so it should be discarded somewhere upstream.
(Optional) Default System Routes
Certain routes are only added by Azure into route tables when there is a specific configuration is enabled. For example, in Figure 9 route to a peered VNet was automatically added by Azure. Table 2 lists such routes that can appear in a route table.
| Route | Next hop type | Comments |
|---|---|---|
| Peered VNet's CIDR block | VNet Peering | Created in route tables of every subnet of a VNet |
| Public IP addresses of Azure Services | VirtualNetworkServiceEndpoint | Created only in a route table of the subnet that has service endpoint enabled |
Table 2. Optional Azure System Routes
Network Gateway Propagated Routes
Figure 11 shows a sample topology with a Virtual Network Gateway.
Virtual Network Gateways provide connectivity to on-premises and external networks connected over VPN or ExpressRoute. The routes to remote networks can (with ExpressRoute gateways – must) be exchanged using Border Gateway Protocol (BGP).
A gateway propagates this information by installing learned routes into route tables. The next-hop type in this case is set to Virtual network gateway.
The propagation is enabled by default, and can be disabled for custom route table using a setting called “Virtual network gateway route propagation”.
Custom Routes
In addition to Network Gateway Propagated routes, an administrator can manually create custom ones. Custom routes are preferred in Azure VNet Route Selection Process. We will discuss this in detail in one of the sections below.
To perform this operation a route table must be first created and associated with a subnet. We will demonstrate how to do it in the practical part of this section.
A custom route is identified by its name and requires address prefix and next-hop type to be selected. There are 5 types of next-hops available for Azure custom route:
- Virtual network gateway
- Virtual network
- Internet
- Virtual appliance
- None
Virtual network gateway next-hop type can be used only for VPN gateways. ExpressRoute must use BGP-based network propagation, so no custom routes can be created to use ExpressRoute Gateway. Notice that there is also no VNet Peering type of the next-hop, which can be created only by Azure. You can, however, create a route that will be pointing to Virtual appliance with Next-hop IP in a peered VNet.
Virtual network next-hop type is used in topologies when you want to make traffic flowing between subnets within a single VNet to go via the virtual appliance. For example, traffic from one subnet to another within a VNet by default is routed via the default system route as discussed in the system routes section. You can override this by selecting the next hop as a virtual appliance. To keep direct traffic flow within the subnet, you can create a custom route with the next hop of the virtual network for the subnet prefix.
Internet next-hop can be used in situations when the default route is overridden and, for example, is sent to the on-premises network to use corporate Internet breakout. To send a subset of traffic via Azure Internet, you create a route for a specific prefix to go via Azure Internet.
Virtual appliance next-hop type requires specifying an IP address of the appliance which will receive traffic. Useful for introducing firewalls and traffic analysis devices.
None next-hop type drops the matching traffic.
Create a Custom Route Table Step-by-Step
Start typing in “route table” in Microsoft Azure search bar and click on Route tables service. Click on the Add button to create a route table. Fill-in required parameters, including route table name, resource group, and press Create.
As the next step, we will associate the created route table with Subnet A. Click on the newly created route table and then on the Subnets button on the side menu. Click on the associate button and select the VNet and the Subnet. After this step, SUBNET-A-123 will have a default implicit route table replaced with the custom one.
At this stage, the content of the new custom route table will be exactly the same as the routes in the implicit route table. Let’s create a custom route by pressing the Routes button on the side menu and then clicking Add.
We will create a test route that will discard any packets sent to the Google DNS server IP address of 8.8.8.8/32. To accomplish this, select None as the Next hop type. There are also different types of next-hop types available as we discussed in the theory section.
Once we press ok, let’s validate that the route was added and we can also confirm from the virtual machine if the connectivity to 8.8.8.8 no longer works. To validate routes in the custom table, we can use Effective routes accessible via network interface menu or route table menu directly. To display routes via both options, we need to have a network interface attached to a powered-on VM in a subnet that the route table associated with.
The next screenshot shows that connectivity to 8.8.8.8/32 no longer works after we applied a custom route to None.
Azure Best Route Selection Process
The final topic of this article is how the best matching route is selected in Azure.
The route selection is intuitive if you follow the rule of not advertising more specific (or the same length) routes that are used in Azure using BGP from an on-premises network or remote VPN networks. If this restriction is maintained, the route selection can be summarized as the following list with the preferred conditions on the top:
1. Longest match
2. User-defined routes
3. BGP-derived
4. System routes
Condition is violated if a network gateway injects via BGP a network that is more specific than (or the same as) VNet address space or a peered VNet prefix. In this case, the routes with the next-hop type of Virtual Network or VNet Peering are preferred over more specific BGP routes. We recommend to design network, so prefixes from Azure VNet address space is not used anywhere else on the network. As a workaround in migration scenarios, remote networks can be network address translated (NAT-ed) to the non-overlapping range.
Let’s consider a few simple examples with 2 Azure VNets with the address prefixes of 10.0.0.0/16 and 10.1.0.0/16. All other networks do not overlap with VNets’ address spaces.
To identify the best route for a specific destination there are several steps, with each minimizing number of candidate routes to select from. Once a single route is left the process stops and this route is used.
In the diagram below, a packet with the destination of 10.1.0.20 matches 3 routes of different lengths. The most specific route (with the longest prefix length of /16) is selected.
The next example shows a packet with a destination of 10.30.1.34. After selecting all matching candidates, we have 4 routes left. In the second step, we keep only the longest prefix routes (/24). Out of two remaining routes, we know that user-defined routes are more preferred. The single route that points to a virtual appliance with an IP address of 10.0.0.100 is used to forward the traffic.
The final example is for a packet with the destination of 192.168.1.100. There are 2 routes left after step 2 with the same length. As BGP routes are preferred over system routes, the packet will be sent to a Virtual network gateway.
