Azure VNet Route Selection

In this blog post we continue discovery of the virtual networking in Microsoft Azure.

Microsoft Azure Administrator Associate Exam (AZ-104) includes these topics in its blueprint:

• configure private and public IP addresses, network routes, network interface, subnets, and virtual network
• create and configure VNet peering

We covered basic concepts listed in the first bullet point in the previous article (url).

This article starts with discussing VNet peering, and also then dives into additional topics not covered in the first part of the article, such as Azure route tables, network routes and best route selection process.

VNet Peering

VNet peering connects two VNets together, so resources, such as virtual machines deployed in different networks can communicate with each other. VNet peering provides high-speed, private connectivity using Microsoft backbone.

The peering VNets can be in the same or different regions. If VNets are in the same region, the communication performance between VMs via VNet peering matches the speed and latency of intra-VNet connectivity.

VNet peering is configured by provisioning of a peering link in each of the VNets. If VNets are under the same account, the second link can be automatically created. In all other cases, each side needs to be provisioned separately using Resource IDs of the peer VNet. In the lab section we will demonstrate both options.

For practical part of this blog post we will use the lab topology shown in Figure 1.

Create VNet Peering Step-by-Step

Let’s start this example with creating 2 sets of VNets and subnets. Both VNets will be under the same subscription and in the same region. The detailed procedure on how to create a VNet is available here (url). To create a peering open one of the created VNets and select Peerings in the side menu and click Add.

On the “Add peering” page we configured the following settings:

1. Descriptive name of peering link on the source VNet side. This is the VNet-A that we selected in the previous screenshot.
2. Peering virtual network using drop-down box to select VNET-B, which is accessible from our account.
3. Descriptive name for the peering that will be automatically added in the target VNet.

To create a peering to VNet that is not in our account, select a checkbox called “I know my resource ID”. Then, instead of Steps 2 and 3, enter resource ID of the partner VNet. The operation must be repeated in the peering VNet.

To get the Resource ID of a VNet, navigate to the Properties menu and press the copy to clipboard button next to Resource ID field.

Once peering links are created we can navigate to VNET-A and to VNET-B to validate settings on each side. Screenshot on Figure 5 shows information on peering status and provisioned state, and address space behind the peer. The page also has several configurable parameters.

Virtual network access setting controls if IP range of a peer is classified as VirtualNetwork service tag. Both inbound and outbound traffic is allowed between the networks with this tag.

Forwarded traffic, if enabled, permits entry of the traffic that is not originated in the peer VNet. Such traffic usually comes via VPN or ExpressRoute Gateways in the peer VNet. This setting is often used with combination with other two checkboxes:

• Allow gateway transit
• Use remote gateway

Allow gateway transit allows the peer to use the local VNet’s gateways. Such peering VNet must have “Use remote gateway” checkbox enabled on the peering toward the VNet containing a gateway.

Deploy VMs and Testing

Let’s deploy VMs into corresponding VNets and Subnets according to Figure 1. Use this article for information on how to do it (url).

Let’s validate IP address of VM-A by running Windows ipconfig command.

Let’s test that we can ping the VM-B from VM-A. To make this test work we had to disable Windows Firewall on VM-B. In the production environment it is recommended to keep Windows Firewall enabled, and create a specific rule that allows ICMP through.

Notice that traceroute doesn’t display any hops between two VMs. This is related to how Azure operates on the back-end. It is different to how packet forwarding works in the traditional data center, where traceroute displays a default gateway as the first hop.

Let’s check Azure VMs security rules and effective routes. Click on the Networking menu of the VM-A. The displayed window has tabs for Inbound and Outbound port rules. They are applied via Network security group to the network interface, which we will cover in one of the future articles.

In the default configuration both Inbound and Outbound security rules include a rule (#65000) that allows traffic with the source and destination tagged with VirtualNetwork.

As we discussed earlier, peering’s Access Control Setting checkbox, if enabled, classifies IPs in the peering VNet as part of VirtualNetwork. This effectively allows the traffic to be allowed on Azure level. Guest OS configuration is still required to allow the traffic in.

Click on the Network Interface name to proceed to the next step, red rectangle #2 in Figure 8.

Once the Network interfaces window opens, click on Effective routes menu, as shown in Figure 9. The routing table has a route to 10.1.0.0/16 with the next hop type of VNet peering.

Routes and route tables

When you create a subnet, it has a default implicit route table associated with it. Azure portal will not display such tables, and an administrator cannot add custom routes into them. The default route tables contain only system routes, which are automatically added by Azure.

To define a custom route an administrator must create a custom route table and associate it with the subnet. Custom route table will also have automatically added system routes, but these routes can be overridden.

To display routes, use the Effective routes tab of a network adapter, as described in the previous section and shown in Figure 9. The system routes have Default as their source. Figure 10 shows conceptual diagram of relationships between subnets and route tables. Each subnet can have one route table associated with it. A route table, however, can be associated with multiple subnets at the same time.

(Mandatory) Default System Routes

Azure populates every route table, including the custom route tables, with the following routes.

The default system routes from Table 1 ensure that there is a communication within VNet and all traffic is sent to Internet, with the exception of private address space. Traffic to 3 private ranges is basically discarded.

Interestingly, 172.16.0.0/12 (also part of Private RFC 1918 range) is not among the default null routes. Traffic to this range matches 0.0.0.0/0 and sent to the Internet next hop. It is not routable on Internet, so it should be discarded somewhere upstream.

(Optional) Default System Routes

Certain routes are only added by Azure into route tables when there is a specific configuration is enabled. For example, in Figure 9 route to a peered VNet was automatically added by Azure. Table 2 lists such routes that can appear in a route table.

Network Gateway Propagated Routes

Figure 11 shows an example topology with a Virtual Network Gateway.

Virtual Network Gateways provide connectivity to on-premises and external networks connected over VPN or ExpressRoute. The routes to remote networks can (with ExpressRoute gateways - must) be exchanged using Border Gateway Protocol (BGP).

A gateway propagates this information by installing learned routes into route tables. The next hop type in this case is set to Virtual network gateway.

The propagation is enabled by default, and can be disabled for custom route table using setting called “Virtual network gateway route propagation”.

Custom Routes

In addition to Network Gateway Propagated routes, an administrator can manually create custom ones. To perform this operation a route table must be first created and associated with a subnet. We will demonstrate how to do it in the practical part of this section.

A custom route is identified by its name and requires address prefix and next hop type to be selected. There are 5 types of next-hops available for Azure custom route:

• Virtual network gateway
• Virtual network
• Internet
• Virtual appliance
• None

Virtual network gateway next-hop type can be used only for VPN gateways. ExpressRoute must use BGP-based network propagation, so no custom routes can be created to use ExpressRoute Gateway. Notice that there is also no VNet Peering type of the next-hop, which can be created only by Azure. You can, however, create a route that will be pointing to Virtual appliance with Next-hop IP in a peered VNet.

Virtual network next-hop type is used in topologies when you want to make traffic flowing between subnets within a single VNet to go via virtual appliance. For example, traffic from one subnet to another within a VNet by default is routed via default system route as discussed in system routes section. You can override this by selecting next hop as virtual appliance. To keep direct traffic flow within subnet, you can create a custom route with the next hop of virtual network for the subnet prefix.

Internet next-hop can be used in situation when the default route is overridden and, for example, is sent to the on-premises network to use corporate Internet breakout. To send subset of traffic via Azure Internet, you create a route for a specific prefix to go via Azure Internet.

Virtual appliance next-hop type requires specifying an IP address of the appliance which will receive traffic. Useful for introducing firewalls and traffic analysis devices.

None next-hop type drops the matching traffic.

Create a Custom Route Table Step-by-Step

Start typing in “route table” in Microsoft Azure search bar and click on Route tables service. Click on Add button to create a route table. Fill-in required parameters, including route table name, resource group and press Create.

As the next step we will associate the created route table with Subnet A. Click on the newly create route table and then on Subnets button on the side menu. Click on the associate button and select the VNet and the Subnet. After this step SUBNET-A-123 will have default implicit route table replaced with the custom one.

At this stage the content of the new custom route table will be exactly the same as the routes in the implicit route table. Let’s create a custom route by pressing the Routes button on the side menu and then clicking Add.

We will create a test route that will discard any packets sent to the Google DNS server IP address of 8.8.8.8/32. To accomplish this, select None as the Next hop type. There are also different types of next hop types available as we discussed in the theory section.

Once we press ok, let’s validate that the route was added and we can also confirm from the virtual machine if the connectivity to 8.8.8.8 no longer works.

To validate routes in the custom table, we can use Effective routes accessible via network interface menu or route table menu directly. To display routes via both options, we need to have a network interface attached to a powered on VM in a subnet that the route table associated with.

The next screenshot shows that connectivity to 8.8.8.8/32 no longer works after we applied custom route to None.

Azure Best Route Selection Process

The final topic of this article is how the best matching route is selected in Azure.

The route selection is intuitive if you follow the rule of not advertising more specific (or the same length) routes that are used in Azure using BGP from on-premises network or remote VPN networks. If this restriction is maintained, the route selection can be summarized as following list with the preferred conditions on the top:

1. Longest match
2. User-defined routes
3. BGP-derived
4. System routes

Condition is violated if a network gateway injects via BGP a network that is more specific than (or the same as) VNet address space or a peered VNet prefix. In this case the routes with next hop type of Virtual Network, VNet Peering are preferred over more specific BGP routes. We recommend to design network, so prefixes from Azure VNet address space is not used anywhere else on the network. As a workaround in migration scenarios, remote networks can be network address translated (NAT-ed) to non-overlapping range.

Let’s consider few simple examples with 2 Azure VNets with the address prefixes of 10.0.0.0/16 and 10.1.0.0/16. All other networks do not overlap with VNets address spaces.

To identify the best route for a specific destination there are several steps, with each minimizing number of candidate routes to select from. Once a single route is left the process stops and this route is used.

In the diagram below, a packet with the destination of 10.1.0.20 matches 3 routes of different length. The most specific route (with the longest prefix length of /16) is selected.

The next example shows a packet with the destination of 10.30.1.34. After selecting all matching candidates, we have 4 routes left. In the second step, we keep only longest prefix routes (/24). Out of two remaining routes, we know that user-defined routes are more preferred. The single route that points to a virtual appliance with IP address of 10.0.0.100 is used to forward the traffic.

The final example is for a packet with the destination of 192.168.1.100. There are 2 routes left after the step 2 with the same length. As BGP routes are preferred over system routes, the packet will be sent to Virtual network gateway.

Self-Test Questions

What is VNet peering?

VNet peering connects two VNets together using high-speed Microsoft backbone. Resources, such as virtual machines deployed in these VNets can communicate with each other.

Is it possible to peer VNets located in different regions?

Yes

What configuration elements are created for VNet peering?

Peering is configured with 2 peering links – one within each VNet.

VNET-A contains a Virtual network gateway that provides connectivity to an external network. You tasked to provision a VNET-B that needs connectivity to the same external network. What configuration is required so VNET-B can use VNET-A’s Virtual network gateway?

Create a peering between VNET-A and VNET-B. On VNET-A’s peering link enable “Allow gateway transit” option. On VNET-B’s peering link enable “Use remote gateway”.

What is relationship between a subnet and a route table in Azure?

A subnet can have a single route table associated with. By default, a subnet has an implicit default route table associated with it. A route table can be associated with multiple subnets.

What does next-hop type of Virtual network mean in Azure route table?

Represents address space of the VNet. The traffic for the prefixes with this next-hop type will be delivered directly.

Is it possible to create a custom route that uses VNet peering as a next-hop in Azure?

False, VNet peering routes are added by Azure automatically. It is possible, however, to use Virtual appliance as the next hop with IP address in a peered VNet.

When a custom table is associated with a subnet what happens to the Azure system routes?

Azure adds system routes to the custom route tables too. An administrator can override those routes in custom route tables by creating user-defined routes.

How to check routes in a route table in Azure?

An administrator needs a powered-on VM, which has a network interface in the subnet that has the route table associated with it. Navigate to network interface settings page and select Effective Routes menu.

What are the steps of the best route selection process in Azure. Assume that VNet and its peers have unique address spaces, that doesn’t overlap with any other routes?

From the most to the least preferred: Longest match > User-defined routes > BGP-derived routes > System routes