In this blog post I will summarize available information on Cisco ISR and ASR performance. The following platforms will be covered: ISR G2, ISR 1100, ISR 4000, ASR 1000.
Update: check my new article on SD-WAN routers and platforms here.
ISR G2
Let’s start with ISR G2 performance numbers. ISR G2s are legacy products with Classic IOS, however, they are still around and it is important to know how they perform to properly size newer replacement routers.
Important: These are not real-world numbers. Please read further.
Model
Packets Per Second
Megabits Per Second
Cisco 860
25,000
197
Cisco 880
50,000
198
Cisco 890
100,000
1,400
Cisco 1921
290,000
2,770
Cisco 1941
330,000
2,932
Cisco 2901
330,000
3,114
Cisco 2911
352,000
3,371
Cisco 2921
479,000
3,502
Cisco 2951
579,000
5,136
Cisco 3925
833,000
6,903
Cisco 3925E
1,845,000
6,703
Cisco 3945
982,000
8,025
Cisco 3945E
2,924,000
8,675
Table 1. Cisco ISR G2 RFC 2544 Performance
The second column displays the number of packets per second that the platform can forward under maximum CPU utilization just before starting to drop the packets. For a router’s CPU it takes the same amount of effort to route the 64-byte packet as it would take for 1500-byte one. So it is usually a more reliable metric that removes packet size from the equation.
The third column displays the value in bytes per second (i.e. packet size in bytes x packets per second). As the results can differ more than 20x times based on the size of the packets selected, the specification must provide average packet size that was used during the test.
What is IMIX? The traffic doesn’t consist of packets of the same size, many tests are using packets of different sizes (called Internet Mix (IMIX)). For example, in a simple IMIX sample in every 12 packets transmitted – 7 will be 40 bytes long, 4 – 576, and 1 – 1500. The average packet size in this case will be 340 bytes.
Values provided in Table 1 are based only on IP packet routing without any additional processing, such as QoS, encryption, or NAT, so it is a maximum performance that a platform can deliver. The real-world number will be significantly smaller.
Another important thing to note is how a packet is counted, for example, it can be counted twice – as it enters an ingress interface and exits egress one. Cisco counts this is as a single packet, as it is seen by the forwarding engine. On the other hand, to select a router for a specific WAN interface bandwidth utilization in each direction must be added. For example, in the case of 10Mbps WAN with expected 9Mbps download and 3Mbps upload – calculation should be based on 12Mbps of the load.
For G2 platforms Cisco recommended WAN-link based sizing is as per the table below. Values are much smaller compared to normal IP forwarding. It is also expected that the router will not be running at 99% CPU and will be dropping packets.
Platform
WAN Link
860
4
880
8
890
15
1921
15
1941
25
2901
25
2911
35
2921
50
2951
75
3925
100
3945
150
3925E
250
3945E
350
Table 2. ISR G2 Recommended Sizing Based on WAN Link Speed
ISR 4000
ISR 4000s are running IOS-XE and have introduced performance-based licensing with 3 tiers:
Default
Performance (x2-3 of default throughput level)
Boost (removes shaping completely)
Cisco publishes the following statistics for basic IP routing without services with IMIX traffic (~330 bytes packets).
*- bottleneck was the physical interface speed, not forwarding CPU
As the routers are capable to forward significantly more traffic than default and performance license allows, the numbers in table 3 for these license tiers are close to real-life when services are getting added. It is safe to choose ISR 4000 with “factory default” and “performance” levels and in most cases lower models with a “performance” license if you plan to use multiple services.
Recently added boost license removes shaping completely. Table 3 displays PPS values for ISR 4000, however, most of the routers didn’t have high CPU utilization, as the bottleneck was at the interface clock speed. The calculation is based on an IMIX size of 330 Bytes.
The data provided should be used as an only approximation, as there are many variables that can affect actual device performance which also will not scale linearly with CPU load increase.
ISR 1100
ISR 1100 is a new branch office platform running IOS-XE and similar to Cisco 890 and 1921. Published performance numbers are listed in Table 4. IP forwarding of ISR 1100 is comparable to ISR 4221 with a boost license. Note that ISR 1100 doesn’t support voice features.
Platform
RFC-2544
(Mbps, IMIX)
RFC-2544
(pps, IMIX)
Encryption
(Mbps, AES 256, IMIX)
NAT (Mbps, IMIX)
ACL + NAT + HQoS (Mbps, IMIX)
C1100-4P
1,252
475,000
230
660
330
C1100-8P
1,750
660,000
335
960
510
Table 4. ISR 1100 Performance
ASR 1000
In the cases when you need more than 10Gbps of throughput provided by ISR 4461, ASR 1000 will be the platform of choice. All models in the ASR 1000 range have 2 dedicated hardware components – RP (Route Processor) and ESP (Embedded Service Processor). RP is responsible for control-plane operations and ESP for data forwarding.
Lower-end models, such as ASR1001-X and ASR1002-X have RP and ESP integrated into chassis. The throughput of the system depends on ESP, which runs Cisco-proprietary programmable ASICs called Quantum Flow Processor (QFP).
The performance of 3 integrated models is shown in Table 5. For the models presented in Table 5, an incremental throughput license is required.
Model
ESP Bandwidth (Mbps)
Throughput (pps)
ASR1001-X
20,000
19,000,000
ASR1002-X
30,000
36,000,000
ASR1002-HX
100,000
58,000,000
Table 5. ASR 1000 Performance (integrated ESP models)
Related Links
RFC-2544: Provides information on recommended way to perform testing
To configure IPv6 in IOS, as with IPv4, addresses need to be assigned to interfaces. You can assign a single link-local address and multiple global addresses. In comparison to IPv4, IPv6 unicast routing is disabled by default and needs to be globally enabled.
Figure 1. Link-Local Address Configuration
First, we will use a command that automatically generates link-local address for the interface.
Let’s check what IPv6 address has been allocated by IOS.
R01 R01#show ipv6 interface GigabitEthernet3 is up, line protocol is up IPv6 is enabled, link-local address is FE80::20C:29FF:FEB8:6296 No Virtual link-local address(es): No global unicast address is configured Joined group address(es): FF02::1 FF02::1:FFB8:6296
As the listing shows, the link-local address has been automatically assigned. It is derived from the MAC address of the interface displayed in the listing below.
R01 R01#show interface GigabitEthernet3 GigabitEthernet3 is up, line protocol is up Hardware is CSR vNIC, address is 000c.29b8.6296 (bia 000c.29b8.6296)
The interface has automatically joined 2 multicast groups – FF02::1, which is the all-nodes address, and solicited-node for this address -FF002::1:FFB8:6296.
See the details on different addresses format and how they are derived in this blog post.
Let’s manually assign the link-local address, so it is not modified EUI-64 based. “ipv6 enable” command can be removed in this case, as its purpose is to just allocate a link-local address. For link-local addresses, no prefix-length needs to be specified, as it has a fixed format.
R01 R01#show ipv6 interface GigabitEthernet3 is up, line protocol is up IPv6 is enabled, link-local address is FE80::1 No Virtual link-local address(es): No global unicast address is configured Joined group address(es): FF02::1 FF02::1:FF00:1
Notice the new link-local address and solicited-node multicast group addresses.
The next example demonstrates that only one single link-local address is allowed. If a new address is typed in it will overwrite the previous one.
IOS automatically assigns link-local addresses as soon as you configure IPv6 address on the interface. In most cases, you will start your configuration with allocating addresses from global unicast or unique local ranges.
Diagram 2 shows lab topology that we will be using in the next examples.
Figure 2. IPv6 Lab Topology
First let’s configure R1’s interface and verify it’s settings.
R01 R01(config)#interface GigabitEthernet 2.123 R01(config-subif)#encapsulation dot1Q 123 R01(config-subif)#ipv6 address 2001:0:0:123::1/64 R01(config-subif)#end R01#show ipv6 interface Gi2.123 GigabitEthernet2.123 is up, line protocol is up IPv6 is enabled, link-local address is FE80::20C:29FF:FEB8:628C No Virtual link-local address(es): Global unicast address(es): 2001:0:0:123::1, subnet is 2001:0:0:123::/64 Joined group address(es): FF02::1 FF02::1:FF00:1 FF02::1:FFB8:628C
IOS automatically assigned a link-local address. The global unicast address is now assigned and the interface has joined the corresponding solicited-node multicast group – FF02::1:FF00:1.
The similar configuration is now applied to the remaining routers.
Let’s now test connectivity to confirm that we can reach routers on the same segment. As in IPv4, the ping command recognizes IPv6 address format.
R01 R01#ping 2001:0:0:123::3 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 2001:0:0:123::3, timeout is 2 seconds: ….. Success rate is 0 percent (0/5) R01#ping 2001:0:0:123::2 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 2001:0:0:123::2, timeout is 2 seconds: ….. Success rate is 0 percent (0/5)
The connectivity will not work without enabling ipv6 unicast-routing globally, as it is disabled by default in the version of IOS used in this example. Future versions most likely will have it enabled by default.
R01#ping 2001:0:0:123::2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 2001:0:0:123::2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms
R01#ping 2001:0:0:123::3
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 2001:0:0:123::3, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms
R01#
Similar to the “show ip arp” command that displays IP-to-ARP information, in the IPv6 world, there is a “show ipv6 neighbors” command.
R01 R01#show ipv6 neighbors IPv6 Address Age Link-layer Addr State Interface 2001:0:0:123::2 0 000c.29ae.3524 REACH Gi2.123 2001:0:0:123::3 0 000c.29fe.c0ba REACH Gi2.123 FE80::20C:29FF:FEAE:3524 1 000c.29ae.3524 STALE Gi2.123 FE80::20C:29FF:FEFE:C0BA 1 000c.29fe.c0ba STALE Gi2.123
IOS provides another command that can automatically derive the host portion (modified EUI-64) of IPv6 address based on its MAC address. For readability, I’ve assigned the host portion of the address to match the router name. Each interface in IPv6 can have multiple global unicast addresses, so let’s assign the second address to R01.
R01#show ipv6 interface GigabitEthernet2.123 is up, line protocol is up IPv6 is enabled, link-local address is FE80::20C:29FF:FEB8:628C No Virtual link-local address(es): Global unicast address(es): 2001:0:0:123::1, subnet is 2001:0:0:123::/64 2001::123:20C:29FF:FEB8:628C, subnet is 2001:0:0:123::/64 [EUI] Joined group address(es): FF02::1 FF02::2 FF02::1:FF00:1 FF02::1:FFB8:628C
Notice that the router now has 2 IP addresses allocated. As both global addresses share the same last 24-bits, they are mapped to the same solicited-node multicast group – FF02::1:FFB8:628C.
Static Routes Configuration
To enable full reachability we will setup static IPv6 routes. R3 knows about both networks, as it has directly attached interfaces in 2001:0:0:123::/64 and 2001:0:0:34::/64. All other routers require a single static route to a remote subnet. First, let’s configure the static route for R1.
Notice that the route has interface and next-hop information. This type of static route is called a fully specified static route. The next-hop address must be directly attached to the interface. The interface can be omitted, in which case the router will do a lookup to identify the egress interface for the next-hop address. This is called a recursive route lookup. With recursive routes, next-hop is not required to be directly attached.
Let’s check the resulting routing table using the “show ipv6 route” command to confirm that the static route is now present.
R01 R01#show ipv6 route IPv6 Routing Table - default - 5 entries Codes: C - Connected, L - Local, S - Static, U - Per-user Static route B - BGP, R - RIP, H - NHRP, I1 - ISIS L1 I2 - ISIS L2, IA - ISIS interarea, IS - ISIS summary, D - EIGRP EX - EIGRP external, ND - ND Default, NDp - ND Prefix, DCE - Destination NDr - Redirect, RL - RPL, O - OSPF Intra, OI - OSPF Inter OE1 - OSPF ext 1, OE2 - OSPF ext 2, ON1 - OSPF NSSA ext 1 ON2 - OSPF NSSA ext 2, la - LISP alt, lr - LISP site-registrations ld - LISP dyn-eid, lA - LISP away, a - Application S 2001:0:0:34::/64 [1/0] via 2001:0:0:123::3, GigabitEthernet2.123 C 2001:0:0:123::/64 [0/0] via GigabitEthernet2.123, directly connected L 2001:0:0:123::1/128 [0/0] via GigabitEthernet2.123, receive L 2001::123:20C:29FF:FEB8:628C/128 [0/0] via GigabitEthernet2.123, receive L FF00::/8 [0/0] via Null0, receive
Reachability to remote subnet now works, as shown in the next listing. As R4 doesn’t have a static route for return traffic at this stage, it is still not reachable.
R01 R01#ping 2001:0:0:34::3 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 2001:0:0:34::3, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/5 ms R01#ping 2001:0:0:34::4 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 2001:0:0:34::4, timeout is 2 seconds: ….. Success rate is 0 percent (0/5)
As the next step, we will configure all remaining routers. R4 has connectivity only via R3 and is called stub router, so it will have an only default route configured instead of a specific route.
To see detailed debug-level information on neighbor discovery we will use the “debug ipv6 nd” command. All debug commands must be used with care in aproduction environment, as they can cause performance degradation and in some cases can overload the router’s CPU.
In the example below, R01 doesn’t have information about R02’s MAC address. Debug shows that R1 sends Neighbor Solicitation messages and in response gets Neighbor Advertisement message with Link-Local Address (LLA) of R02. Debug also shows that after the exchange of Global Unicast information completed routers exchange link-local IPv6 information.
R01 R01#show ipv6 neighbors IPv6 Address Age Link-layer Addr State Interface 2001:0:0:123::3 1 000c.29fe.c0ba STALE Gi2.123 FE80::20C:29FF:FEFE:C0BA 1 000c.29fe.c0ba STALE Gi2.123
R01#debug ipv6 nd ICMP Neighbor Discovery events debugging is on ICMP ND HA events debugging is ON R01#terminal monitor R01# R01#ping 2001:0:0:123::2 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 2001:0:0:123::2, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 1/3/14 ms R01# *Nov 18 00:08:01.516: ICMPv6-ND: (GigabitEthernet2.123,2001:0:0:123::2) DELETE -> INCMP *Nov 18 00:08:01.519: ICMPv6-ND: (GigabitEthernet2.123,2001:0:0:123::2) Sending NS *Nov 18 00:08:01.519: ICMPv6-ND: (GigabitEthernet2.123,2001:0:0:123::2) Queued data for resolution *Nov 18 00:08:01.524: ICMPv6-ND: (GigabitEthernet2.123,2001:0:0:123::2) Received NA from 2001:0:0:123::2 *Nov 18 00:08:01.524: ICMPv6-ND: Validating ND packet options: valid *Nov 18 00:08:01.524: ICMPv6-ND: (GigabitEthernet2.123,2001:0:0:123::2) LLA 000c.29ae.3524 *Nov 18 00:08:01.524: ICMPv6-ND: (GigabitEthernet2.123,2001:0:0:123::2) INCMP -> REACH *Nov 18 00:08:01.528: ICMPv6-ND: (GigabitEthernet2.123,2001:0:0:123::1) Received NS from 2001:0:0:123::2 *Nov 18 00:08:01.528: ICMPv6-ND: Validating ND packet options: valid *Nov 18 00:08:01.528: ICMPv6-ND: (GigabitEthernet2.123,2001:0:0:123::1) Sending NA to 2001:0:0:123::2 *Nov 18 00:08:06.586: ICMPv6-ND: (GigabitEthernet2.123,FE80::20C:29FF:FEB8:628C) Received NS from FE80::20C:29FF:FEAE:3524 *Nov 18 00:08:06.586: ICMPv6-ND: Validating ND packet options: valid *Nov 18 00:08:06.586: ICMPv6-ND: (GigabitEthernet2.123,FE80::20C:29FF:FEAE:3524) Glean *Nov 18 00:08:06.586: ICMPv6-ND: (GigabitEthernet2.123,FE80::20C:29FF:FEAE:3524) LLA 000c.29ae.3524 *Nov 18 00:08:06.586: ICMPv6-ND: (GigabitEthernet2.123,FE80::20C:29FF:FEAE:3524) INCMP -> STALE *Nov 18 00:08:06.587: ICMPv6-ND: (GigabitEthernet2.123,FE80::20C:29FF:FEB8:628C) Sending NA to FE80::20C:29FF:FEAE:3524 *Nov 18 00:08:06.588: ICMPv6-ND: (GigabitEthernet2.123,FE80::20C:29FF:FEAE:3524) STALE -> DELAY *Nov 18 00:08:11.649: ICMPv6-ND: (GigabitEthernet2.123,FE80::20C:29FF:FEAE:3524) DELAY -> PROBE *Nov 18 00:08:11.650: ICMPv6-ND: (GigabitEthernet2.123,FE80::20C:29FF:FEAE:3524) Sending NS *Nov 18 00:08:11.651: ICMPv6-ND: (GigabitEthernet2.123,FE80::20C:29FF:FEAE:3524) Received NA from FE80::20C:29FF:FEAE:3524 *Nov 18 00:08:11.651: ICMPv6-ND: Packet contains no options *Nov 18 00:08:11.652: ICMPv6-ND: Validating ND packet options: valid *Nov 18 00:08:11.652: ICMPv6-ND: Packet contains no options *Nov 18 00:08:11.652: ICMPv6-ND: (GigabitEthernet2.123,FE80::20C:29FF:FEAE:3524) PROBE -> REACH
R01#show ipv6 neighbors IPv6 Address Age Link-layer Addr State Interface 2001:0:0:123::2 0 000c.29ae.3524 REACH Gi2.123 2001:0:0:123::3 4 000c.29fe.c0ba STALE Gi2.123 FE80::20C:29FF:FEAE:3524 2 000c.29ae.3524 STALE Gi2.123 FE80::20C:29FF:FEFE:C0BA 4 000c.29fe.c0ba STALE Gi2.123
Troubleshooting Commands
The other debug command we can use for troubleshooting is the “debug ipv6 packets” command. In the production environment always use access-list based filters to limit the amount of output.
R01 R01#ping 2001:0:0:34::4 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 2001:0:0:34::4, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/2 ms R01# *Nov 18 00:16:37.609: IPv6-Fwd: Destination lookup for 2001:0:0:34::4 : i/f=GigabitEthernet2.123, nexthop=2001:0:0:123::3 *Nov 18 00:16:37.610: IPv6-Fwd: SAS picked source 2001:0:0:123::1 for 2001:0:0:34::4 (GigabitEthernet2.123) *Nov 18 00:16:37.610: IPv6-Fwd: nexthop 2001:0:0:123::3, *Nov 18 00:16:37.610: IPV6: source 2001:0:0:123::1 (local) *Nov 18 00:16:37.610: dest 2001:0:0:34::4 (GigabitEthernet2.123) *Nov 18 00:16:37.610: traffic class 0, flow 0x0, len 100+0, prot 58, hops 64, originating *Nov 18 00:16:37.610: IPv6-Fwd: Created tmp mtu cache entry for 2001:0:0:123::1 2001:0:0:34::4 00000000 *Nov 18 00:16:37.610: IPv6-Fwd: L3 injection feature enabled: skipping pak_encap *Nov 18 00:16:37.611: IPv6-Fwd: Destination lookup for 2001:0:0:123::1 : Local, i/f=GigabitEthernet2.123, nexthop=2001:0:0:123::1 *Nov 18 00:16:37.611: IPV6: source 2001:0:0:34::4 (GigabitEthernet2.123) *Nov 18 00:16:37.611: dest 2001:0:0:123::1 (GigabitEthernet2.123) *Nov 18 00:16:37.611: traffic class 0, flow 0x0, len 100+18, prot 58, hops 63, forward to ulp
This command produces detailed information on the packet forwarding, including information on which source IPv6 address and the outgoing interface were chosen.
This blog post covers the following CCNA blueprint topics:
1.9 Compare IPv6 address types
1.9.a Global unicast
1.9.b Unique local
1.9.c Link local
1.9.d Anycast
1.9.e Multicast
1.9.f Modified EUI 64
The article content can be useful in preparation for other certification exams and IPv6 gradually becomes as important as IPv4.
Address Format
IPv6 addresses are 128-bit long. An address is divided into 8 groups each containing 4 hexadecimal digits. Groups are separated from each other by colons. Each group is 16-bit long and called hextet.
Hexadecimal digit is called a nibble (4-bit long, half-byte) and can be any number between 0 up to letter F. Figure 1 displays the difference between IPv4 and IPv6 address formats and terms.
Figure 1. IPv4 vs IPv6 Address Format
IPv6 addresses are much longer in size than IPv4 addresses. Zeroes in the address can compress to make it more readable using the following rules:
Leading zeroes within a hextet can be removed. For example, 00FC can be compressed to FC. Using this rule, 0000 can be compressed to 0, as it requires at least a single-digit to be preserved. The IPv6 address from Figure 1 can be compressed to FE80:0:0:0:C4A8:D1BF:7583:1166.
Hextets of all zeroes can be replaced with double-colon once. The address from Figure 1 can be compressed as FE80::C4A8:D1BF:7583:1166. Only one group of zero-hextets can be replaced with double-colon.
IPv6 addresses use prefix notation similar to IPv4 CIDR (Classless Inter-Domain Routing) to identify subnet portion of the address. For example,
2001:C3A9:D1BF:7423:2345:112A:BCDE:F119/64
is part of the subnet:
2001:C3A9:D1BF:7423:0:0:0:0/64
Address Types
There are 3 types of
IPv6 addresses:
Unicast addresses are allocated to a single device. Traffic sent to a unicast address is delivered only to this interface.
Ananycast address can be assigned to many devices at the same time. Traffic sent to an anycast address will be delivered to the closest device.
Multicast. The address associated with multiple hosts and traffic sent to a multicast address delivered to many devices at the same time.
Unicast and anycast use the same IPv6 address space and multicast has its own dedicated prefix range.
In comparison to IPv4, there are no broadcast addresses in IPv6, which used to represent the delivery of traffic to all interfaces. Multicast delivery is now replacing this functionality.
Address Space
IPv6 address space is managed by Internet Assigned Numbers Authority (IANA). Current address space allocation is available via this URL. The special-purpose address registry is available here.
Figure 2. IPv6 Address Space Allocation
Figure 2 demonstrates how address IPv6 address blocks are distributed proportionally to the full address space. Global Unicast address space (2000::/3) is the block that IANA uses for the current allocation and its size should provide enough addresses in the foreseeable future. For example, the entire IPv4 address space would be much smaller than a pixel in Figure 2.
Unicast Address Types
0000::/3
Addresses in 0000::/3 range contain several reserved addresses, such as an unspecified address, loopback, and IPv4-mapped addresses.
Unspecified address, 0:0:0:0:0:0:0:0 or :: means that the address is not present. It cannot be used as a destination address in any packets, however, it is used as a source when a host doesn’t have an address allocated.
Loopback address is allocated address of 0:0:0:0:0:0:0:1/128 or ::1/128. As in IPv4, this address represents the host itself. It cannot be assigned to any of the physical interfaces. As a result, it will not be seen outside of the host, and routers will not forward packets sent to this address.
IPv4-mapped IPv6 addresses are used in Dual-Stack (IPv4/IPv6) systems, so IPv6 application can recognize packet delivered to IPv4 address. For example, packet going to IPv4 address 1.2.3.4 will be mapped to IPv6 address of 0:0:0:0:0:FFFF:1.2.3.4 or ::FFFF:1.2.3.4.
Figure 3 displays the IPv4-mapped IPv6 address format. For all diagrams in this post, black dotted-line annotations mean that the value is static. Yellow dotted-line annotations mean that the value is variable.
Figure 3. IPv4-Mapped Address
IPv4-compatible addresses are another way of encoding IPv4 within IPv6 address. However, they are deprecated and IPv4-mapped addresses should be used instead.
2000::/3
Addresses in 2000::/3 range is what called Global Unicast Address space. The current allocation to registries is listed here.
The format of addresses in this range is shown in Figure 4. Global Unicast Address starts with binary 001 and the first digit can be either 2 or 3.
Figure 4. Global Unicast Address
The first part of the address is variable-length Global Routing Prefix, which can be allocated to a site. It follows by variable-length Subnet ID for allocation within that site. The second half of the address (64-bits) is used by interface ID. Interface ID must be unique on the subnet and are derived from the hardware (MAC) address of the device. The resulting format is called modified EUI-64.
The process of deriving address from MAC address is displayed in Figure 5.
Figure 5. Deriving Modified EUI-64 from MAC Address
E000::/3
The last /3 block is e000::/3. It contains Unique-Local Addresses, Link-Local Unicast Addresses, and Multicast Addresses.
Unique Local Addresses can be used within a site or between multiple sites or organizations. They are not routable on the Internet. They have very similar features as RFC1918 private IPv4 addresses. What makes Unique Local different, is that they are designed to be unique most of the time. Part of the address named Global ID is a 40-bit pseudo-random number that ensures that there is a very small probability of having the same address range as the other party.
Figure 6. Unique Local Address
The allocated range for Unique Local Addresses is FC::/7. However, the 8th bit of the address is a flag. When it has a value of 1 it means that the address is locally defined. This is the only value currently defined, so all addresses will start with FD.
As displayed in Figure 6, Unique Local Address consists of 3 parts: Global ID, Subnet ID, and an Interface ID. A single Global ID prefix can accommodate more than 65000 subnets.
Link-Local IPv6 Unicast Addresses are designed
to be used on a single layer-2 domain. They must not appear as the source or
destination address for packets traversing routers. Link-Local Addresses are
used for local traffic, such as automatic address configuration and neighbor
discovery.
The format of the Link-Local IPv6 Unicast Address is shown in the Figure 7.
Figure 7. Link-Local Address
Multicast Addresses
The multicast address format is shown in Figure 8. The first 8 bits are always 1s (FF). Next 4 bits are reserved for flags followed by 4 bits representing group scope.
Figure 8. Multicast Address
The flag bits (0RPT)
are shown in Figure 9. The first bit (left-most or high-order) is always 0.
R-flag, if set to 1, means Rendezvous Point address is encoded within the group address.
P-flag means that it
is unicast prefix-based allocation. When
an entity receives Unicast Global address allocation, it gets delegated a
corresponding multicast range as well. When P-Flag is set to 1, multicast
address contains encoded unicast prefix of owner-organization.
T-flag stands for transient. If set to 1, the address is non-permanent. If set to 0, the address is well-known and has been permanently assigned by IANA.
Figure 9. IPv6 Address Multicast Address Flags
Scope bits define how far multicast traffic can propagate. Table below shows some of the scopes defined:
Scope bits
Hex
Scope
0001
1
Interface-Local
0010
2
Link-Local
0100
4
Admin-Local
0101
5
Site-Local
1000
8
Organization-Local
1110
E
Global
There are several reserved multicast addresses. The next table shows some of these addresses.
Address
Description
FF02:0000:0000:0000:0000:0000:0000:0001
All Nodes (Link-Local)
FF02:0000:0000:0000:0000:0000:0000:0002
All Routers (Link-Local)
FF02:0000:0000:0000:0000:0001:FFXX:XXXX
Solicited-Node Address (XX:XXXX – last 24 bits of Unicast Address)
FF3S:0000:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX
Source-Specific Multicast (FF3S::/32). S – is scope, X – group address bits
Update: check our new article for new CCNA exam blueprint.
The next topic from the CCNA routing and switching blueprint that I thought would be interesting to go through is LAN-design related:
1.5 Compare and contrast collapsed core and three-tier architectures
While there might be more modern approaches for LAN or Campus designs, for the purpose of this specific exam objective let’s start with three-tier architecture. The 3 tiers are – Access, Distribution, and Core. Tiers are logical, so it doesn’t have to be a dedicated device on each one of them.
Access layer exists in every design because it’s where end devices, such as computers and phones are connected. QoS classification and marking, 802.1X authentication is performed on access tier switches, as these services should be applied as close to the source as possible. Access switches usually have high copper port density and Power Over Ethernet functionality for the phones and access points.
What are the current LAN access platforms? Catalyst 2960-X/XR (can be stackable), Catalyst 3650 (can be stackable), and Catalyst 3850 (stackable), Catalyst 9300 (stackable), and Catalyst 9400 (modular).
CCNA Routing and Switching exam has this topic in the blueprint, which I will try to cover in this blog post:
1.4 Describe the effects of cloud resources on enterprise network
architecture
•
1.4.a Traffic path to internal and external cloud services
•
1.4.b Virtual services
• 1.4.c Basic virtual network infrastructure
What is a cloud service or resource?
There are many definitions of the term. Many of them refer to public clouds reachable over the Internet. For example, AWS, Microsoft Azure, or Google Cloud Platform. It is often opposed to on-premises private infrastructure. However, cloud service can be private if it has characteristics of public clouds, such as self-service and automation of infrastructure provisioning.
Cisco routers are one of the most widely deployed WAN devices. Traditionally they are individually managed and for the larger networks, administrators require additional tools to monitor, perform configuration backup, and to automate tasks.
Many newer Cisco technologies have some form of a central controller and managed data-plane devices. For example, ACI in the data center and SD-Access for the campus. In WAN space, the Cisco portfolio included IWAN (Intelligent WAN) technology and cloud-managed products from Meraki acquisition. In 2017 Cisco has acquired Viptela and its SD-WAN product line. This post contains an overview of this technology and some basic terminology.
Traditional WAN design
To understand the benefits of SD-WAN, let’s consider how most of the Wide Area Networks are designed. Multiple branch offices connect via an MPLS network to one or two data centers, which also provide centralized Internet access. It is secured by high-performance firewalls, intrusion protection, and web filtering platforms. Each branch or remote office has a single or pair of routers forwarding multiple types of traffic, such as:
Business applications (SAP, ERP)
Office 365 (Outlook, Sharepoint, etc)
Internet browsing
Video and IP telephony
Interactive applications, such as remote desktops
Management and Operational Issues
The device-centric approach has many challenges. For example, application performance troubleshooting requires an administrator to check every router in path hop-by-hop and takes a significant amount of time.
In many WAN environments, quality of service (QoS) configuration is static in nature, as a change in QoS design may take several maintenance windows to deploy across the network.
In a similar way, wireless deployments have transformed from autonomous to controller-based, as many tasks require a coordinated approach in management. For example, Radio Resource Management is one of such tasks, when the channel and transmit power selection is very difficult to maintain manually on every access point.
WAN links are also relatively expensive. In many networks, standby WAN links are required for high availability. Establishing these links takes time and service providers may require fixed-term commitment. In contrast, Internet links are affordable and have shorter lead times to provision.
With traditional design described earlier, traffic going to the workload and applications in a data center has to compete with the services reachable via the public Internet. It is cost-effective to offload Internet traffic to a branch local Internet link.
This interface can also be used as a secondary WAN link connecting sites over VPN connections. However, it is difficult to manage multiple tunnels as the number of routers goes up while providing consistent user experience and ensuring that the security is not compromised.
SD-WAN Design Approach
SD-WAN addresses these issues. A centralized set of controller devices provides a level of abstraction, so network administrators can spend more time on creating policies and configuration templates without having to touch every device on the network.
WAN is treated as a transport-agnostic fabric. Underlay network provides connectivity between tunnel endpoints and doesn’t need to have knowledge about reachability information behind these gateways. As a result, overlay tunnels can be created dynamically and networks can recognize application traffic and select the best path in real-time.
Components and Architecture
SD-WAN operations comprise of 4 planes, implemented by a set of controllers and gateways:
Management plane controller (vManage)
Orchestration plane controller (vBond)
Control plane controller (vSmart)
Data plane forwarding device (vEdge)
Controllers can be hosted and managed by Cisco as a subscription-based product or can be deployed on-premises. vManage, vBond, and vSmart are virtual machines available for download as OVA files. ESXi and KVM are the supported hypervisors.
Figure 1. SD-WAN Architecture
The first component to be configured in a new SD-WAN network is vManage, which can be deployed as a single appliance or cluster of at least 3 nodes. vManage implements a management plane and is the place where all configuration happens. It also performs fabric monitoring and can expose centralized API access for external applications to the SD-WAN network.
vBond is responsible for accepting registration and authenticating vSmart controllers and vEdges. Every device needs to be pointed to vBond during provisioning. It then ensures that all other elements are able to locate each other. vBond must have a public IP address and should be placed into DMZ, so it can be accessed over the Internet.
vSmart controls all overlay routing and secure tunnel establishment between vEdges. The control protocol between vSmart and vEdge elements is called OMP (Overlay Management Protocol). It is protected by DTLS and carries not only reachability information, but also security associations details for IPSec tunnels. vSmart performs policy propagation to the edge devices.
vEdge devices are gateways performing data forwarding over overlay networks. This can be Viptela appliances (vEdge Routers), or Cisco devices running SD-WAN image such as Cisco ISR 4000. There is an option of software vEdge Cloud routers hosted in the public cloud – AWS or Azure.
Cisco works on getting routers with SD-WAN image to have feature parity with Viptela appliances, so always check release notes, as there might be a feature not yet supported on Cisco ISRs.
The next few sections explain the most important terms and concepts of SD-WAN, such as VPNs, TLOCs, and OMP.
Figure 2. SD-WAN Terminology
VPNs
Viptela SD-WAN uses the concept of VPN which is a way to segregate networks. Each VPN has interface allocation and a routing table isolated from other VPNs. It is similar to the Cisco VRF (Virtual Routing and Forwarding) instance. VPN number is globally significant and must match for communication to happen. Encapsulated IP packets carry VPN tag, so egress gateway can determine which VPN packet belongs to.
There are 513 VPNs with the first and last reserved for fabric operations. VPN 0 is transport VPN and is similar to the global VRF context. Interfaces in VPN 0 are called tunnel interfaces and have IP addresses visible by transit networks and form underlay of the fabric. Communication between the network controllers of SD-WAN happens over VPN 0.
VPN 512 is used for Out-Of-Band-Management network.
All other VPNs 1-511 can be used to forward user data.
In Figure 2, VPN 100 and VPN 200 are created in the network. Subnets A, B, and E can communicate with each other within VPN 100. And subnets C and D can communicate with each other within VPN 200.
TLOCs (Transport LOCators)
One of the tasks of OMP is to distribute reachability information. Each destination can be reachable via a specific interface on one of the vEdges on the network. TLOC is a composite structure describing this interface and consists of:
System IP address of the OMP
Color of the link
Encapsulation of the tunnel (IPSec or GRE)
TLOC is similar in concept to the next hop in BGP. Color is a pre-defined tag that describes type of the WAN interface, for example mpls, 3g or biz-internet.
OMP (Overlay Management Protocol)
vSmart exchanges information with vEdges using OMP. This protocol covers all control-plane aspects required to transmit data on top of the overlays.
OMP is responsible for exchange of 3 types of routes:
vRoutes, reachability on the LAN side of the router. vEdge supports static routes, dynamic protocols – BGP and OSPF are supported. Information about a source routing protocol, its metric is carried along with these routes. VPN, the Site ID is another important information present in vRoutes as well.
Service Routes. The way to perform service chaining and insert a firewall or a load balancer
TLOC Routes. Carries information on how to reach specific TLOC such as IP addresses of the interface.
VPC or Virtual Port Channel is a Cisco proprietary feature available on the Nexus platform. Two switches of the same model can be combined into a VPC pair, which can establish a single EtherChannel, also known as a link aggregate or a port channel, across both switches to a third switch or server.
This peering device doesn’t know that it is connected to two different switches and it just needs to support link aggregation either statically or using Link Aggregation Control Protocol (LACP).
MultiChassis EtherChannel (MCEC) or MultiChassis Link Aggregation (MLAG) terms refer to the technique of bundling links across more than 1 device. VPC is Cisco’s implementation of MCEC/MLAG on the Nexus line of switches. Similarly, Cisco Catalyst switches support Virtual Switching System (VSS) or StackWise-based MLAGs.
VPC channel group – is similar to traditional EtherChannel, however, has its member ports on different switches
VPC port or VPC member port – is a port that is part of VPC-based port-channel
Without VPC, parallel links are considered as a Layer 2 loop. Spanning-Tree Protocol (STP) would block one of these links. To enable simultaneous use of these parallel links, STP priority adjustments were required and load balancing was done on per-VLAN or per-MST instance basis. Such configuration adds complexity and doesn’t provide even traffic distribution.
VPC addresses these issues. With VPC, multiple uplinks from an access switch are treated as a single link. Layer-2 topology becomes loop-free and no port blocking by Spanning Tree Protocol is required.
Servers and hypervisors can also more optimally balance traffic without having to pin virtual machines to a specific uplink.
VPC Components
To enable VPC a high-bandwidth interface known as VPC peer-link is required. It is recommended to bundle at least two 10Gbps ports. Peer-link is used to perform state synchronization and some data traffic.
An additional interface is required for keepalive exchange. It provides physically diverse connectivity, so heartbeats are not lost when VPC peer-link goes down. This mechanism protects against split-brain scenarios during VPC peer link failure.
Switches can be directly connected using 1Gbps or higher bandwidth ports. Out-of-band management interfaces also can be used as a peer-keepalive link.
Figure 1. VPC Components
Peer keepalives are sent every second by default and must be explicitly bound to a specific IP address. As a result, heartbeats can be routed across the network.
Nexus switches have dedicated mgmt0 interfaces for out-of-band management. This interface belongs to management VRF and designed to be connected to a dedicated Out-Of-Band Management (OOBM) network. Standard practice is using this network for peer-keepalives. This saves a front data port.
One of VPC peers becomes primary and the other one is secondary. The configurable priority value controls preference of a switch’s role, however, it is runtime parameter and secondary switch can become operational primary. Primary peer exclusively runs some of control plane features, but most importantly during peer link failures it keeps its interfaces up, while secondary has to shutdown links participating in VPC downstream to prevent loops.
As both VPC peers have their own management plane and configured separately, the protocol must ensure that both switches are configured in a consistent manner. There are 2 types of configuration parameters – Type 1 and Type 2. If Type 2 parameters mismatch both switches continue to operate normally, however, some traffic will be forwarded not optimally. With Type 1 settings mismatch, the secondary switch stops forwarding traffic for VPC enabled VLANs.
VPC VLAN – is VLAN that is allowed on VPC peer link
Orphan port – Port that is connected to a single switch and not part of a VPC port channel. To be qualified as an orphan port, it has to be a member of a VPC VLAN or have it enabled if it is 802.1q trunk.
The Data plane operation of the VPC switch pair follows some rules to prevent loops. During normal operations, peer-link is not used for data-plane packets with the exceptions of traffic to and from orphan ports or during one of the VPC member ports failure. VPC switches prefer local VPC ports when selecting egress interface.
VPC Configuration
Configuration of VPC consists of several steps:
Prepare L3 peer-keepalive connectivity, in the example below, out-of-band management interface was used
Enable VPC and LACP features
Enable and configure global VPC options under “vpc domain” sub-mode. The most important settings are peer-keepalive and role priority (to make one of the switches a primary).
Configure VPC peer-link
The diagram below shows the lab diagram and example of basic VPC configuration.
Figure 2. VPC Lab Diagram
Sample configuration for VPC switches N9K-4 and N9K-5 is shown below.
N9K-4
N9K-5
N9K-4(config)# interface mgmt0 N9K-4(config-if)# ip address 10.4.4.4/24 !Ensure that there is ip connectivity, see listing 1
N9K-5(config)# interface mgmt0 N9K-5(config-if)# ip address 10.4.4.5/24
N9K-4(config-if)# int E1/2 N9K-4(config-if)# channel-group 2 mode active N9K-4(config-if)# int Po2 N9K-4(config-if)# vpc 2 !See listing 4 for the verification of connectivity to N9K-1
N9K-5(config-if)# int E1/3 N9K-5(config-if)# channel-group 2 mode active N9K-5(config-if)# int Po2 N9K-5(config-if)# vpc 2
Listing 1
N9K-4(config-if)# ping 10.4.4.5 vrf management PING 10.4.4.5 (10.4.4.5): 56 data bytes 36 bytes from 10.4.4.4: Destination Host Unreachable Request 0 timed out 64 bytes from 10.4.4.5: icmp_seq=1 ttl=254 time=14.936 ms 64 bytes from 10.4.4.5: icmp_seq=2 ttl=254 time=0.639 ms 64 bytes from 10.4.4.5: icmp_seq=3 ttl=254 time=0.497 ms 64 bytes from 10.4.4.5: icmp_seq=4 ttl=254 time=0.504 ms
--- 10.4.4.5 ping statistics --- 5 packets transmitted, 4 packets received, 20.00% packet loss round-trip min/avg/max = 0.497/4.144/14.936 ms
Listing 2
N9K-4(config-vpc-domain)# show vpc Legend: (*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 100 Peer status : peer link not configured vPC keep-alive status : Suspended (Destination IP not reachable) Configuration consistency status : failed Per-vlan consistency status : failed Configuration inconsistency reason: vPC peer-link does not exist Type-2 consistency status : failed Type-2 inconsistency reason : vPC peer-link does not exist vPC role : none established Number of vPCs configured : 0 Peer Gateway : Disabled Dual-active excluded VLANs : - Graceful Consistency Check : Disabled (due to peer configuration) Auto-recovery status : Disabled Delay-restore status : Timer is off.(timeout = 30s) Delay-restore SVI status : Timer is off.(timeout = 10s) Operational Layer3 Peer-router : Disabled
!After peer-keepalive configured on both switches
N9K-4(config-vpc-domain)# show vpc Legend: (*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 100 Peer status : peer link not configured vPC keep-alive status : peer is alive Configuration consistency status : failed Per-vlan consistency status : failed Configuration inconsistency reason: vPC peer-link does not exist Type-2 consistency status : failed Type-2 inconsistency reason : vPC peer-link does not exist vPC role : none established Number of vPCs configured : 0 Peer Gateway : Disabled Dual-active excluded VLANs : - Graceful Consistency Check : Disabled (due to peer configuration) Auto-recovery status : Disabled Delay-restore status : Timer is off.(timeout = 30s) Delay-restore SVI status : Timer is off.(timeout = 10s) Operational Layer3 Peer-router : Disabled N9K-4(config-vpc-domain)#
Listing 3
N9K-4(config-if)# show vpc Legend: (*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 100 Peer status : peer adjacency formed ok vPC keep-alive status : peer is alive Configuration consistency status : success Per-vlan consistency status : success Type-2 consistency status : success vPC role : primary Number of vPCs configured : 0 Peer Gateway : Disabled Dual-active excluded VLANs : - Graceful Consistency Check : Enabled Auto-recovery status : Disabled Delay-restore status : Timer is off.(timeout = 30s) Delay-restore SVI status : Timer is off.(timeout = 10s) Operational Layer3 Peer-router : Disabled
vPC Peer-link status --------------------------------------------------------------------- id Port Status Active vlans -- ---- ------ ------------------------------------------------- 1 Po100 up 1
N9K-1(config-if-range)# show port-channel summary Flags: D - Down P - Up in port-channel (members) I - Individual H - Hot-standby (LACP only) s - Suspended r - Module-removed b - BFD Session Wait S - Switched R - Routed U - Up (port-channel) p - Up in delay-lacp mode (member) M - Not in use. Min-links not met -------------------------------------------------------------------------------- Group Port- Type Protocol Member Ports Channel -------------------------------------------------------------------------------- 2 Po2(SU) Eth LACP Eth1/2(P) Eth1/3(P)
Listing 5
N9K-4(config-if)# show vpc Legend: (*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 100 Peer status : peer adjacency formed ok vPC keep-alive status : peer is alive Configuration consistency status : success Per-vlan consistency status : success Type-2 consistency status : success vPC role : primary Number of vPCs configured : 1 Peer Gateway : Disabled Dual-active excluded VLANs : - Graceful Consistency Check : Enabled Auto-recovery status : Disabled Delay-restore status : Timer is off.(timeout = 30s) Delay-restore SVI status : Timer is off.(timeout = 10s) Operational Layer3 Peer-router : Disabled
vPC Peer-link status --------------------------------------------------------------------- id Port Status Active vlans -- ---- ------ ------------------------------------------------- 1 Po100 up 1
vPC status ---------------------------------------------------------------------------- Id Port Status Consistency Reason Active vlans -- ------------ ------ ----------- ------ --------------- 2 Po2 up success success 1
Built-in into IOS, TCL interpreter can be useful in several scenarios.
You may need to apply several commands on a Cisco device when some of the earlier commands can prevent later commands to be delivered. For example, one may need to move an external interface of a remote router to a different VRF. The moment the command that changes VRF delivered, the router removes IP address on the interface and the interactive terminal will not be able to send the remaining commands.
To avoid a lockout, prior to applying the configuration, save the configuration and enter the “reload in 10″ exec command, which will restart the router in 10 minutes unless canceled. Don’t forget to do “reload cancel” once you are happy with the outcome.
The following example shows how to enter interface configuration mode and change its description.
ROUTER#tclsh
ROUTER(tcl)#ios_config “interface GigabitEthernet2” “description WAN Interface”
Instead of using an interactive interpreter, one can create a script file and save it on the flash to be later invoked.
flash:/script.tcl file content:
ios_config “interface GigabitEthernet1” “description LAN Interface”
To execute this file, pass it as an argument to tclsh command:
ROUTER#tclsh
ROUTER(tcl)#tclsh flash:/script.tcl
To upload script files to the router’s flash memory use one of the protocols that router supports, such as TFTP. If such access is not available, one can use puts function of TCL that writes the list of strings into a text file.
