Cisco SD-WAN devices can be either in vManage or CLI mode.
In vManage mode, the configuration is performed on vManage and then pushed to the device. Local configuration changes are not allowed. In CLI mode, changes are performed locally on the device. vManage mode is the preferred and recommended option for most SD-WAN implementations. However, you can occasionally switch devices into CLI mode to perform specific tasks.
Administrators can connect to the device using SSH or serial console to use various show and debug commands in both modes.
We will focus on edge devices in this article. However, controllers can also be in one of these modes. Cisco-hosted controllers are initially provisioned in CLI mode and then converted to vManage mode by a network administrator. Some features require controllers to be switched to vManage mode.
vManage mode
Consider a sample network shown in Figure 1. Our task is to prepare an edge router for a remote office. We’ve applied the required feature and device templates with dynamic IP assignment on the Internet-facing interface. After the router is shipped to the remote location and plugged in, it cannot establish connectivity to vManage. We discover that the ISP doesn’t have DHCP enabled, and a static IP configuration is required.
Let’s assume that we can access the router via console cable attached to a laptop on-site.
Run the “show sdwan system status” command on the edge device to validate the device mode. In the sample output below, the CSR01 router is in vManage mode (vManaged: true).
CSR01#show sdwan system status Viptela (tm) vEdge Operating System Software Copyright (c) 2013-2021 by Viptela, Inc. Controller Compatibility: 20.3 Version: 17.03.03.0.4762 <output omitted> Personality: vEdge Model name: CSR1000V Services: None vManaged: true Commit pending: false Configuration template: CSR01 <output omitted>
Firstly, we try to set a static IP address on the transport interface using CLI. We use the “config-t” command to enter the configuration mode. The “commit” command activates the configuration.
CSR01#config-t CSR01(config)# interface GigabitEthernet 2 CSR01(config-if)# ip address 21.1.1.2 255.255.255.252 CSR01(config-if)# commit The following warnings were generated: 'system is-vmanaged': This device is being managed by the vManage. Any configuration changes to this device will be overwritten by the vManage after the control connection to the vManage comes back up. Proceed? [yes,no] yes Commit complete. CSR01#show run interface GigabitEthernet 2 Building configuration... <output omitted> interface GigabitEthernet2 description Transport Interface ip address 21.1.1.2 255.255.255.252
In our example, we didn’t have connectivity to vManage, so the changes were committed. However, as the warning message above advises vManage will overwrite modifications done on the device (see the example in the following listing) once the control connection is up.
CSR01#show run interface GigabitEthernet 2 Building configuration... <output omitted> interface GigabitEthernet2 description Transport Interface ip address dhcp
Rollback to the original configuration (i.e. setting dynamic IP on the transport interface) causes connectivity to vManage fail, which introduces the connectivity problem we tried to solve in the first place.
When we have connectivity to vManage, the CLI commands are just rejected, as shown in the listing below. vManage mode doesn’t permit changes on the edge device.
CSR01#config-t CSR01(config)# interface GigabitEthernet2 CSR01(config-if)# ip address 21.1.1.2 255.255.255.252 CSR01(config-if)# commit Aborted: 'system is-vmanaged': This device is being managed by the vManage. Configuration through the CLI is not allowed.
CLI Mode
There are several solutions to the problem described above. The first is to switch the device into CLI mode beforehand before sending it to the remote location. We can then adjust the configuration and set static IP address, and move the device back to vManage mode.
The second solution is using hidden support commands, which is not a supported way of configuring the device unless directed by Cisco TAC. However, it works and can be helpful in lab environments or as a quick fix to get the branch running (with full router reset later).
Let’s see how both options work with the examples below.
Switching device to CLI mode via vManage
An administrator can switch the device to CLI mode (and back) via vManage. Navigate to Configuration > Devices. Click on the Change Mode drop-down menu and select CLI mode.
In the dialog window, select and move to the right panel the router you want to switch into CLI mode and click on Upgrade to CLI Mode.
After the conversion, we can use “show sdwan system status” command to validate that the vManaged property is false.
CSR01#show sdwan system status Viptela (tm) vEdge Operating System Software Copyright (c) 2013-2021 by Viptela, Inc. Controller Compatibility: 20.3 Version: 17.03.03.0.4762 Build: Not applicable <output omitted> Personality: vEdge Model name: CSR1000V Services: None vManaged: false Commit pending: false Configuration template: None <output omitted>
Now we can change the IP address via CLI.
CSR01#config-t CSR01(config)# interface GigabitEthernet 2 CSR01(config-if)# ip address 21.1.1.2 255.255.255.2 CSR01(config-if)# commit
Switching device back to vManage mode
After the device comes back online, we can update templates to include static IP address configuration and switch it back to the vManage mode. To do this, we need to attach the device template to the vEdge router. If the device had a template attached before, the variable values would be populated automatically.
Switching device to CLI mode using hidden commands
This procedure in this section uses hidden commands and should be used only as a last resort. Cisco TAC may not support it; use it at your own risk.
When the router hasn’t been switched to CLI mode beforehand and cannot establish a control connection to vManage, refer to the following example to switch to CLI mode.
After entering the configuration mode, let’s see available options under the system command:
CSR01#config-t CSR01(config)# system ? Possible completions: admin-tech-on-failure Collect admin-tech before reboot due to daemon failure allow-same-site-tunnels Allow tunnels to be formed between vEdges in the same site console-baud-rate Console baud-rate control-session-pps Control session policer rate, in packets per second controller-group-list Controller group list debug description System description device-groups List of vManage groups to which the device belongs disable domain-id Domain ID environment fnf gps-location GPS latitude and longitude of the device host-policer-pps Rate at which to police packets bound to the control plane (in pps) per QOS level icmp-error-pps Rate at which to police ICMP error messages either generated or received (in pps). idle-timeout Idle CLI timeout, in minutes ignore location Location description of the device max-controllers (DEPRECATED) Set the maximum number of controllers to which the device can connect - Deprecated in 15.4 max-omp-sessions Set the maximum number of OMP sessions the device can have mode-button mtu on-demand Set various configuration for On-demand tunnels organization-name Organization name overlay-id Overlay ID port-hop Enable port hopping for all tlocs port-offset Port offset (unique value; use only if multiple Viptela devices are behind the same NAT) site-id Site ID sp-organization-name Service Provider Organization name system-ip System IP address system-tunnel-mtu Control tunnel MTU tcp-optimization-enabled Carve out a dedicated core to use for TCP optimization - applies after reboot timer Set various timer timeouts tls ssl-opt cert management config track-default-gateway Enable/Disable default gateway tracking track-interface-tag OMP Tag attached to routes based on interface tracking track-transport Enable transport tracking upgrade-confirm Configure software upgrade confirmation timeout vbond Configure remote vBond or local IPv4 vbond address <cr>
“unhide viptela_internal” command enables the display of “system” command hidden options. The example below shows these subcommands.
CSR01(config)# unhide viptela_internal CSR01(config)# system ? Possible completions: admin-tech-on-failure Collect admin-tech before reboot due to daemon failure allow-same-site-tunnels Allow tunnels to be formed between vEdges in the same site allow-sw-vedge (HIDDEN) Allow non-release software vedges to operate without certificates console-baud-rate Console baud-rate control-session-pps Control session policer rate, in packets per second controller-group-list Controller group list daemon-reboot (HIDDEN) Reboot device if a non-restartable daemon fails daemon-restart (HIDDEN) Restart restartable daemons if they fail debug description System description device-groups List of vManage groups to which the device belongs disable dnsd-ttl config DNS reply TTL in secs domain-id Domain ID dpi-cache-expiry (HIDDEN) Cache expiry time in minutes dpi-cache-size (HIDDEN) Cache size dpi-disable-track-tx (HIDDEN) Enable/Disable DPI TRACK TX dpi-enable (HIDDEN) Enable/Disable DPI dpi-gc-time (HIDDEN) Garbage collect time in secs dpi-multicore (HIDDEN) Enable multi-core for dpi dpi-stat-time (HIDDEN) Stats collection time for dpi environment fnf fp-buffer-check (HIDDEN) Enable fastpath buffer validity check fp-qos-interval config fp qos interval fp-qos-weight-percent-factor config fp qos weight percent factor gps-location GPS latitude and longitude of the device host-policer-pps Rate at which to police packets bound to the control plane (in pps) per QOS level icmp-error-pps Rate at which to police ICMP error messages either generated or received (in pps). idle-timeout Idle CLI timeout, in minutes ignore increase-bp-count (HIDDEN) Increase port backpressure threshold for all ports is-vmanaged Device is managed by the vmanage last-vmanage-transaction-id Used by vManage to maintain integrity of transactions initiated by it towards device location Location description of the device max-controllers (DEPRECATED) Set the maximum number of controllers to which the device can connect - Deprecated in 15.4 max-omp-sessions Set the maximum number of OMP sessions the device can have mode-button mtu on-demand Set various configuration for On-demand tunnels organization-name Organization name overlay-id Overlay ID patch-confirm (HIDDEN) Configure software patch confirmation timeout port-bp-threshold configure port backpressure threshold port-hop Enable port hopping for all tlocs port-offset Port offset (unique value; use only if multiple Viptela devices are behind the same NAT) pseudo-confirm-commit Only valid for vmanage .. reboot-on-failure (HIDDEN) Reboot device if any daemon fails simulated-color Simulated device's color simulated-devices Additional number of simulated devices simulated-wan-ip Starting IP address for the simulated interface site-id Site ID sp-organization-name Service Provider Organization name system-ip System IP address system-tunnel-mtu Control tunnel MTU tcp-optimization-enabled Carve out a dedicated core to use for TCP optimization - applies after reboot timer Set various timer timeouts tls ssl-opt cert management config track-default-gateway Enable/Disable default gateway tracking track-interface-tag OMP Tag attached to routes based on interface tracking track-transport Enable transport tracking unpin-flows-with-reboot (HIDDEN) Enable with reboot OR Disable flow pinning to FP cores upgrade-confirm Configure software upgrade confirmation timeout vbond Configure remote vBond or local IPv4 vbond address ztp-status ZTP status <cr>
To switch the device to CLI mode, use is-vmanaged option.
CSR01(config)# interface GigabitEthernet 2 CSR01(config-if)# ip address 21.1.1.2 255.255.255.0 CSR01(config-if)# commit Aborted: 'system is-vmanaged': This device is being managed by the vManage. Configuration through the CLI is not allowed. CSR01(config-if)# exit CSR01(config)# system is-vmanaged false CSR01(config-system)# commit Commit complete. CSR01#show run int GigabitEthernet 2 Building configuration... interface GigabitEthernet2 description Transport Interface ip dhcp client default-router distance 1 ip address 21.1.1.2 255.255.255.0
The device will connect to vManage. However, the controller will not overwrite the configuration. At this stage, vManage’s view of the router configuration will not reflect the actual state. We need to remediate this as soon as possible, either by switching the device to CLI mode or applying a configuration template that brings consistency.
Let’s update the device template to use static interface configuration to get to the consistent state. In the screenshot below, we see the template adjustments. The bottom part of the figure contains the snippet of the command difference. As we saw in the previous command listing, the device already had a static IP address configured, but vManage is not aware of that change, so from its perspective, the device still has DHCP settings configured.
Changing WAN Edge System IP
In the final section of this blog post, let’s see another example when switching to CLI mode can be helpful. When done via template change, the change of System IP can cause the configuration push to be stuck and not apply correctly.
As a workaround, you can switch the device into CLI mode, change System IP, and move it to template configuration back. Follow the steps below.
Step 1. To switch the device to CLI mode, refer to the procedure described in the first section and Figure 2.
Step 2. Using CLI apply the configuration change. In the example below, we change System IP from 1.1.1.101 to 1.1.1.11.
Step 3. Move device to vManaged mode by applying template and ensuring that the System IP variable matches the new value.