Cisco SD-WAN CLI vs. vManage Mode
Cisco SD-WAN devices can be either in vManage or CLI mode.
In vManage mode, the configuration is performed on vManage and then pushed to the device. Local configuration changes are not allowed. In CLI mode, changes are performed locally on the device. vManage mode is the preferred and recommended option for most SD-WAN implementations. However, you can occasionally switch devices into CLI mode to perform specific tasks.
Administrators can connect to the device using SSH or serial console to use various show and debug commands in both modes.
We will focus on edge devices in this article. However, controllers can also be in one of these modes. Cisco-hosted controllers are initially provisioned in CLI mode and then converted to vManage mode by a network administrator. Some features require controllers to be switched to vManage mode.
vManage mode
Consider a sample network shown in Figure 1. Our task is to prepare an edge router for a remote office. We’ve applied the required feature and device templates with dynamic IP assignment on the Internet-facing interface. After the router is shipped to the remote location and plugged in, it cannot establish connectivity to vManage. We discover that the ISP doesn’t have DHCP enabled, and a static IP configuration is required.
Let’s assume that we can access the router via console cable attached to a laptop on-site.
Run the “show sdwan system status” command on the edge device to validate the device mode. In the sample output below, the CSR01 router is in vManage mode (vManaged: true).
CSR01#show sdwan system status
Viptela (tm) vEdge Operating System Software
Copyright (c) 2013-2021 by Viptela, Inc.
Controller Compatibility: 20.3
Version: 17.03.03.0.4762
<output omitted>
Personality: vEdge
Model name: CSR1000V
Services: None
vManaged: true
Commit pending: false
Configuration template: CSR01
<output omitted>
Firstly, we try to set a static IP address on the transport interface using CLI. We use the “config-t” command to enter the configuration mode. The “commit” command activates the configuration.
CSR01#config-t
CSR01(config)# interface GigabitEthernet 2
CSR01(config-if)# ip address 21.1.1.2 255.255.255.252
CSR01(config-if)# commit
The following warnings were generated:
'system is-vmanaged': This device is being managed by the vManage. Any configuration changes to this device will be overwritten by the vManage after the control connection to the vManage comes back up.
Proceed? [yes,no] yes
Commit complete.
CSR01#show run interface GigabitEthernet 2
Building configuration...
<output omitted>
interface GigabitEthernet2
description Transport Interface
ip address 21.1.1.2 255.255.255.252
In our example, we didn’t have connectivity to vManage, so the changes were committed. However, as the warning message above advises vManage will overwrite modifications done on the device (see the example in the following listing) once the control connection is up.
CSR01#show run interface GigabitEthernet 2
Building configuration...
<output omitted>
interface GigabitEthernet2
description Transport Interface
ip address dhcp
Rollback to the original configuration (i.e. setting dynamic IP on the transport interface) causes connectivity to vManage fail, which introduces the connectivity problem we tried to solve in the first place. When we have connectivity to vManage, the CLI commands are just rejected, as shown in the listing below. vManage mode doesn’t permit changes on the edge device.
CSR01#config-t
CSR01(config)# interface GigabitEthernet2
CSR01(config-if)# ip address 21.1.1.2 255.255.255.252
CSR01(config-if)# commit
Aborted: 'system is-vmanaged': This device is being managed by the vManage. Configuration through the CLI is not allowed.
CLI Mode
There are several solutions to the problem described above. The first is to switch the device into CLI mode beforehand before sending it to the remote location. We can then adjust the configuration and set static IP address, and move the device back to vManage mode.
The second solution is using hidden support commands, which is not a supported way of configuring the device unless directed by Cisco TAC. However, it works and can be helpful in lab environments or as a quick fix to get the branch running (with full router reset later).
Let’s see how both options work with the examples below.
Switching device to CLI mode via vManage
An administrator can switch the device to CLI mode (and back) via vManage. Navigate to Configuration > Devices. Click on the Change Mode drop-down menu and select CLI mode.
In the dialog window, select and move to the right panel the router you want to switch into CLI mode and click on Upgrade to CLI Mode.
After the conversion, we can use “show sdwan system status” command to validate that the vManaged property is false.
CSR01#show sdwan system status
Viptela (tm) vEdge Operating System Software
Copyright (c) 2013-2021 by Viptela, Inc.
Controller Compatibility: 20.3
Version: 17.03.03.0.4762
Build: Not applicable
<output omitted>
Personality: vEdge
Model name: CSR1000V
Services: None
vManaged: false
Commit pending: false
Configuration template: None
<output omitted>
Now we can change the IP address via CLI.
CSR01#config-t
CSR01(config)# interface GigabitEthernet 2
CSR01(config-if)# ip address 21.1.1.2 255.255.255.2
CSR01(config-if)# commit
Switching device back to vManage mode
After the device comes back online, we can update templates to include static IP address configuration and switch it back to the vManage mode. To do this, we need to attach the device template to the vEdge router. If the device had a template attached before, the variable values would be populated automatically.
Switching device to CLI mode using hidden commands
This procedure in this section uses hidden commands and should be used only as a last resort. Cisco TAC may not support it; use it at your own risk.
When the router hasn’t been switched to CLI mode beforehand and cannot establish a control connection to vManage, refer to the following example to switch to CLI mode.
After entering the configuration mode, let’s see available options under the system command:
CSR01#config-t
CSR01(config)# system ?
Possible completions:
admin-tech-on-failure Collect admin-tech before reboot due to daemon failure
allow-same-site-tunnels Allow tunnels to be formed between vEdges in the same site
console-baud-rate Console baud-rate
control-session-pps Control session policer rate, in packets per second
controller-group-list Controller group list
debug
description System description
device-groups List of vManage groups to which the device belongs
disable
domain-id Domain ID
environment
fnf
gps-location GPS latitude and longitude of the device
host-policer-pps Rate at which to police packets bound to the control plane (in pps) per QOS level
icmp-error-pps Rate at which to police ICMP error messages either generated or received (in pps).
idle-timeout Idle CLI timeout, in minutes
ignore
location Location description of the device
max-controllers (DEPRECATED) Set the maximum number of controllers to which the device can connect - Deprecated in 15.4
max-omp-sessions Set the maximum number of OMP sessions the device can have
mode-button
mtu
on-demand Set various configuration for On-demand tunnels
organization-name Organization name
overlay-id Overlay ID
port-hop Enable port hopping for all tlocs
port-offset Port offset (unique value; use only if multiple Viptela devices are behind the same NAT)
site-id Site ID
sp-organization-name Service Provider Organization name
system-ip System IP address
system-tunnel-mtu Control tunnel MTU
tcp-optimization-enabled Carve out a dedicated core to use for TCP optimization - applies after reboot
timer Set various timer timeouts
tls ssl-opt cert management config
track-default-gateway Enable/Disable default gateway tracking
track-interface-tag OMP Tag attached to routes based on interface tracking
track-transport Enable transport tracking
upgrade-confirm Configure software upgrade confirmation timeout
vbond Configure remote vBond or local IPv4 vbond address
<cr>
“unhide viptela_internal” command enables the display of “system” command hidden options. The example below shows these subcommands.
CSR01(config)# unhide viptela_internal
CSR01(config)# system ?
Possible completions:
admin-tech-on-failure Collect admin-tech before reboot due to daemon failure
allow-same-site-tunnels Allow tunnels to be formed between vEdges in the same site
allow-sw-vedge (HIDDEN) Allow non-release software vedges to operate without certificates
console-baud-rate Console baud-rate
control-session-pps Control session policer rate, in packets per second
controller-group-list Controller group list
daemon-reboot (HIDDEN) Reboot device if a non-restartable daemon fails
daemon-restart (HIDDEN) Restart restartable daemons if they fail
debug
description System description
device-groups List of vManage groups to which the device belongs
disable
dnsd-ttl config DNS reply TTL in secs
domain-id Domain ID
dpi-cache-expiry (HIDDEN) Cache expiry time in minutes
dpi-cache-size (HIDDEN) Cache size
dpi-disable-track-tx (HIDDEN) Enable/Disable DPI TRACK TX
dpi-enable (HIDDEN) Enable/Disable DPI
dpi-gc-time (HIDDEN) Garbage collect time in secs
dpi-multicore (HIDDEN) Enable multi-core for dpi
dpi-stat-time (HIDDEN) Stats collection time for dpi
environment
fnf
fp-buffer-check (HIDDEN) Enable fastpath buffer validity check
fp-qos-interval config fp qos interval
fp-qos-weight-percent-factor config fp qos weight percent factor
gps-location GPS latitude and longitude of the device
host-policer-pps Rate at which to police packets bound to the control plane (in pps) per QOS level
icmp-error-pps Rate at which to police ICMP error messages either generated or received (in pps).
idle-timeout Idle CLI timeout, in minutes
ignore
increase-bp-count (HIDDEN) Increase port backpressure threshold for all ports
is-vmanaged Device is managed by the vmanage
last-vmanage-transaction-id Used by vManage to maintain integrity of transactions initiated by it towards device
location Location description of the device
max-controllers (DEPRECATED) Set the maximum number of controllers to which the device can connect - Deprecated in 15.4
max-omp-sessions Set the maximum number of OMP sessions the device can have
mode-button
mtu
on-demand Set various configuration for On-demand tunnels
organization-name Organization name
overlay-id Overlay ID
patch-confirm (HIDDEN) Configure software patch confirmation timeout
port-bp-threshold configure port backpressure threshold
port-hop Enable port hopping for all tlocs
port-offset Port offset (unique value; use only if multiple Viptela devices are behind the same NAT)
pseudo-confirm-commit Only valid for vmanage ..
reboot-on-failure (HIDDEN) Reboot device if any daemon fails
simulated-color Simulated device's color
simulated-devices Additional number of simulated devices
simulated-wan-ip Starting IP address for the simulated interface
site-id Site ID
sp-organization-name Service Provider Organization name
system-ip System IP address
system-tunnel-mtu Control tunnel MTU
tcp-optimization-enabled Carve out a dedicated core to use for TCP optimization - applies after reboot
timer Set various timer timeouts
tls ssl-opt cert management config
track-default-gateway Enable/Disable default gateway tracking
track-interface-tag OMP Tag attached to routes based on interface tracking
track-transport Enable transport tracking
unpin-flows-with-reboot (HIDDEN) Enable with reboot OR Disable flow pinning to FP cores
upgrade-confirm Configure software upgrade confirmation timeout
vbond Configure remote vBond or local IPv4 vbond address
ztp-status ZTP status
<cr>
To switch the device to CLI mode, use is-vmanaged option.
CSR01(config)# interface GigabitEthernet 2
CSR01(config-if)# ip address 21.1.1.2 255.255.255.0
CSR01(config-if)# commit
Aborted: 'system is-vmanaged': This device is being managed by the vManage. Configuration through the CLI is not allowed.
CSR01(config-if)# exit
CSR01(config)# system is-vmanaged false
CSR01(config-system)# commit
Commit complete.
CSR01#show run int GigabitEthernet 2
Building configuration...
interface GigabitEthernet2
description Transport Interface
ip dhcp client default-router distance 1
ip address 21.1.1.2 255.255.255.0
The device will connect to vManage. However, the controller will not overwrite the configuration. At this stage, vManage’s view of the router configuration will not reflect the actual state. We need to remediate this as soon as possible, either by switching the device to CLI mode or applying a configuration template that brings consistency.
Let’s update the device template to use static interface configuration to get to the consistent state. In the screenshot below, we see the template adjustments. The bottom part of the figure contains the snippet of the command difference. As we saw in the previous command listing, the device already had a static IP address configured, but vManage is not aware of that change, so from its perspective, the device still has DHCP settings configured.
Changing WAN Edge System IP
In the final section of this blog post, let’s see another example when switching to CLI mode can be helpful. When done via template change, the change of System IP can cause the configuration push to be stuck and not apply correctly.
As a workaround, you can switch the device into CLI mode, change System IP, and move it to template configuration back. Follow the steps below.
Step 1. To switch the device to CLI mode, refer to the procedure described in the first section and Figure 2.
Step 2. Using CLI apply the configuration change. In the example below, we change System IP from 1.1.1.101 to 1.1.1.11.
Step 3. Move device to vManaged mode by applying template and ensuring that the System IP variable matches the new value.