On AD
- Make server domain
- Add Certificate role
- Create DNS entries for network devices
- Create root cert
vManage
config
system
host-name vManage
system-ip 10.200.200.1
site-id 100
admin-tech-on-failure
sp-organization-name hwtechnet
organization-name hwtechnet
clock timezone America/Toronto
vbond vbond.hwtechnet.com
!
ntp
server ntp.hwtechnet.com
version 4
exit
!
!
vpn 0
host vbond.hwtechnet.com ip 10.1.100.3
dns 10.1.100.230 primary
interface eth0
ip address 10.1.100.1/24
!
no shutdown
!
ip route 0.0.0.0/0 10.1.100.253
commit and-quit
vSmart
config
system
host-name vSmart
system-ip 10.200.200.2
site-id 100
admin-tech-on-failure
sp-organization-name hwtechnet
organization-name hwtechnet
clock timezone America/Toronto
vbond vbond.hwtechnet.com
!
ntp
server ntp.hwtechnet.com
version 4
exit
!
!
vpn 0
host vbond.hwtechnet.com ip 10.1.100.3
dns 10.1.100.230 primary
interface eth0
ip address 10.1.100.2/24
!
no shutdown
!
ip route 0.0.0.0/0 10.1.100.253
commit and-quit
vBond-1
config
system
host-name vBond-1
system-ip 10.200.200.3
site-id 100
admin-tech-on-failure
no route-consistency-check
organization-name hwtechnet
clock timezone America/Toronto
vbond vbond.hwtechnet.com local
ntp
server ntp.hwtechnet.com
version 4
exit
!
!
vpn 0
host vbond.hwtechnet.com ip 10.1.100.3
dns 10.1.100.230 primary
interface ge0/0
ip address 10.1.100.3/24
no tunnel-interface
no shutdown
!
ip route 0.0.0.0/0 10.1.100.253
commit and-quit
After adding another vBond
In order to resync vManage Database we must enter vmanage FQDN and enter this address:
https://vmanage.hwtechnet.com.com/dataservice/system/device/sync/rootcertchain
- The result must be
{"syncRootCertChain":"done"}
in JSON format:
Adding Controllers through the vManage GUI
- Now we login to the vManage Web-GUI and go to the Configuration > Devices section and then Controllers Tab, and add the controllers to the vManage:
- Make Sure to uncheck Generate CSR box. Make Sure to uncheck Generate CSR box. ke Sure to uncheck Generate CSR box. Now we go to the Administration > Settings section on vManage, and put some information there, including vBond address, Organization Name, and change the Controller CA to Enterprise Root Certificate and select the Root Certificate file. We don’t need to specify CSR propertise there, but make sure to put the correct consistant (in the entire SD-WAN domain) Organization-Name, otherwise you will face problem when you want to add vEdge or cEdge devices to the SD-WAN domain.
vEdge configurations
vEdge-51
config
system
host-name vEdge-51
system-ip 10.200.200.51
site-id 50
admin-tech-on-failure
no route-consistency-check
organization-name hwtechnet
clock timezone America/Toronto
vbond vbond.hwtechnet.com
!
ntp
server ntp.hwtechnet.com
version 4
exit
!
!
vpn 0
dns 10.1.100.230 primary
router
ospf
timers spf 200 1000 10000
area 0
interface ge0/3
exit
exit
!
!
interface ge0/0
ip address 200.50.1.51/24
ipv6 dhcp-client
nat
!
tunnel-interface
encapsulation ipsec
color biz-internet
allow-service all
!
no shutdown
!
interface ge0/3
ip address 172.16.51.51/24
tunnel-interface
encapsulation ipsec
color mpls
allow-service all
!
no shutdown
!
ip route 0.0.0.0/0 200.50.1.92
!
commit
On vEdge devices, we configure System and VPN 0 settings. For vEdge-51, which uses MPLS transport, we set up OSPF as the PE-CE protocol with the ISP. This involves configuring OSPF and enabling it on the MPLS-facing WAN interface (ge0/3).
Next, we configure VPN 0 interfaces, enable tunnel-interface on them, set the tunnel encapsulation to IPSec, and assign a color to the link. It's important to allow OSPF service on the tunnel interface to form an OSPF neighborship with the ISP PE device. While we use allow-service all
here, in a production environment, only necessary services should be allowed.
After configuring the vEdge device, we upload the root_ca.cer
file to the /home/admin/
directory using SFTP. We then install it using the command.
Send cert using SCP
To use scp
to send a file from vManage (IP: 10.1.100.1) to vEdge devices, follow these steps:
-
Ensure you have SSH access: Make sure you can SSH from vManage to vEdge. You may need the correct username and password or SSH keys set up.
-
Use the
scp
command: On the vManage terminal, use the following command to copy a file (e.g.,root_ca.cer
) to vEdge:scp /path/to/root_ca.cer admin@200.50.1.51:/home/admin/
Replace
/path/to/root_ca.cer
with the actual path of the file on the vManage system. -
Authentication: If prompted, enter the password for the
admin
user on the vEdge device.
Here's a step-by-step breakdown:
-
Open terminal on vManage:
ssh admin@10.1.100.1
-
Execute the
scp
command:scp /path/to/root_ca.cer admin@200.50.1.51:/home/admin/
-
Enter the password when prompted to complete the file transfer.
This will securely copy the root_ca.cer
file from the vManage device to the specified directory on the vEdge device.
request root-cert-chain install /home/admin/root_ca.cer priv exec mode
Registering Edge Devices
-
Navigate to Device Configuration
- Go to
Configuration > Devices
.
- Go to
-
Generate Bootstrap Configuration
- Click on the ellipsis on the right side of the device in the inventory.
- Select "Generate Bootstrap Configuration."
-
Select Bootstrap Configuration Options
- For Bootstrap Configuration, select "Cloud-init."
- Keep "Include Default Root Certificate" selected and click OK.
-
Gather Required Information
- This information will be used to register the vEdge device through the CLI.
- Depending on the version of vManage, find the following:
- Chassis Number: Copy the content next to or under "uuid" (e.g.,
e1098056-90c4-6ba3-6e8d-6ec36e7fa6ea
). - Token Number: Copy the content under "otp" (e.g.,
eb018e42db2a44feaef059edce163ad1
).
- Chassis Number: Copy the content next to or under "uuid" (e.g.,
-
Activate the vEdge Device
- Use the vEdge device's CLI to enter the following command:
request vedge-cloud activate chassis-number e1098056-90c4-6ba3-6e8d-6ec36e7fa6ea token eb018e42db2a44feaef059edce163ad1
- Use the vEdge device's CLI to enter the following command:
-
Verify Activation
- Wait a few seconds to see the device's hostname and site-id pop up in the inventory. It should now show as "In Sync."
- Confirm on the CLI with the following command:
show control local-properties
- This command provides information on the vBond, the certificate installation, chassis number/unique-id, serial number, and more.
-
Repeat for All Edge Devices
- Follow the same steps for all edge devices.
-
Output of Control Connections Command
- Example output of the
show control connections
command on vEdge-51:vEdge-51# show control connections PEER PEER CONTROLLER PEER PEER PEER SITE DOMAIN PEER PRIV PEER PUB GROUP TYPE PROT SYSTEM IP ID ID PRIVATE IP PORT PUBLIC IP PORT ORGANIZATION LOCAL COLOR PROXY STATE UPTIME ID ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- vsmart dtls 10.200.200.2 250 1 10.1.100.2 12446 10.1.100.2 12446 hwtechnet mpls No up 0:01:20:58 0 vsmart dtls 10.200.200.2 250 1 10.1.100.2 12446 10.1.100.2 12446 hwtechnet biz-internet No up 0:01:20:58 0 vbond dtls 0.0.0.0 0 0 10.1.100.3 12346 10.1.100.3 12346 hwtechnet mpls - up 0:01:20:59 0 vbond dtls 0.0.0.0 0 0 10.1.100.3 12346 10.1.100.3 12346 hwtechnet biz-internet - up 0:01:20:59 0 vmanage dtls 10.200.200.1 250 0 10.1.100.1 12446 10.1.100.1 12446 hwtechnet biz-internet No up 0:01:21:00 0
- Example output of the
-
Verify DTLS Control Connections
- Ensure DTLS control connections are up using both WAN links.
- Check that vEdge-51 has formed OMP peering with the vSmart Controller. Note that in a Hub-and-Spoke overlay control plane topology, every Edge device forms an OMP peering with vSmart controllers. If there were redundant vSmart controllers, vEdge-51 would form OMP peering with both of them, not with other vEdge devices.
- Remember, the default Data-Plane topology in Cisco SD-WAN is Fully-Meshed, allowing every vEdge to form an IPSec tunnel with other vEdge devices when needed for direct traffic.
-
Output of Control Local Properties Command
- Example output of the
show control local-properties
command:INDEX IP PORT ----------------------------------------------------- 0 10.1.100.3 12346 number-active-wan-interfaces 2 NAT TYPE: E -- indicates End-point independent mapping A -- indicates Address-port dependent mapping N -- indicates Not learned Note: Requires minimum two vbonds to learn the NAT type RESTRICT/ LAST VM PUBLIC PUBLIC PRIVATE PRIVATE PRIVATE MAX CONTROL/ LAST SPI TIME NAT CON REGION INTERFACE IPv4 PORT IPv4 IPv6 PORT VS/VM COLOR STATE CNTRL STUN LR/LB CONNECTION REMAINING TYPE PRF IDs ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ge0/0 200.50.1.51 12366 200.50.1.51 :: 12366 1/1 biz-internet up 2 no/yes/no No/No 0:00:00:06 0:11:43:50 N 5 Default ge0/3 172.16.51.51 12366 172.16.51.51 :: 12366 1/0 mpls up 2 no/yes/no No/No 0:00:00:08 0:11:43:48 N 5 Default
- Example output of the
vEdge-52 Configuration
vEdge-52:
config
system
host-name vEdge-52
system-ip 10.200.200.52
site-id 50
admin-tech-on-failure
no route-consistency-check
organization-name hwtechnet
clock timezone America/Toronto
vbond vbond.hwtechnet.com
!
ntp
server ntp.hwtechnet.com
version 4
exit
!
!
vpn 0
dns 10.1.100.230 primary
router
ospf
timers spf 200 1000 10000
area 0
interface ge0/3
exit
exit
!
!
interface ge0/0
ip address 200.50.2.52/24
ipv6 dhcp-client
nat
!
tunnel-interface
encapsulation ipsec
color biz-internet
allow-service all
!
no shutdown
!
interface ge0/3
ip address 172.16.52.52/24
tunnel-interface
encapsulation ipsec
color mpls
allow-service all
!
no shutdown
!
ip route 0.0.0.0/0 200.50.2.94
!
vEdge-10 Configuration
vEdge-10:
config
system
host-name vEdge-10
system-ip 10.200.200.10
site-id 10
admin-tech-on-failure
no route-consistency-check
organization-name hwtechnet
clock timezone America/Toronto
vbond vbond.hwtechnet.com
!
ntp
server ntp.hwtechnet.com
version 4
exit
!
!
vpn 0
dns 10.1.100.230 primary
interface ge0/0
ip address 192.168.10.10/24
ipv6 dhcp-client
tunnel-interface
encapsulation ipsec
color biz-internet
allow-service all
!
no shutdown
!
ip route 0.0.0.0/0 192.168.10.11
!
vEdge-100 Configuration
vEdge-100:
config
system
host-name vEdge-100
system-ip 10.200.200.100
site-id 100
admin-tech-on-failure
no route-consistency-check
sp-organization-name hwtechnet
organization-name hwtechnet
clock timezone America/Toronto
vbond vbond.hwtechnet.com
!
ntp
server ntp.hwtechnet.com
version 4
exit
!
!
vpn 0
dns 10.1.100.230 primary
router
ospf
timers spf 200 1000 10000
area 0
interface ge0/2
exit
exit
!
!
interface ge0/1
ip address 200.100.1.100/24
tunnel-interface
encapsulation ipsec
color biz-internet
allow-service all
!
no shutdown
!
interface ge0/2
ip address 172.16.100.100/24
tunnel-interface
encapsulation ipsec
color mpls
allow-service all
!
no shutdown
!
ip route 0.0.0.0/0 200.100.1.94
!
vEdge-20 Configuration
vEdge-20:
config
system
host-name vEdge-20
system-ip 10.200.200.20
site-id 20
admin-tech-on-failure
no route-consistency-check
sp-organization-name hwtechnet
organization-name hwtechnet
clock timezone America/Toronto
vbond vbond.hwtechnet.com
!
ntp
server ntp.hwtechnet.com
version 4
exit
!
!
vpn 0
dns 10.1.100.230 primary
router
bgp 65020
neighbor 172.16.22.93
no shutdown
remote-as 100
address-family ipv4-unicast
!
!
neighbor 200.10.2.95
no shutdown
remote-as 100
address-family ipv4-unicast
!
!
!
!
interface ge0/0
ip address 172.16.22.20/24
tunnel-interface
encapsulation ipsec
color mpls
allow-service all
!
no shutdown
!
interface ge0/1
ip address 200.10.2.20/24
tunnel-interface
encapsulation ipsec
color biz-internet
allow-service all
!
The PE-CE routing protocol for both Transport Links (WAN Links) of vEdge-20 is BGP, that is why we have configured BGP on vEdge-20 VPN 0 instance:
For the vEdge-31 and vEdge-32 we have used an interesting feature name TLOC Extension. If you take a look at the topology you realize that vEdge-31 has only MPLS Transport link from the ISP using it’s ge0/0 interface and vEdge-32 has only Internet Transport WAN Link from ISP using it’s ge0/0 interface.
We can extend the WAN link to the other vEdges using TLOC Extension feature. For example vEdge-31 can extend it’s MPLS WAN link to the ge0/3 interface, so vEdge-32 can use ge0/3 in order to use it for it’s MPLS WAN transport. The configuration is simple:
vEdge-31:
system
host-name vEdge-31
system-ip 10.200.200.31
site-id 30
admin-tech-on-failure
no route-consistency-check
sp-organization-name hwtechnet
organization-name hwtechnet
clock timezone America/Toronto
vbond vbond.hwtechnet.com
!
ntp
server ntp.hwtechnet.com
version 4
exit
!
!
vpn 0
dns 10.1.100.230 primary
router
ospf
timers spf 200 1000 10000
area 0
interface ge0/0
exit
interface ge0/3
exit
exit
!
!
interface ge0/0
ip address 172.16.31.31/24
tunnel-interface
encapsulation ipsec
color mpls restrict
allow-service all
!
no shutdown
!
interface ge0/1
ip address 31.32.200.31/24
tunnel-interface
encapsulation ipsec
color biz-internet
allow-service all
!
no shutdown
!
interface ge0/3
ip address 31.32.172.31/24
tloc-extension ge0/0
no shutdown
!
ip route 0.0.0.0/0 31.32.200.32
In the color mpls color mpls color mpls command section, we have added restrict restrict restrict option and also we specified that ge0/3 is going to be used to extend ge0/0 to the vEdge-32.
For the vEdge-32 we do the same, but this time for Internet Transport:
vEdge-32:
system
host-name vEdge-32
system-ip 10.200.200.32
site-id 30
admin-tech-on-failure
no route-consistency-check
sp-organization-name hwtechnet
organization-name hwtechnet
clock timezone America/Toronto
vbond vbond.hwtechnet.com
!
ntp
server ntp.hwtechnet.com
version 4
exit
!
!
vpn 0
dns 10.1.100.230 primary
router
ospf
timers spf 200 1000 10000
area 0
interface ge0/3
exit
exit
!
!
interface ge0/0
ip address 200.30.1.32/24
!
tunnel-interface
encapsulation ipsec
color biz-internet restrict
allow-service all
!
no shutdown
!
interface ge0/1
ip address 31.32.200.32/24
tloc-extension ge0/0
no shutdown
!
interface ge0/3
ip address 31.32.172.32/24
tunnel-interface
encapsulation ipsec
color mpls
allow-service all
!
no shutdown
!
ip route 0.0.0.0/0 200.30.1.95
!
The PE-CE routing protocol for both Transport Links (WAN Links) of vEdge-20 is BGP, which is why we have configured BGP on the vEdge-20 VPN 0 instance.
For vEdge-31 and vEdge-32, we have utilized the TLOC Extension feature. As you can see in the topology, vEdge-31 has only an MPLS Transport link from the ISP using its ge0/0 interface, and vEdge-32 has only an Internet Transport WAN Link from the ISP using its ge0/0 interface.
We can extend the WAN link to the other vEdges using the TLOC Extension feature. For instance, vEdge-31 can extend its MPLS WAN link to the ge
0/3 interface, allowing vEdge-32 to use ge0/3 for its MPLS WAN transport. The configuration is straightforward:
In the command section for the color mpls, we have added the restrict option and specified that ge0/3 will be used to extend ge0/0 to vEdge-32.
For vEdge-32, we do the same, but this time for the Internet Transport:
vEdge-32:
system
host-name vEdge-32
system-ip 10.200.200.32
site-id 30
admin-tech-on-failure
no route-consistency-check
organization-name hwtechnet
clock timezone America/Toronto
vbond vbond.hwtechnet.com
!
ntp
server ntp.hwtechnet.com
version 4
exit
!
!
vpn 0
dns 10.1.100.230 primary
router
ospf
timers spf 200 1000 10000
area 0
interface ge0/3
exit
exit
!
!
interface ge0/0
ip address 200.30.1.32/24
!
tunnel-interface
encapsulation ipsec
color biz-internet restrict
allow-service all
!
no shutdown
!
interface ge0/1
ip address 31.32.200.32/24
tloc-extension ge0/0
no shutdown
!
interface ge0/3
ip address 31.32.172.32/24
tunnel-interface
encapsulation ipsec
color mpls
allow-service all
!
no shutdown
!
ip route 0.0.0.0/0 200.30.1.95
!
Converting IOS-XE to SD-WAN and Initial Configuration
Before we start adding the vEdge devices, we need to convert the IOS-XE device to an SD-WAN device and configure the initial settings.
Steps to Convert IOS-XE to SD-WAN
-
Load the SD-WAN Image:
- Ensure the IOS-XE device is running the SD-WAN image. If not, load the appropriate SD-WAN image using the following commands:
- Go to vManage to create a ciscosdwan_cloud_init.cfg using custom template file and tftp it to the device then enter the following command.
- Make sure to add the cert file before putting it in Controller mode
controller-mode enable [confirm] yes
-
Initial Configuration for cEdge Device:
- Once the device has rebooted into the SD-WAN image, configure the initial settings:
system system-ip 10.200.200.40 domain-id 1 site-id 40 admin-tech-on-failure sp-organization-name hwtechnet organization-name hwtechnet vbond vbond.hwtechnet.com ! memory free low-watermark processor 72347 no service pad service password-encryption service timestamps debug datetime msec service timestamps log datetime msec no service tcp-small-servers no service udp-small-servers platform console serial platform punt-keepalive disable-kernel-core hostname cEdge-40 username admin privilege 15 secret 5 $1$jlBF$eVLnifZfTd6m2Iw6ICl9X1 vrf definition 100 address-family ipv4 exit-address-family ! ! vrf definition 200 address-family ipv4 exit-address-family ! ! no ip dhcp use class ip name-server 10.1.100.100 ip route 0.0.0.0 0.0.0.0 200.40.1.92 no ip source-route ip ssh version 2 ip access-list extended meraki-fqdn-dns ! no ip http server ip http secure-server no ip http ctc authentication no ip igmp ssm-map query dns ipv6 unicast-routing interface GigabitEthernet1 no shutdown ip address 200.40.1.40 255.255.255.0 no mop enabled no mop sysid negotiation auto exit interface GigabitEthernet2 no shutdown ip address 172.16.40.40 255.255.255.0 ip ospf 40 area 0 ip ospf network point-to-point no mop enabled no mop sysid negotiation auto exit interface GigabitEthernet3 no shutdown vrf forwarding 100 ip address 192.168.40.40 255.255.255.0 no mop enabled no mop sysid negotiation auto exit interface GigabitEthernet4 no shutdown vrf forwarding 200 ip address 192.168.140.40 255.255.255.0 no mop enabled no mop sysid negotiation auto exit interface Tunnel1 no shutdown ip unnumbered GigabitEthernet1 tunnel source GigabitEthernet1 tunnel mode sdwan exit interface Tunnel2 no shutdown ip unnumbered GigabitEthernet2 tunnel source GigabitEthernet2 tunnel mode sdwan exit logging buffered 262144 no logging rate-limit aaa authentication login default local aaa authorization exec default local login on-success log no crypto ikev2 diagnose error no crypto isakmp diagnose error router ospf 40 ! line con 0 login authentication default stopbits 1 ! line vty 0 4 login authentication default transport input ssh ! line vty 5 80 login authentication default transport input ssh ! ntp server ntp.hwtechnet.com sdwan interface GigabitEthernet1 tunnel-interface encapsulation ipsec color biz-internet allow-service all no allow-service bgp allow-service dhcp allow-service dns allow-service icmp no allow-service sshd no allow-service netconf no allow-service ntp no allow-service ospf no allow-service stun no allow-service snmp exit exit interface GigabitEthernet2 tunnel-interface encapsulation ipsec color mpls allow-service all no allow-service bgp allow-service dhcp allow-service dns allow-service icmp no allow-service sshd no allow-service netconf no allow-service ntp no allow-service ospf no allow-service stun no allow-service snmp exit exit omp no shutdown graceful-restart no as-dot-notation address-family ipv4 advertise connected advertise static ! address-family ipv6 advertise connected advertise static ! ! ! licensing config enable false licensing config privacy hostname false licensing config privacy version false licensing config utility utility-enable false netconf-yang cisco-ia blocking cli-blocking-enabled omp no as-dot-notation ! security ipsec authentication-type ah-sha1-hmac sha1-hmac ! ! nacm cmd-read-default deny nacm cmd-exec-default deny