The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.
This document describes the methods used in Performance Routing version 3 (PfRv3) to perform load balancing on Branch router's WAN links.
Cisco recommends that you have basic knowledge of Performance Routing version 3 (PfRv3).
This document is not restricted to specific software and hardware versions.
The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.
One of the main applications of PfR is WAN load balancing even on links with different physical characterstics like Delay, Jitter, Bandwidth. To do this PfR keeps a check of the link utilization levels on the WAN links to efficiently utilize them across various Traffic Classes(TC) flowing through the edge routers.
Traffic Classes are divided in two groups:
Note: Load Balancing only affects non-performance Traffic Classes.
There are four different roles a device can play in PfRv3 configuration:
Branch-master controller — The branch-master controller is the master controller at the branch site. There is no policy configuration on this device. It receives policy from the hub-master controller. This device acts as master controller for the branch site and makes optimization decision.
Branch- border router — The border device at the branch-site. There is no configuration other than enabling of PfRv3 border-master controller on the device. The WAN interface that terminates on the device is detected automatically.
Load-balancing mechanism in PfRv3 works only for traffic that gets classified in default class. When load balancing is disabled, PfRv3 deletes this default class and traffic is not load balanced and is routed based on the routing table information.
In PfRv3, load-balancing kicks in as soon as the difference in the link performance of the Border routers reaches 20% and the "load-balance" command is configured on the Hub-Master Controller. This value is fixed and non-configurable.
Note: The load-balancing is only achieved for the traffic-classes which are not speicified in the Hub-Master Controller policy list.
Following image would be used as a sample topology for rest of the document:
R1- Server, Initiating traffic.
R3- Hub-Master Controller.
R4- Hub-Border Router.
R5- Hub-Border Router.
R9- Branch-Master Controller for Spoke Location
R10- Branch-Master Controller for Spoke Location
R9 is having two DMVPN tunnels i.e. Tunnel 100 and Tunnel 200 . Tunnel 100 is terminating on R4 and Tunnel 200 is termintaing on R5 .
hostname R3
!
!
domain one
vrf default
master hub
source-interface Loopback0
load-balance -----> Command to enable PfRv3 Load-balancing
class TEST sequence 10
match dscp ef policy voice
path-preference INET1 fallback INET2
!
!
interface Loopback0
ip address 10.3.3.3 255.255.255.255
!
Note: Load-balance is disabled by default
hostname R4
!
!
domain one
vrf default
border
source-interface Loopback0
master 10.3.3.3
domain one path INET1
!
!
interface Loopback0
ip address 10.4.4.4 255.255.255.255
!
hostname R5
!
domain one
vrf default
border
source-interface Loopback0
master 10.3.3.3
domain one path INET2
!
!
interface Loopback0
ip address 10.5.5.5 255.255.255.255
R3 (Master Router) has been configured to keep sending traffic for all traffic classes.
R3#show domain one master status
*** Domain MC Status ***
Master VRF: Global
Instance Type: Hub
Instance id: 0
Operational status: Up
Configured status: Up
Loopback IP Address: 10.3.3.3
Load Balancing:
Admin Status: Enabled <<<<<<<<<<<<<<< Disabled by default
Operational Status: Up
Enterprise top level prefixes configured: 0
Max Calculated Utilization Variance: 13%
Last load balance attempt: 00:05:03 ago
Last Reason: Variance less than 20%
Total unbalanced bandwidth:
External links: 0 Kbps Internet links: 0 Kpbs
Route Control: Enabled
Mitigation mode Aggressive: Disabled
Policy threshold variance: 20
Minimum Mask Length: 28
Sampling: off
Borders:
IP address: 10.5.5.5
Connection status: CONNECTED (Last Updated 01:18:20 ago )
Interfaces configured:
Name: Tunnel200 | type: external | Service Provider: INET2 | Status: UP
Number of default Channels: 2
Tunnel if: Tunnel0
IP address: 10.4.4.4
Connection status: CONNECTED (Last Updated 01:18:15 ago )
Interfaces configured:
Name: Tunnel100 | type: external | Service Provider: INET1 | Status: UP
Number of default Channels: 2
Tunnel if: Tunnel0
R3#show domain one master traffic-classes summary
APP - APPLICATION, TC-ID - TRAFFIC-CLASS-ID, APP-ID - APPLICATION-ID
SP - SERVICE PROVIDER, PC = PRIMARY CHANNEL ID,
BC - BACKUP CHANNEL ID, BR - BORDER, EXIT - WAN INTERFACE
UC - UNCONTROLLED, PE - PICK-EXIT, CN - CONTROLLED, UK - UNKNOWN
Dst-Site-Pfx Dst-Site-Id APP DSCP TC-ID APP-ID State SP PC/BC BR/EXIT
10.10.14.0/24 10.9.9.9 N/A default 38 N/A CN INET2 3/4 10.5.5.5/Tunnel200
10.10.14.0/24 10.9.9.9 N/A af31 32 N/A CN INET2 5/6 10.5.5.5/Tunnel200
10.10.12.0/24 10.9.9.9 N/A default 34 N/A CN INET1 4/3 10.4.4.4/Tunnel100
10.10.12.0/24 10.9.9.9 N/A af31 29 N/A CN INET2 5/6 10.5.5.5/Tunnel200
10.10.10.0/24 10.9.9.9 N/A default 31 N/A CN INET1 4/3 10.4.4.4/Tunnel100
10.10.10.0/24 10.9.9.9 N/A af31 24 N/A CN INET2 5/6 10.5.5.5/Tunnel200
10.10.3.0/24 10.9.9.9 N/A default 15 N/A CN INET2 3/4 10.5.5.5/Tunnel200
10.10.3.0/24 10.9.9.9 N/A af31 10 N/A CN INET2 5/6 10.5.5.5/Tunnel200
10.10.8.0/24 10.9.9.9 N/A default 26 N/A CN INET2 3/4 10.5.5.5/Tunnel200
10.10.8.0/24 10.9.9.9 N/A af31 21 N/A CN INET2 5/6 10.5.5.5/Tunnel200
10.10.19.0/24 10.9.9.9 N/A default 8 N/A CN INET2 3/4 10.5.5.5/Tunnel200
10.10.19.0/24 10.9.9.9 N/A af31 1 N/A CN INET2 5/6 10.5.5.5/Tunnel200
10.10.17.0/24 10.9.9.9 N/A af31 39 N/A CN INET2 5/6 10.5.5.5/Tunnel200
10.10.17.0/24 10.9.9.9 N/A default 3 N/A CN INET2 3/4 10.5.5.5/Tunnel200
10.10.11.0/24 10.9.9.9 N/A default 33 N/A CN INET1 4/3 10.4.4.4/Tunnel100
10.10.11.0/24 10.9.9.9 N/A af31 27 N/A CN INET2 5/6 10.5.5.5/Tunnel200
10.10.2.0/24 10.9.9.9 N/A default 13 N/A CN INET2 3/4 10.5.5.5/Tunnel200
10.10.2.0/24 10.9.9.9 N/A af31 7 N/A CN INET2 5/6 10.5.5.5/Tunnel200
10.10.7.0/24 10.9.9.9 N/A default 25 N/A CN INET1 4/3 10.4.4.4/Tunnel100
10.10.7.0/24 10.9.9.9 N/A af31 18 N/A CN INET2 5/6 10.5.5.5/Tunnel200
10.10.18.0/24 10.9.9.9 N/A af31 40 N/A CN INET2 5/6 10.5.5.5/Tunnel200
10.10.18.0/24 10.9.9.9 N/A default 5 N/A CN INET1 4/3 10.4.4.4/Tunnel100
10.10.4.0/24 10.9.9.9 N/A default 19 N/A CN INET2 3/4 10.5.5.5/Tunnel200
10.10.4.0/24 10.9.9.9 N/A af31 12 N/A CN INET2 5/6 10.5.5.5/Tunnel200
10.10.9.0/24 10.9.9.9 N/A default 28 N/A CN INET1 4/3 10.4.4.4/Tunnel100
10.10.9.0/24 10.9.9.9 N/A af31 23 N/A CN INET2 5/6 10.5.5.5/Tunnel200
10.10.1.0/24 10.9.9.9 N/A default 11 N/A CN INET2 3/4 10.5.5.5/Tunnel200
10.10.13.0/24 10.9.9.9 N/A default 36 N/A CN INET1 4/3 10.4.4.4/Tunnel100
10.10.13.0/24 10.9.9.9 N/A af31 30 N/A CN INET2 5/6 10.5.5.5/Tunnel200
10.10.15.0/24 10.9.9.9 N/A af31 35 N/A CN INET2 5/6 10.5.5.5/Tunnel200
10.10.15.0/24 10.9.9.9 N/A default 2 N/A CN INET2 3/4 10.5.5.5/Tunnel200
10.10.0.0/16 10.9.9.9 N/A default 17 N/A CN INET2 3/4 10.5.5.5/Tunnel200
10.10.16.0/24 10.9.9.9 N/A af31 37 N/A CN INET2 5/6 10.5.5.5/Tunnel200
10.10.16.0/24 10.9.9.9 N/A default 4 N/A CN INET2 3/4 10.5.5.5/Tunnel200
10.10.5.0/24 10.9.9.9 N/A default 20 N/A CN INET1 4/3 10.4.4.4/Tunnel100
10.10.5.0/24 10.9.9.9 N/A af31 14 N/A CN INET2 5/6 10.5.5.5/Tunnel200
10.10.10.0/24 10.9.9.9 N/A default 9 N/A CN INET2 3/4 10.5.5.5/Tunnel200
10.10.10.0/24 10.9.9.9 N/A af31 6 N/A CN INET2 5/6 10.5.5.5/Tunnel200
10.10.6.0/24 10.9.9.9 N/A default 22 N/A CN INET1 4/3 10.4.4.4/Tunnel100
Total Traffic Classes: 39 Site: 39 Internet: 0
R3#show domain one master exits
BR address: 10.5.5.5 | Name: Tunnel200 | type: external | Path: INET2 |
Egress capacity: 1000 Kbps | Egress BW: 173 Kbps | Ideal:177 Kbps | under: 4 Kbps | Egress Utilization: 17 %
DSCP: default[0]-Number of Traffic Classes[14]
BR address: 10.4.4.4 | Name: Tunnel100 | type: external | Path: INET1 |
Egress capacity: 1000 Kbps | Egress BW: 181 Kbps | Ideal:177 Kbps | over: 4 Kbps | Egress Utilization: 18 %
DSCP: default[0]-Number of Traffic Classes[6]
DSCP: af31[26]-Number of Traffic Classes[19]
The above outputs shows that there are total 39 traffic classes being initialized from R1 out of which the default class traffic and af31 class traffic flows through R4 but only default class traffic flows through R5. The traffic class defined on Hub-Master Controller is only for traffic marked with DSCP EF. So, for load-balancing all traffic which is marked non-EF will be considered which is DSCP 0 and DSCP 26 i.e. AF31.
In order to depict load-balancing, the bandwidth of the external link (Tunnel 100) of R4 interface is modified to 500Kbps from 1000 Kbps.
R4#sh run int tunnel 100
Building configuration...
Current configuration : 429 bytes
!
interface Tunnel100
bandwidth 500 <<<<<<<<<<<<<<<<<<<< Reduced to 500Kbps from 1000Kbps
ip address 10.0.100.84 255.255.255.0
no ip redirects
ip mtu 1400
ip nhrp authentication cisco
ip nhrp map multicast dynamic
ip nhrp network-id 1
ip nhrp holdtime 600
ip tcp adjust-mss 1360
load-interval 30
delay 5100
tunnel source Ethernet0/1
tunnel mode gre multipoint
tunnel key 100
tunnel vrf INET1
tunnel protection ipsec profile DMVPN-PROFILE1
domain one path INET1
end
R3#show domain one master exits
BR address: 10.5.5.5 | Name: Tunnel200 | type: external | Path: INET2 |
Egress capacity: 1000 Kbps | Egress BW: 147 Kbps | Ideal:177 Kbps | under: 4 Kbps | Egress Utilization: 14 %
DSCP: default[0]-Number of Traffic Classes[14]
BR address: 10.4.4.4 | Name: Tunnel100 | type: external | Path: INET1 |
Egress capacity: 500 Kbps | Egress BW: 199 Kbps | Ideal:177 Kbps | over: 4 Kbps | Egress Utilization: 39 %
DSCP: default[0]-Number of Traffic Classes[6]
DSCP: af31[26]-Number of Traffic Classes[19] <<<<<<<<<<<<<
--------------------------------------------------------------------------------
R3#show domain one master exits
BR address: 10.5.5.5 | Name: Tunnel200 | type: external | Path: INET2 |
Egress capacity: 1000 Kbps | Egress BW: 147 Kbps | Ideal:230 Kbps | under: 3 Kbps | Egress Utilization: 22 %
DSCP: default[0]-Number of Traffic Classes[14]
DSCP: af31[26]-Number of Traffic Classes[19] <<<<<<<<<<<<
BR address: 10.4.4.4 | Name: Tunnel100 | type: external | Path: INET1 |
Egress capacity: 500 Kbps | Egress BW: 199 Kbps | Ideal:115 Kbps | over: 4 Kbps | Egress Utilization: 23 %
DSCP: default[0]-Number of Traffic Classes[6]
The above outputs contains two sets of "show domain one master exits". The first set of output shows that the bandwidth has been changed to 500Kbps and the load balancing has not kicked in yet since the af31 class traffic still flows through R4. The second set of output which was taken moments later shows the af31 class traffic shifted and flows through R5 which confirms that load balancing has been achieved.