Understand BGP RPKI With XR7 Cisco8000 Whitepaper
Available Languages
Contents
Introduction
This document describes the Border Gateway Protocol (BGP) Resource Public Key Infrastructure (RPKI) feature on the Cisco IOS® XR Platform.
Background Information
Preface
This document discusses the BGP RPKI feature and how it safeguards BGP with routers against false/malicious BGP prefix updates.
Scope
This document uses Cisco 8000 with XR 7.3.1 release for demonstration. However, BGP RPKI is a platform-independent feature, the concepts discussed in this document applies to other Cisco platforms (with Cisco IOS, Cisco IOS-XE .) with appropriate equivalent CLI conversions. This document does not cover the procedure to add Route Origin Authorizations (ROAs) on regional internet registries.
Prerequisites
The reader needs knowledge of BGP protocol.
Disclaimer
Any Internet Protocol (IP) addresses used in this document are not intended to be actual addresses. Any examples, command display output, and figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses in illustrative content is unintentional and coincidental.
BGP Problems due to Bad Prefix Advertisement
BGP serves as the backbone of internet traffic. Even though it is the most important component of internet core, it lacks the capability to verify if the ingress BGP announcement originated from an authorised autonomous system or not.
This limitation of BGP makes it an easy candidate for various kind of attacks. One common attack is called ‘route hijack’. This attack can be used exploited to:
- Steal IPs to send spam results in IP gets rejected and hence denial of service.
- Spy on traffic to obtain sensitive information like passwords.
- Disruptions due to incorrect configurations by administrator.
- Prevent delivery of traffic by with up fake servers ensuesin denial of service.
Denial of Service attack (commonly known as DoS) is a malicious attempt to disrupt normal traffic to a router, switch, server and so on. There are variety of DoS attacks and few are discussed here.
Route Hijacking
Consider the scenario shown here. Autonomous System 3 (AS3) sends out legal BGP advertisement for its prefix 10.0.0.0/24. By BGP’s design, there is nothing in BGP that would prevent an attacker from advertising same prefix to the internet.
As shown, attacker in AS4 advertises the same prefix 10.0.0.0/24. BGP best path algorithm prefers a path with shorter AS_Path. AS_Path 1,5,4 wins over longer path via AS 1,5,2,3. Therefore, the traffic from the clients will now be redirected to attacker’s environment and can be black holed which ensues in denial of service to end clients.
Route hijack
Degrade System Performance
This section discusses another way in which services can be denied. If Cisco’s BGP route dampening feature is configured, it could be exploited if the attacker introduces rapid route flaps in the network causing a constant churn.
The dampening feature will impose penalties on the legitimate route and will make it unavailable for the actual traffic. In addition, this kind of unethically induced flaps will cause strain on the router’s resources like CPU, memory and so on.
Route Dampening
Sub-Prefix Hijacking
As discussed in the previous section, how an attacker can originate a prefix illegally and cause a disruption of traffic. Unfortunately, a disruption is not the only cause of concern. In such attacks, actual data can be compromised wherein an attacker can scan received data for unethical use.
Similarly, hijacking of a route could be done by illegally advertising a more precise route. BGP prefers prefixes that are a longer match and this behaviour can be wrongly exploited as shown in the image.
Sub-prefix hijack
All the attacks that are discussed stem from the fact that BGP couldn’t identify if the origin AS of these maliciously advertised prefixes was valid or not. In order to fix this, a ‘true’ and ‘trusted’ source of data is needed which a router can keep in its database. Then upon every receipt of a new advertisement, the router now becomes capable of cross verifying prefix’s AS origin information received from BGP peer with its local database information from the validator.
Thus, the router is able to distinguish the good advertisements from the bad (illegal) ones and the capability to avoid all attacks discussed previously are inherently added on the router. BGP RPKI provides the needed trusted source of information.
RPKI
RPKI makes use of a repository that contains ROAs. A ROA contains information about prefix and their associated BGP AS number. Route origin authorization is a cryptographically signed statement.
The 5 Regional Internet Registries (RIRs) are the trust anchors of the RPKI. Internet Assigned Numbers Authority (IANA) is the top of the tree that hands out IP prefixes. The RIRs are next in the hierarchy. They assign sub-prefixes to Local Internet Registries (LIRs) and large Internet Service Provider (ISP). They sign a Certificate for these prefixes. The next level allocates sub-prefixes of those and uses the certificates from above to sign their own certificates to certify their own allocations. They typically use their own publication points to host the certificates and ROAs. Each certificate lists the publication points of the child certificates it signs. Thus, RPKI forms a tree of certificates that mirrors the tree of IP address allocations. The RPKI validators owned by the relying parties poll all the publication points to find updated certificates and ROAs (and CRLs and manifests). They start at the trust anchors and follow the links to the publication points of the child certificates.
ROAs are entered in the repository through RIRs but the same can be done through other registries (national or local). This responsibility can also be delegated to ISPs with proper supervision and verification by RIRs.
At this moment, there are five ROA repositories maintained by RIPE NCE, ARIN, APNIC, LACNIC and AFRINIC.
A validator present in the network communicates with these repositories and downloads a trusted ROA database to build its cache. This is a coalesced copy of the RPKI, which is periodically fetched/refreshed directly or indirectly from the Global RPKI. Validator then feeds this information to the routers enabling them to compare the incoming BGP announcements with the RPKI table in order to make a safe decision.
RPKI infrastructure connectivity
Validator
This demonstration makes use of the RIPE validator. The validator will communicate with the router by establishing a TCP session. In this demonstration, the validator listens on its IP 192.168.122.120 and port 3323.
routinator server --rtr 192.168.122.120:3323 --refresh=900
IANA has specified port 3323 for this communication. The refresh timer defines the time interval after which local repository will be synced and updated to stay updated.
BGP RPKI Demonstration
Topology
Topology
Configure
router bgp 100
bgp router-id 10.1.1.1
rpki server 192.168.122.120
transport tcp port 3323
refresh-time 900
address-family ipv4 unicast
!
neighbor 10.0.12.2
remote-as 8100
address-family ipv4 unicast
route-policy Pass in
route-policy Pass out
!
!
neighbor 10.0.13.3
remote-as 100
address-family ipv4 unicast
!
!
// ‘Pass’ is a permit all route-policy.
BGP RPKI Session
The router establishes a TCP session with a validator (IP: 192.168.122.120, port 3323) in order to download the ROA cache to the router’s memory.
RP/0/RP0/CPU0:Cisco8000#show bgp rpki server 192.168.122.120
Wed Jan 20 22:54:15.763 UTC
RPKI Cache-Server 192.168.122.120
Transport: TCP port 3323
Bind source: (not configured)
Connect state: ESTAB
Conn attempts: 1
Total byte RX: 4428792
Total byte TX: 1400
Last reset
Timest: Jan 20 05:59:58 (16:54:17 ago)
Reason: protocol error
ROA Downloads on Router
Validator feeds the ROA information to the router. This cache is refreshed at periodic intervals in order to minimize the possibility of the router holding stale information. In this demonstration, a refreshing time of 900 seconds has been configured. As shown here, the Cisco 8000 router has downloaded 172632 IPv4 and 28350 IPv6 ROAs from the validator.
RP/0/RP0/CPU0:Cisco8000#show bgp rpki server summary
Wed Jan 20 23:01:59.432 UTC
Hostname/Address Transport State Time ROAs (IPv4/IPv6)
192.168.122.120 TCP:3323 ESTAB 17:00:21 172632/28350
RP/0/RP0/CPU0:Cisco8000#show bgp rpki table ipv4
Wed Jan 20 23:09:26.899 UTC
>>>Snipped output<<<
Network Maxlen Origin-AS Server
10.0.0.0/24 24 13335 192.168.122.120
10.0.4.0/22 22 38803 192.168.122.120
10.0.4.0/24 24 38803 192.168.122.120
10.0.5.0/24 24 38803 192.168.122.120
10.0.6.0/24 24 38803 192.168.122.120
10.0.7.0/24 24 38803 192.168.122.120
10.1.1.0/24 24 13335 192.168.122.120
10.1.4.0/22 22 4134 192.168.122.120
10.1.16.0/20 20 4134 192.168.122.120
10.2.9.0/24 24 4134 192.168.122.120
10.2.10.0/24 24 4134 192.168.122.120
10.2.11.0/24 24 4134 192.168.122.120
10.2.12.0/22 22 4134 192.168.122.120
10.3.0.0/16 16 4134 192.168.122.120
10.6.0.0/22 24 9583 192.168.122.120
Verify
This section demonstrates how BGP RPKI in action and how it prevents the router from wrong/illegal advertisements.
Enabling Origin-As Validity
By default, the router fetches ROAs from the validator but does not begin using them until it is configured to do so. As a result, these prefixes are marked as ‘D’ or disabled.
RP/0/RP0/CPU0:Cisco8000#show bgp origin-as validity
Wed Jan 20 23:27:37.268 UTC
BGP router identifier 10.1.1.1, local AS number 100
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0xe0000000 RD version: 30
BGP main routing table version 30
BGP NSR Initial initsync version 2 (Reached)
BGP NSR/ISSU Sync-Group versions 0/0
BGP scan interval 60 secs
Status codes: s suppressed, d damped, h history, * valid, > best
i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
Origin-AS validation codes: V valid, I invalid, N not-found, D disabled
Network Next Hop Metric LocPrf Weight Path
D*> 203.0.113.0/24 10.0.12.2 0 0 8100 ?
D*> 203.0.113.1/24 10.0.12.2 0 0 8100 ?
D*> 192.168.122.1/32 10.0.12.2 0 0 8100 ?
In order to enable the router for as-origin validity check, activate this command for the concerned address family.
router bgp 100
address-family ipv4 unicast
bgp origin-as validation enable
!
When you activate this command, it causes the router to scan the prefixes present in its BGP table against the ROA information received from the validator and one of the three states is assigned to prefixes .
RP/0/RP0/CPU0:Cisco8000#show bgp origin-as validity
Thu Jan 21 00:04:58.136 UTC
Status codes: s suppressed, d damped, h history, * valid, > best
i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
Origin-AS validation codes: V valid, I invalid, N not-found, D disabled
Network Next Hop Metric LocPrf Weight Path
V*> 203.0.113.0/24 10.0.12.2 0 0 8100 ?
I* 203.0.113.1/24 10.0.12.2 0 0 8100 ?
N*> 192.168.122.1/32 10.0.12.2 0 0 8100 ?
In order to enable the router to use prefix validation state information while making the best path calculation, this command is needed. This is not enabled by default as it gives you the option of not using the validity information for best path calculation but still allowing you to use it in route policies which are discussed later in this document.
router bgp 100
address-family ipv4 unicast
bgp bestpath origin-as use validity
!
Prefix Validity States
There are three states a prefix could be found in.
RP/0/RP0/CPU0:Cisco8000#show bgp origin-as validity
Thu Jan 21 00:04:58.136 UTC
Status codes: s suppressed, d damped, h history, * valid, > best
i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
Origin-AS validation codes: V valid, I invalid, N not-found, D disabled
Network Next Hop Metric LocPrf Weight Path
V*> 203.0.113.0/24 10.0.12.2 0 0 8100 ?
I* 203.0.113.1/24 10.0.12.2 0 0 8100 ?
N*> 192.168.122.1/32 10.0.12.2 0 0 8100 ?
- Invalid - Indicates the prefix meets either of these two conditions: 1. It matches one or more Route Origin Authorizations (ROAs), but there is no ROA match where the origin AS matches the origin AS on the AS-PATH. 2. It matches one or more ROAs at the minimum-length specified in the ROA, but for all ROAs where it matches the minimum length, it is longer than the specified maximum length. Origin AS does not matter for condition #2.
-
Valid - Indicates the prefix and AS pair are found in the RPKI cache table.
-
Not Found - Indicates the prefix is not among the valid or invalid prefixes.
This section discusses each prefix and its state in detail.
1. 203.0.113.0/24 - Valid
eBGP peer in AS 8100 originated this route and advertised to Cisco8000 node. Since the origin AS (8100) matches the origin AS in ROA (received from validator), this prefix is marked valid and is installed in the router’s routing table.
RP/0/RP0/CPU0:Cisco8000#show bgp rpki table | in "203.0.113.0|Max"
Thu Jan 21 00:21:26.026 UTC
Network Maxlen Origin-AS Server
203.0.113.0/24 24 8100 192.168.122.120
The route is installed in the BGP table.
RP/0/RP0/CPU0:Cisco8000#show bgp 203.0.113.0/24
Thu Jan 21 05:30:13.858 UTC
BGP routing table entry for 203.0.113.0/24
Versions:
Process bRIB/RIB SendTblVer
Speaker 31 31
Last Modified: Jan 21 00:03:33.344 for 05:26:40
Paths: (1 available, best #1)
Not advertised to any peer
Path #1: Received by speaker 0
Not advertised to any peer
8100
10.0.12.2 from 10.0.12.2 (192.168.122.105)
Origin incomplete, metric 0, localpref 100, valid, external, best, group-best
Received Path ID 0, Local Path ID 1, version 31
Origin-AS validity: valid
Since this is the best BGP prefix and also valid per RPKI, it is successfully installed in the routing table.
RP/0/RP0/CPU0:Cisco8000#show route 203.0.113.0/24
Thu Jan 21 00:29:43.667 UTC
Routing entry for 203.0.113.0/24
Known via "bgp 100", distance 20, metric 0
Tag 8100, type external
Installed Jan 21 00:03:33.731 for 00:26:10
Routing Descriptor Blocks
10.0.12.2, from 10.0.12.2, BGP external
Route metric is 0
No advertising protos.
RP/0/RP0/CPU0:Cisco8000#show bgp origin-as validity invalid
Thu Jan 21 00:34:38.171 UTC
BGP router identifier 10.1.1.1, local AS number 100
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0xe0000000 RD version: 33
BGP main routing table version 33
BGP NSR Initial initsync version 2 (Reached)
BGP NSR/ISSU Sync-Group versions 0/0
BGP scan interval 60 secs
Status codes: s suppressed, d damped, h history, * valid, > best
i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
* 203.0.113.1/24 10.0.12.2 0 0 8100 ?
However, the ROA received from the validator shows that this prefix belongs to AS 10021.
RP/0/RP0/CPU0:Cisco8000#show bgp rpki table 203.0.113.1/24 max 24
Thu Jan 21 00:37:05.615 UTC
RPKI ROA entry for 203.0.113.1/24-24
Origin-AS: 10021 from 192.168.122.120
Version: 124211
Since AS origin information in the received BGP announcement (AS 8100) did not match the actual AS origin received in ROA (AS 10021), the prefix is marked Invalid and is not installed in the routing table.
RP/0/RP0/CPU0:Cisco8000#show bgp 203.0.113.1/24
Thu Jan 21 05:37:26.714 UTC
BGP routing table entry for 203.0.113.1/24
Versions:
Process bRIB/RIB SendTblVer
Speaker 32 32
Last Modified: Jan 21 00:03:33.344 for 05:33:53
Paths: (1 available, no best path)
Not advertised to any peer
Path #1: Received by speaker 0
Not advertised to any peer
8100
10.0.12.2 from 10.0.12.2 (192.168.122.105)
Origin incomplete, metric 0, localpref 100, valid, external
Received Path ID 0, Local Path ID 0, version 0
Origin-AS validity: invalid
3. 192.168.122.1/32 Not Found
This is a private prefix and is not present in the ROA cache. BGP declared this prefix as ‘Not found’.
RP/0/RP0/CPU0:Cisco8000#show bgp 192.168.122.1/32
Thu Jan 21 05:44:39.861 UTC
BGP routing table entry for 192.168.122.1/32
Versions:
Process bRIB/RIB SendTblVer
Speaker 33 33
Last Modified: Jan 21 00:03:33.344 for 05:41:06
Paths: (1 available, best #1)
Not advertised to any peer
Path #1: Received by speaker 0
Not advertised to any peer
8100
10.0.12.2 from 10.0.12.2 (192.168.122.105)
Origin incomplete, metric 0, localpref 100, valid, external, best, group-best
Received Path ID 0, Local Path ID 1, version 33
Origin-AS validity: not-found
Since RPKI is still gets adopted, 'not-found' prefixes are installed in the routing table. Otherwise will cause BGP to ignore these legitimate prefixes that are not registered in the RPKI database.
Allow Invalid Prefix
Although it is not recommended, the software does provide a knob to allow invalid prefixes to participate in the best path calculation algorithm.
router bgp 100
address-family ipv4 unicast
bgp bestpath origin-as allow invalid
!
With this configuration, the router does consider invalid prefixes for best path calculation while This marked as ‘invalid’. This output shows ‘203.0.113.1/24’ marked as the best path.
RP/0/RP0/CPU0:Cisco8000#show bgp
Thu Jan 21 06:21:34.294 UTC
BGP router identifier 10.1.1.1, local AS number 100
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0xe0000000 RD version: 34
BGP main routing table version 34
BGP NSR Initial initsync version 2 (Reached)
BGP NSR/ISSU Sync-Group versions 0/0
BGP scan interval 60 secs
Status codes: s suppressed, d damped, h history, * valid, > best
i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*> 203.0.113.0/24 10.0.12.2 0 0 8100 ?
*> 203.0.113.1/24 10.0.12.2 0 0 8100 ?
*> 192.168.122.1/32 10.0.12.2 0 0 8100 ?
As shown in this output, the prefix is marked as best despite kept invalid.
RP/0/RP0/CPU0:Cisco8000#show bgp 203.0.113.1/24
Thu Jan 21 06:23:26.994 UTC
BGP routing table entry for 203.0.113.1/24
Versions:
Process bRIB/RIB SendTblVer
Speaker 34 34
Last Modified: Jan 21 06:05:31.344 for 00:17:55
Paths: (1 available, best #1)
Not advertised to any peer
Path #1: Received by speaker 0
Not advertised to any peer
8100
10.0.12.2 from 10.0.12.2 (192.168.122.105)
Origin incomplete, metric 0, localpref 100, valid, external, best, group-best
Received Path ID 0, Local Path ID 1, version 34
Origin-AS validity: invalid
It is to be noted that a router still treats invalid prefix as the last option and always prefers a valid prefix over an invalid prefix if it is available.
Manual ROA Configuration on Router
If for some reason, a ROA for a certain prefix is not yet created, received or is delayed, a manual ROA could be configured on the router. For example, the prefix ‘192.168.122.1/32’ is marked as ‘Not Found’ as shown here.
RP/0/RP0/CPU0:Cisco8000#show bgp origin-as validity
Thu Jan 21 06:36:31.041 UTC
BGP router identifier 10.1.1.1, local AS number 100
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0xe0000000 RD version: 34
BGP main routing table version 34
BGP NSR Initial initsync version 2 (Reached)
BGP NSR/ISSU Sync-Group versions 0/0
BGP scan interval 60 secs
Status codes: s suppressed, d damped, h history, * valid, > best
i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
Origin-AS validation codes: V valid, I invalid, N not-found, D disabled
Network Next Hop Metric LocPrf Weight Path
V*> 203.0.113.0/24 10.0.12.2 0 0 8100 ?
I*> 203.0.113.1/24 10.0.12.2 0 0 8100 ?
N*> 192.168.122.1/32 10.0.12.2 0 0 8100 ?
A manual ROA could be configured as shown here. This command associates’ prefix ‘192.168.122.1/32’ with AS 8100.
router bgp 100
rpki route 192.168.122.1/32 max 32 origin 8100
With this configuration, the state of the prefix changes from ‘N’ to ‘V’.
RP/0/RP0/CPU0:Cisco8000#show bgp origin-as validity
Thu Jan 21 06:36:34.151 UTC
BGP router identifier 10.1.1.1, local AS number 100
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0xe0000000 RD version: 35
BGP main routing table version 35
BGP NSR Initial initsync version 2 (Reached)
Status codes: s suppressed, d damped, h history, * valid, > best
i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
Origin-AS validation codes: V valid, I invalid, N not-found, D disabled
Network Next Hop Metric LocPrf Weight Path
V*> 203.0.113.0/24 10.0.12.2 0 0 8100 ?
I*> 203.0.113.1/24 10.0.12.2 0 0 8100 ?
V*> 192.168.122.1/32 10.0.12.2 0 0 8100 ?
Route-Policy and Prefix Validation State
Prefix state result can be used to create route policies. These states can be used in a match statement and administrator desired actions can be taken. This example matches all the prefixes with an invalid state and sets a weight value of 12345 for them.
route-policy Invalid
if validation-state is invalid then
set weight 12345
endif
end-policy
!
router bgp 100
remote-as 8100
address-family ipv4 unicast
route-policy Invalid in
!
!
!
This output shows an invalid prefix applied weight of 12345.
RP/0/RP0/CPU0:Cisco8000#show bgp 203.0.113.1/24
Thu Jan 21 06:57:33.816 UTC
BGP routing table entry for 203.0.113.1/24
Versions:
Process bRIB/RIB SendTblVer
Speaker 38 38
Last Modified: Jan 21 06:54:04.344 for 00:03:29
Paths: (1 available, best #1)
Not advertised to any peer
Path #1: Received by speaker 0
Not advertised to any peer
8100
10.0.12.2 from 10.0.12.2 (192.168.122.105)
Origin incomplete, metric 0, localpref 100, weight 12345, valid, external, best, group-best
Received Path ID 0, Local Path ID 1, version 38
Origin-AS validity: invalid
Share Prefix Validation Information through Extended Community
As BGP router can also share the prefix-validation state with other routers (with no local cache from validator) via BGP extended community. This saves the overhead of each and every router in the network with a session with the validator and downloading all the ROAs.
This is made possible by the BGP extended community.
This command enables the router to share ‘prefix-validation’ information with iBGP peers.
router bgp 100
address-family ipv4 unicast
bgp origin-as validation signal ibgp
Once Cisco 8000 router is configured as shown, BGP updates to peers enclose prefix validation information. In this case, the neighbor iBGP router is an IOS-XE router.
csr2#show ip bgp 203.0.113.1/24
BGP routing table entry for 203.0.113.1/24, version 14
Paths: (1 available, best #1, table default)
Not advertised to any peer
Refresh Epoch 1
8100
10.0.12.2 from 10.0.13.1 (10.1.1.1)
Origin IGP, metric 0, localpref 100, valid, internal, best
Extended Community: 0x4300:0:2
rx pathid: 0, tx pathid: 0x0
Updated on Jan 21 2021 18:16:56 UTC
This extended community mapping can be understood with the use of 0x4300 0x0000 (4 bytes indicating state).
The four bytes indicating state are treated as a 32-bit unsigned integer having one of the values:
- 0 - Valid
- 1 - Not Found
- 2 - Invalid
Prefix 203.0.113.1/24’s community is 0x4300:0:2 which maps to the ‘Invalid’ prefix. This way, the csr2 router despite no local cache of its own is still able to make decisions based on prefix-validation state.
Prefix validation state now can be used to match in a route-map or in BGP best path algorithm.
Recommendations for BGP RPKI Implementation
Good Practices for ROA Creation
These are some recommendations based on unreachable networks observed at RPKI- Observatory. The RPKI Observatory analyzes multiple aspects of the deployed RPKI landscape.
-
If a ROA is created for any prefix, then it is recommended to announce that prefix in BGP. In absence of it, someone else can announce it by simply pretending to be ASN contained in that ROA and use the prefix.
-
If ROA is created with a maxlen greater than the prefix length, then it is equivalent to creating ROAs for all possible prefixes under the original prefix up to the maxlen. It is strongly recommended to announce all those prefixes in BGP.
-
If a ROA is created for a prefix and the prefix owner announces a sub-prefix of the original prefix, then the ROA will invalidate that sub-prefix. A ROA for the sub-prefix as well or the maxlen of the original ROA must be extended to cover the sub-prefix.
-
If an organization owns a prefix, but plan not to announce it in BGP, then a ROA for the prefix for AS0 must be created. This will invalidate any announcement of prefix because AS0 cannot appear in any AS path.
-
If there are multiple ASNs originating the same prefix, then ROAs for that prefix must be created for each of the ASNs. Consequently, if a router has multiple ROAs for the same prefix, a BGP advertisement that matches any of them will be valid. Multiple ROAs for the same prefix do not conflict with each other.
-
If ‘A’ is originating a prefix for its customer ‘B’ and create a ROA for that prefix on behalf of ‘B’, then ‘A’ must prepend ‘B’s’ ASN to the announcement or have the ‘B’ originate the prefix itself.
Performance Impact of RPKI on XR BGP Routers
Effect of ROA Update on CPU with Route-Policy
When ROAs are updated and if the router has a local ingress route-policy for a neighbor that contains a "validation-state is", then it becomes important to re-validate the status of prefixes based on new updated ROAs. This is achieved by the router sending a BGP REFRESH request to its peer.
When BGP neighbors receive this message as shown, neighbors send their prefixes again and the inbound route-policy can revalidate the incoming prefixes .
Jan 22 18:28:41.360: BGP: 10.0.12.1 rcv message type 5, length (excl. header) 4
Jan 22 18:28:41.360: BGP: 10.0.12.1 rcvd REFRESH_REQ for afi/safi: 1/1, refresh code is 0
The problem amplifies when a lot of neighbors refresh at the same time whenever ROAs are updated. If neighbor inbound route-policy are complex and require a lot of processing, then high CPU results for a few minutes after a ROA update. These REFRESH messages do not occur if the neighbor inbound route-policy does not contain a "validation-state is" command.
If “soft-reconfiguration inbound always” is configured for a neighbor, then BGP REFRESH messages will not be sent, but the same route policies will still be executed at the same rate and the same CPU usage can be expected.
It is recommended to prefer ‘bgp bestpath origin-as use validity’ approach over configuring a route- policy for the reasons explained in 6.2.2 below.
Minimize CPU Impact Caused by ROA Update
The best way to avoid the problem explained here is to use bestpath origin-as use validity without validation-state is in the policy.
router bgp 100
address-family ipv4 unicast
bgp bestpath origin-as use validity
!
This command keeps a received invalid route on router but prevents it from becoming a best-path. It will not be installed or further advertised. It is as good as dropping it. If with the next ROA update it becomes valid, no REFRESH is required, and it will automatically become eligible for the best path with no policy execution necessary.
If the user prefers to allow ‘invalid’ prefixes and not use them, then in addition to bestpath origin-as use validity, use the configuration best path origin-as allow invalid.
In this case, when a ROA changes, the best path is automatically updated without requiring a REFRESH message. In order to de-prefer, a route means that during the BGP route selection the RPKI invalid path is considered less preferable than any other path to the same destination. It is similar to assigning it weight or local preference less than 0.
The number of RPKI invalids is relatively small and kept in the table does not result in a significant impact on resources.
IBGP routes are not validated by the router against the ROA database. IBGP routes gain an RPKI validity from the RPKI extended community. If the IBGP route is received without this extended community, then its validation-state is set to not-found.
BGP RPKI Memory Footprint
Each ROA consumes memory for the index and the data. If two ROAs are for the same IP prefix, but have different max_len or are received from different RPKI servers, then they share the same index but have separate data. Memory requirements can vary because memory overhead is not constant. An overbudget of 10% is recommended. 64-bit platforms require more memory for each memory object than 32-bit platforms. IOS-XR memory usage in bytes for an index object and a data object is in the table. Some mostly constant overhead is included in the numbers.
|
32-bit platform (bytes) |
64-bit platform(bytes) |
|
|
IPv4 index |
74 |
111 |
|
IPv6 index |
86 | 125 |
|
data |
34 | 53 |
This section takes two scenarios to explain how ROAs consume memory.
Consider a router using 3 RPKI servers, each providing 200,000 IPv4 ROAs and 20,000 IPv6 ROAs on a 64-bit route-processor will require this memory:
20000 * (125 + 3*53) + 200000 * (111 + 3*53) bytes = 59.68 million bytes
While calculating the memory, ROA for the same prefix from three different validators shared the same index value.
Scenario 2. Single RPKI Servers Configured on Router
BGP process memory without ROAs:
RP/0/RP0/CPU0:Cisco8000#show processes memory detail location 0/RP0/CPU0 | in $
Fri Jan 22 17:19:57.945 UTC
JID Text Data Stack Dynamic Dyn-Limit Shm-Tot Phy-Tot Process
1069 2M 71M 132K 25M 7447M 50M 74M bgp
RP/0/RP0/CPU0:Cisco8000#show bgp rpki server summary
Fri Jan 22 17:12:09.073 UTC
Hostname/Address Transport State Time ROAs (IPv4/IPv6)
192.168.122.120 TCP:3323 NONE 00:00:25 N/A
BGP process is seen consuming 25 MB memory without any ROAs.
BGP process memory with ROA:
RP/0/RP0/CPU0:Cisco8000#show bgp rpki server summary
Fri Jan 22 17:23:46.769 UTC
Hostname/Address Transport State Time ROAs (IPv4/IPv6)
192.168.122.120 TCP:3323 ESTAB 00:02:42 172796/28411
RP/0/RP0/CPU0:Cisco8000#show processes memory detail location 0/RP0/CPU0 | in $
Fri Jan 22 17:24:14.659 UTC
JID Text Data Stack Dynamic Dyn-Limit Shm-Tot Phy-Tot Process
1069 2M 99M 132K 53M 7447M 50M 102M bgp
BGP process is seen consuming 25 MB memory without any ROAs.
BGP process memory with ROA:
RP/0/RP0/CPU0:Cisco8000#show bgp rpki server summary
Fri Jan 22 17:23:46.769 UTC
Hostname/Address Transport State Time ROAs (IPv4/IPv6)
192.168.122.120 TCP:3323 ESTAB 00:02:42 172796/28411
RP/0/RP0/CPU0:Cisco8000#show processes memory detail location 0/RP0/CPU0 | in $
Fri Jan 22 17:24:14.659 UTC
JID Text Data Stack Dynamic Dyn-Limit Shm-Tot Phy-Tot Process
1069 2M 99M 132K 53M 7447M 50M 102M bgp
Cisco 8000 router runs 64-bit OS. It received 172796 IPv4 ROA and 28411 ROA.
Memory (Bytes) = 172,796 x [111 (index) + 53 (data)] + 28411 x [125 (index) + 53 (data)].
These calculations give ~27 MB which is approximately the increment noticed on the router’s memory above.
Revision History
| Revision | Publish Date | Comments |
|---|---|---|
2.0 |
18-Oct-2022 |
Removed all PII issues and concerns. |
1.0 |
07-Apr-2021 |
Initial Release |
Feedback