This document was written in order to share some implementation experience and configuration examples with Cisco's new Control Plane Policing (CoPP) feature. Since this is relatively new stuff and based on what little feedback I'd gotten when I asked about it on the cisco-nsp mailing list in December 2004 and in other forums, I suspect it hasn't yet gotten much if any deployment in production networks. So in in the hope that others will find it useful as I might have a few months ago, this report was written up detailing some example configurations for CoPP.
CoPP was recently added to the select IOS train for specific hardware platforms. If you are familiar with Juniper routers, you may know that they allow you to filter packets that are destined to the box (via any interface) through the use of firewall filters on the loopback interface. CoPP is really just a way of doing the same kind of thing on Cisco routers. In my opinion the Cisco implementation is not quite as elegant as Juniper's, but we're not here to complain, rather this comment is just to give you some perspective if you are already familiar with the other vendor.
One caveat, I'm not recommending people go deploy CoPP. It may be worth considering, but it is sufficiently new, complicated and limited that it may be unnecessary and risky in your environment. If you are considering using CoPP, hopefully these notes will help you make an informed decision not only on whether you should proceed, but also on how to proceed.
This report is based on experience with a 6509/720b running native IOS 12.2(18)SXD2.
Specific configuration examples will be documented in-line using HTML's preformatted (<pre>) tags. Each configuration section will contain a brief description. In-line comments are included where they are most appropriate. Options that are site specific will be noted in brackets and italicized.
The best place to submit questions or feedback regarding any general configuration information in this text is to the cisco-nsp mailing list. For document editing or style comments, feel free to contact the document editor directly.
Cisco's Deploying Control Plane Policing White Paper is a relatively good way to get started. I used that as my guide. However, as I recall you almost assuredly cannot use the examples it provides in your actual configs. For instance, Rodney Dunn pointed out to me on the mailing list that the default class is not currently supported. CoPP is likely to be enhanced quickly over the next few IOS releases, check your software train release notes for any changes that may render this document obsolete or incorrect.
The simplest use of CoPP would be to simply discard or rate limit away specific traffic you do not want to see. We will use a more complex set of rules in order to explore CoPP as described by Cisco more fully. Here we will use five unique classes of traffic that the router may receive. They are named (our names, you can name them anything you want of course), in descending order of importance cp-critical-in, cp-important-in, cp-normal-in, cp-default-in and cp-undesirable-in. You use Cisco's class-map notation to define these classes and in each you can apply a standard or extended IP access list, which associates packets with the class. For all classes except the default, the access list is named the same as the class in our example. We will define the access lists later.
class-map match-all cp-critical-in description Control plane critcal traffic match access-group name cp-critical-in class-map match-all cp-important-in description Control plane important traffic match access-group name cp-important-in class-map match-all cp-normal-in description Control plane normal traffic match access-group name cp-normal-in class-map match-any cp-undesirable-in description Control plane undesirable traffic match access-group name cp-undesirable-in class-map match-all cp-default-in description Control plane default traffic match access-group 2
Then we associate any needed policies with these class-maps. In this case we use rate limiters to restrict how much traffic the CPU will see for each class of traffic. The cp-critical-in class does not have a rate limit defined, which means there is none. Also note there is an implied default-class, which is used for things like ARP, which cannot be changed and thus cannot be rate limited using CoPP. The values for these rate limits are site specific, but these have been found to be pretty good generally. However, if you have a lot of Microsoft Windows hosts, or other types of hosts that generate a lot of IP broadcast traffic, the default rate may easily fill and you may wish to raise that limit higher. Please note that the undesirable class has both a drop action for conforming and non-conforming traffic. This is effectively a complete deny filter for that category of traffic.
policy-map control-plane-in class cp-critical-in class cp-important-in police 128000 24000 48000 conform-action transmit exceed-action drop class cp-normal-in police 32000 6000 12000 conform-action transmit exceed-action drop class cp-undesirable-in police 32000 1000 1000 conform-action drop exceed-action drop class cp-default-in police 32000 6000 12000 conform-action transmit exceed-action drop
An ACL permit statement causes the associated policy to be applied to the packet (it will be policed), whereas a deny statement will ignore the packet for the associated policy (but it should be evaluated elsewhere). These are example access lists and are by no means meant to be ideal for the packet types listed or for general networking environments. Simpler, relaxed rules may make things more manageable while sacrificing some potential for unwanted packets hitting the CPU. More complex rules may reduce the noise seen by the CPU, but at the expense of making the filters more cumbersome to manage, especially if router configurations change frequently. Hopefully these rules show both some complex and simple examples that help you figure out how to do your own CoPP implementation.
The following access list is used to categorize packets that are critical to the operation of the router. These packets will not be policed at all as defined by the class and policy above. The OSPF rules are specific to the neighboring IP address adjacencies. As are the PIM addresses. However, if this router is a PIM RP, you may find yourself creating a large list of downstream PIM routers that send you register packets. You may wish to simplify that rule if so, otherwise you'll be creating a lot of specific rules if you have a lot of PIM speaking routers. IGMP, a multicast control protocol can be seen on any usable multicast group address so here we allow the entire group address space. If the router is a DHCP relay, you allow broadcast packets from clients, but also responses back from the DHCP server to the router's BOOTPS (67) port. Finally, BGP sessions are protected regardless of which side is doing the active open.
ip access-list extended cp-critical-in remark Control plane critical traffic - inbound remark OSPF permit ospf host [OSPF neighbor A addr] any permit ospf host [OSPF neighbor B addr] any remark PIM permit pim host [PIM neighbor A addr] any permit pim host [PIM neighbor B addr] any permit pim host [PIM RP A addr] any permit pim host [PIM RP B addr] any remark IGMP permit igmp any 18.104.22.168 22.214.171.124 remark DHCP permit udp host 0.0.0.0 host 255.255.255.255 eq bootps permit udp host [DHCP server addr] eq bootps any eq bootps remark BGP permit tcp host [BGP neighbor addr] eq bgp host [local BGP addr] permit tcp host [BGP neighbor addr] host [local BGP addr] eq bgp deny ip any any
The following access list is used to categorize packets that are important to the operation of the router, but if for some reason they weren't being processed, the network could still be operational. The rules here are mainly associated with packets that help manage the router.
ip access-list extended cp-important-in remark Control plane important traffic - inbound remark TACACS permit udp host [TACACS server addr] eq tacacs any permit tcp host [TACACS server addr]eq tacacs any remark SSH/TELNET permit tcp [remote vty mgmt subnet] 0.0.0.255 any range 22 telnet remark SNMP permit udp host [SNMP manager addr] any eq snmp remark NTP permit udp host [NTP server addr] eq ntp any deny ip any any
The following access list is used to categorize packets that are normally seen by the router, but aren't necessary to the operation of the network. This might include ICMP and traceroute packets, which you normally allow some subset of. Note, in this example, we do not try to rate limit traceroute packets, because anything could be a traceroute packet (not just typical ICMP or UDP traceroute packets).
ip access-list extended cp-normal-in remark Control plane normal traffic - inbound remark ICMP permit icmp any any echo permit icmp any any echo-reply permit icmp any any parameter-problem permit icmp any any time-exceeded permit icmp any any unreachable deny ip any any
The following access list is used to categorize packets that should never be processed by the router, because they are just a waste of CPU resources. Between various types of traceroutes and router initiated router initiated connections, you have to be careful about what you block. In this example, we just block ports that I've seen open on a Cisco router, that generally shouldn't need to be. You may wish to block other ports that you know the router won't listen to, but it's not clear there is much value in bothering to drop them rather than just sending an unreachable or a TCP reset. Note, the policy here is that packets matching this access list will never be processed by the CPU. If you are unsure, you might just let the packets fall through to the default class, which throttles all remaining traffic.
ip access-list extended cp-undesirable-in remark Control plane undesirable traffic - inbound remark NTP permit udp any any eq ntp remark SNMPTRAP permit udp any any eq snmptrap deny ip any any
This is a catchall access list for packets that do not match any permit rule in other CoPP access list statements. Packets matching this rate limiter all fight for remaining CPU access. It is important to match on packets in the other classes for traffic that you generally want the router to process, because this class can become a bit of a no man's land, with no expectation of service under adverse conditions.
access-list 2 remark utility ACL to allow everything access-list 2 permit any
mls qos control-plane service-policy input control-plane-in
Unfortunately, troubleshooting CoPP configurations can be pretty difficult. The primary reason for this is the lack of logging support on access list rules used with CoPP. In my experience, I have so far misclassified two types of DHCP packets that a router should see, by having them land in the default class. The symptoms of which, results in clients have sporadic problems in getting DHCP addresses due to the capacity starvation they got in the default rate limiter. One command that can give you a general idea of what is happening is the show policy-map control-plane input command. This will just show you counters associated with average rate and packet drops for each class of traffic so the usefulness of this command is limited. Perhaps a bit more useful for getting an idea about which ACL rules are being used is with the show access-list command. This will show each ACL and after each rule in each ACL the number of packets that have matched a particular rule.
You should have out-of-band access to the router if you are not local to it, otherwise if you configure CoPP incorrectly, you can completely lock yourself out of the router. As a last resort, you can schedule a reload at command before making changes and then cancel it if you changes prove to work. In your initial testing, if you use a configuration like the one above, start with rates set high and bring them down as necessary instead of the reverse. It is useful to have a default deny as the last statement in each class except for the default class. In addition, the default class should have a high enough rate so that if you remove or change any of the other classes, packets should fall through and still get to the router under normal conditions. This helps prevent lock-out.
Rodney Dunn says that you should be able to match arp with the following:
class-map match-all arp match protocol arp
In addition, he and others reminded me that there are various mls rate-limit knobs that can be used to protect the processor as well. Phil Rosenthal and I carried on a private conversation about how he is using CoPP to not only address DoS attacks, but also any magic packet attack that may cause problems for the router. This includes any known or unknown attacks based on bugs that have yet to be found. Phil prefers the default deny approach in his CoPP implementations. One good example Phil demostrated, which I will likely add to the notes above at some point is to completely block IP protocols that are not supported by the router. So for instance, the cp-undesirable-in policy might also include the following:
deny icmp any any deny igmp any any deny tcp any any deny udp any any deny ospf any any deny pim any any permit ip any any
The idea above is to ignore protocols that the router may in some form wish to receive, but then all others should be explicitly denied. Phil and I also discussed the issue allowing certain protocols to be allowed to hit the router for troubleshooting purposes (e.g. in the case of traceroute packets). The stance taken in this document was to be more conservative in what you accept, but you may wish to be even more restrictive. It is a trade-off and each operator will have to decide for themselves what makes sense for their own environment. ISPs for example may make an entirely different choice than an end institution like a research university.
Phil also pointed out that some packets through the box and not necessarily to the box can attack the control plane. For example, IP datagrams with options such as record route and ICMP unreachables that are generated when a host or network is not available can be the cause or symptom of control plane performance degradation.