BGP Routing Needs To Mind Its MANRS
I’m not sure if anyone has told you this yet, but there are people on the Internet who want your money. Not only that, but some of them are prepared to do some pretty shady things to get it, particularly identifying, and then exploiting, system and network vulnerabilities. The global reliance on the Internet by the financial and manufacturing sectors, as well as government and defense, means that the importance of network security can’t be overstated. There is rich irony, then, in the fact that the core routing protocol of the Internet- Border Gateway Protocol- relies largely on the honor system. In this post we’ll discuss BGP routing and its susceptibility to hijacking, what’s being done to stop it, and why it’s still a problem.
The Internet is composed of a whole lot of autonomous systems (AS), and as the name expresses, they are independent of each other. An AS is defined as a set of IP networks, owned by a single network provider, that has a clearly defined external routing policy. An AS uses BGP routing to determine how to route traffic to its peers, and by default, BGP is designed to trust all route announcements sent by peers. This means that it is up to the route advertiser to say who they are, and to accurately state what IP addresses they own.
You might imagine that this system of trust is fertile ground for both inadvertent and malicious misdirection of Internet routes to occur, and you’d be correct. It happens all the time.
Infamous BGP Hijacks
In 2008, Pakistan Telecom received an order from the Pakistan Telecommunications Ministry requiring them to block access to YouTube. Pakistan Telecom took this directive to heart, and in response sent out a route advertisement in which they claimed to be the destination for YouTube’s full range of Internet addresses. PCCW (AS3491), a service provider based in Hong Kong, took the route advertised by Pakistan Telecom and sent it along to all of its peers, and it was a pretty significant list of peers. The result was that YouTube was offline for about two hours while YouTube furiously tried to reclaim its routes by advertising more specific IP address ranges than those advertised by Pakistan Telecom
Then there was the time a couple years before that on Christmas Eve, when an ISP in Turkey (TTNet, AS9121) announced itself as the entire Internet. Telecom Italia (AS6762) believed that most of the paths announced by TTNet were the best paths and sent their traffic there. A lot of other carriers did too, to a greater or lesser extent, with the predictable result of huge numbers of users perceiving the Internet as “down”.
Now, you might be saying to yourself: “Yes, but these examples are from a decade ago. Surely such a glaring, fundamental flaw in the very fabric of the Internet would have been solved by now!”. Well…. No. More often than not, there’s not much filtering on the peering sessions between ISPs. This means that once an ISP accepts a prefix advertised by a customer, that prefix will be widely propagated.
Money Siphoning Via BGP Routing
You don’t have to look any further back than the BGP hijack exploit that happened just last month, whichstole about $150,000 in cryptocurrency from the users of MyEtherWallet, redirecting their traffic to the site to a server in Russia.
Although the IP address range in question is owned by Amazon and used for its Route 53 service, the new route for the range was injected into the BGP infrastructure through eNet (AS10297) in Chicago, and then propagated through Hurricane Electric (AS6939). For about two hours, the bogus DNS servers at the end of the path only responded to queries for myetherwallet.com, redirecting to the Russian server.
ETH University in Zurich tracks routing attacks, and they estimate that possibly hundreds of thousandsof hijacks happen each month, and that some of those events affect up to 30,000 IP prefixes.
Teaching The Internet MANRS
Scary numbers. The good news is that there is a concerted effort led by The Internet Society to persuade providers to adhere to a standard pithily known as MANRS (Mutual Agreed Norms for Routing Security).
MANRS directs network operators to implement four steps:
- Filtering- The network operator defines a clear routing policy, implements a system that ensures correctness of their own announcements and announcements from their customers to adjacent networks, and applies due diligence when checking the correctness of its customer’s announcements, specifically that the customer legitimately holds the ASN and the address space it announces.
- Anti-Spoofing – The network operator enforces the use of valid source addresses on a customer level, implementing anti-spoofing filtering to prevent packets with incorrect source IP address from entering and leaving the network.
- Coordination- The network operator maintains globally accessible up-to-date contact information.
- Global validation- The network operator communicates to their adjacent networks which announcements are correct, and has a publicly documented routing policy, ASNs and prefixes that are intended to be advertised to external parties.
Widespread implementation of these policies by network operators would significantly reduce, if not virtually eliminate, the threat posed by the hijacking of BGP routing. The folks at MANRS say that the MyEtherWallet attack “could have been avoided if more networks across the globe implemented the MANRS for Network Operators actions.”
I put the italics in that quote, because it’s a very big “IF”. The MANRS document is approaching its 4th birthday in August, but the level of participation from operators hasn’t yet reached a level that will reliably prevent or stop the spread of malicious route advertisements. For instance, in the MyEtherWallet incident, eNet is peered with Level 3 (AS3356) and NTT (AS2914), who have implemented MANRS and did not accept the advertisement, but also peered with Hurricane Electric, who has not implemented MANRS.
Overcoming Network Operator Resistance
I think it’s safe to say that operator reluctance to implement MANRS comes down to simple cost-benefit analysis. It’s going to cost them money, and if that implementation doesn’t positively impact their top or bottom line, then there is no motivation to become MANRS-compliant (outside contribution to the greater good). The Internet Society is a cause-driven non-profit institution with a mission to make the Internet as a whole better, but network operators are typically for-profit organizations beholden to their shareholders.
Ultimately this means that we, as consumers of the services provided by network operators, will need to be the agents of change by making MANRS compliance an important part of our vendor selection criteria. A MANRS-commissioned study found that the median value of the price premium that respondents were willing to pay for MANRS compliance was 15%, which is a pretty impressive figure for a commodity service. Almost all (97%) were interested in putting MANRS participation in RFP requirements, but only 13% would make it the exclusive selection criterion. All-in-all, though, these are encouraging numbers, and it will be particularly instructive to see if they change year-over-year.
Other Issues With BGP Routing
This post has highlighted one major issue with BGP routing, but by no means is it the only one. The protocol’s rules for moving traffic between autonomous systems (called EBGP) dictate that traffic between two points will always take the same path regardless of network congestion. On top of that, network operators typically select a path based on lowest cost to them, not highest performance (those shareholders again!). You can find more information on BGP performance in our blog posts here and here. This combination of least cost routing and lack of congestion detection means that the Internet Backbone has endemic performance issues that MANRS does not, and is not intended, to fix.
Overcoming those BGP routing performance problems is what we do at Teridion. Our Internet Overlay Networkcontinually analyzes the performance of the global Internet backbone and predictively routes traffic around congestion and outages. We’ve put thousands of monitoring nodes in the world’s leading public cloud providers that collect real-time Internet performance and reachability data. Then we use an ML/AI powered routing algorithm to automatically construct a dynamically generated, per-customer Internet overlay network. If conditions change we automatically adjust the route taken, even spanning multiple public cloud providers, so that overall performance is as good as possible.
If BGP routing has you worried about the performance of your SaaS application, take a closer look at us. Ensuring a stable user experience is critical to customer retention. I encourage you to find out more about how we work by checking out this architectural whitepaper, or if you’re ready to see how we work in your environment, try us for free.