Categories
About Internet Society Building Trust Securing Border Gateway Protocol (BGP) Security Technology

Claudio Jeker Honored by Internet Security Research Group with Radiant Award

This week another Radiant Award has been awarded by the Internet Security Research Group, the folks behind Let’s Encrypt. The award puts the limelight on the heroes who make the Internet more secure and trustworthy each day.

The newest Radiant Award winner is Claudio Jeker, who receives the prize for his work of a BGP4 implementation on OpenBSD. This makes me horrendously enthusiastic. Why?

OpenBSD is a open-software based operating system that is focused on being secure and feature complete. It comes with a set of tools that make it ideally suited to be deployed, for instance, as a secure route server in an Internet Exchange Point (IXP). A route server is a service that an IXP can host in order to make the participating network service providers lives a little easier. They do not have to get the routing information from each other, but can simply talk to this piece of centralized infrastructure. OpenBSD allows this type of infrastructure to be build from commodity components in a scalable and secure way.

With a route server in place, an IXP can take additional measures to secure the Internet, namely by taking the MANRS actions.

Ultimately this would not be possible if OpenBSD did not have a rock-solid implementation of the Internet routing protocol (BGP4) – and that is exactly what Claudio developed. And to put a cherry on top, his software fully supports authenticated filtering of routes using a protocol called RPKI. RPKI is yet another critical piece of infrastructure needed to secure the Internet routing system and a way to implement one of the MANRS actions.

Claudio’s work will prove to be an important piece towards a better Internet security.

Want to know more about Let’s Encrypt? Read a comprehensive overview of the initiative – from inspiration to implementation, organization, and execution.

Categories
IETF Improving Technical Security Mutually Agreed Norms for Routing Security (MANRS) Securing Border Gateway Protocol (BGP)

Internet Resilience Discussions at IETF 104

Let’s look at what’s happening in the Internet Engineering Task Force (IETF) and the upcoming IETF 104 meeting in the area of Internet infrastructure resilience. As usual, my focus here is primarily on the routing and forwarding planes, and specifically routing security and unwanted traffic of Distributed Denial of Service Attacks (DDoS) attacks. There’s interesting and important work underway at the IETF that can help addressing problems in both areas.

This time there are a lot of new ideas, especially of an operational nature, that people bring to the IETF in the form of Internet Drafts that aim to improve the security and resilience of the Internet infrastructure. So I’d like to introduce some of them to you, but keep in mind that an Internet Draft (I-D) does not necessarily indicate IETF endorsement. It also does not constitute a standard and may even not result in any work at the IETF.

So let’s look at what’s happening in BGP land.

Can BGP Communities be harmful? 

In the recent paper “BGP Communities: Even more Worms in the Routing Can“, the authors demonstrated that Border Gateway Protocol (BGP) communities can be exploited by remote parties to influence routing in unintended ways. Due in part to their ill-defined semantics, BGP communities are often propagated far further than a single routing hop, even though their intended scope is typically limited to nearby ASes. As a consequence, remote adversaries can use BGP communities to trigger remote blackholing, steer traffic, and manipulate routes even without prefix hijacking.

The problem of ill-defined semantics is aggravated by the fact that BGP communities, and especially well-known communities, are manipulated inconsistently by current router implementations. There are differences in the outcome of the “set” directive in several popular BGP implementations. For example, in Juniper Network’s Junos OS, “community set” removes all received communities, well-known or otherwise, whilst in Cisco Systems’ IOS XR “set community” removes all received communities except a few.

An I-D “Well-Known Community Policy Behavior” describes the current behavioural differences in order to “assist operators in generating consistent community-manipulation policies in a multi-vendor environment, and to prevent the introduction of additional divergence in implementations.”

The document also urges network operators never to rely on any implicit understanding of a neighbor ASN’s BGP community handling.  For instance, “before announcing prefixes with NO_EXPORT or any other community to a neighbor ASN, the operator should confirm with that neighbor how the community will be treated.”

BGP Large Communities in the IXP environment

Some networks peer at multiple IXPs in order to improve redundancy and geographical optimization.  It is also common that, in the case of using a Route Server (RS) to implement multilateral peering relationships, Large Communities are used to instruct the RS on how to handle an announcement (e.g. not to advertise to a particular ASN), or to send additional information to the peer, e.g. the status of the validation.

The I-D “BGP Large Communities applications for IXP Route Servers” attempts to document commonly used Large Communities to facilitate consistency of use across multiple IXPs.

Building a more robust routing policy with maximum prefix limits

Has your network experienced a situation where a peer suddenly floods your border router with many more routes than expected, sometimes causing resource exhaustion and other troubles? 

The I-D “BGP Maximum Prefix Limits” describes mechanisms to reduce the negative impact of these types of misconfigurations. Instead of a general limit on the number of prefixes received from a BGP neighbour, as defined in the BGP specification, it proposes a more granular scheme with three control points to mitigate the impact:

  • Pre-Policy Inbound Maximum Prefix Limits – when the limit is checked before any policy is applied (e.g. filtering). These limits are particularly useful to help dampen the effects of full table route leaks and memory exhaustion when the implementation stores rejected routes.
  • Post-Policy Inbound Maximum Prefix Limits – checked after the import policy is applied. They are useful to help prevent FIB exhaustion and prevent accidental BGP session teardown due to prefixes not accepted by policy anyway.
  • Outbound Maximum Prefix Limits – trigger termination of a BGP session with a neighbor when the number of address prefixes to be advertised to that neighbor exceeds a locally configured upper limit. These limits are useful to help dampen the negative effects of a misconfiguration in local policy.  In many cases, it would be more desirable to tear down a BGP session rather than flooding the neighbors with misconfigured announcements.

These recommendations are distilled from a broader framework, presented by Job Snijders at the RIPE 77 meeting last year.

Leveraging RPKI for proven operational practices

A common best practice to ensure that one’s customers only announce their own networks and the networks of their customers, is to build prefix filters. 

In the case there are only direct customer relationships (i.e. the network operator’s customers are ‘stub networks’), the task is relatively easy. One needs to collect prefixes, legitimately originated by these networks, and this is most commonly done by using an IRR of choice and collecting corresponding “route” objects. But with the proliferation of RPKI, it can become a more robust alternative, providing a cryptographically verifiable ROA object that serves a similar purpose.

If you are a bigger network and some of your customers also provide transit services for smaller networks, the task is more difficult. How to determine who are the customers of your customers and so on? 

To help with this task, there is a special IRR object – “as-set”. This object is a list of ASNs or other “as-sets” that defines the customer cone of a particular AS.

However, when it comes to RPKI, there is no way for an operator to assert the routes for its customer networks, making it difficult to use the information carried by RPKI to create meaningful prefix filters without relying on RPSL “as-sets”.

The I-D “RPKI Autonomous Systems Cones: A Profile To Define Sets of Autonomous Systems Numbers To Facilitate BGP Filtering” attempts to fix that problem by introducing a new attestation object called an AS-Cone.  An AS-Cone is a digitally signed object with the goal of enabling operators to define a set of customers that can be found as “right adjacencies” or transit customer networks, facilitating the construction of prefix filters for a given ASN, thus making routing more secure.

By leveraging RPKI, AS-Cone also addresses two fundamental problems with the RPSL “as-set”. The same AS-SET name can exist in multiple IRRs, and a relying party does not necessarily know which “as-set” belongs to which ASN, and which as-set to use. 

Improving AS-PATH validation

The Border Gateway Protocol (BGP) was designed with no mechanisms to validate BGP attributes. The ability to manipulate one of them – AS_PATH – creates one of the most serious vulnerabilities of the Internet routing system. BGPsec was therefore designed to solve the problem of AS_PATH correctness.  

But according to the authors of a new I-D “Verification of AS_PATH Using the Resource Certificate Public Key Infrastructure and Autonomous System Provider Authorization” even leaving aside the complexity, its backward support for ‘insecure’ BGP allows an attacker to mount a downgrade attack to nullify all the work of AS_PATH signing. 

The authors suggest a more pragmatic approach that can help leveraging the benefits of RPKI without the need for the ubiquitous deployment of BGPsec. The idea is that any AS can declare its upstream providers and peers – the networks that can propagate its prefix announcements. The more networks that do that, the more chances to detect misconfigurations whether malicious or not.

The draft defines the semantics of Autonomous System Provider Authorization (ASPA) objects that should become part of RPKI. ASPAs are digitally signed objects that bind in a selected AFI Provider AS number to a Customer AS number (in terms of BGP announcements not business), and are signed by the holder of the Customer AS. An ASPA attests that a Customer AS holder (CAS) has authorized a particular Provider AS (PAS) to propagate the Customer’s IPv4/IPv6 announcements onward, e.g. to the Provider’s upstream providers or peers.

Mitigating DDoS attacks

DDoS attacks are a persistent and growing threat on the Internet.  As they evolve rapidly in the terms of volume and sophistication, a more efficient cooperation between the victims and parties that can help mitigate such attacks is required. The ability to quickly and precisely respond to a attack when it begins, and communicate precise information to a mitigation service provider is crucial.

Addressing this challenge is what keeps the DDoS Open Threat Signaling (DOTS) Working Group busy. The aim of DOTS is to develop a standards based approach for the real-time signaling of DDoS related telemetry and threat handling requests and data between elements concerned with DDoS attack detection, classification, traceback, and mitigation. This protocol should support requests for DDoS mitigation services and status updates across inter-organizational administrative boundaries. Specifications outlining the requirements, architecture and the use cases for DOTs are maturing, and there is a hackathon planned at IETF104 to conduct further interoperability testing of DOTS protocols.

Another interesting case getting more importance, especially with the advent of consumer IoT devices, is mitigation of outbound DDoS attacks originating in a home network. The I-D “Denial-of-Service Open Threat Signaling (DOTS) Signal Channel Call Home” proposes a solution to these cases by proposing a DOTS signal channel Call Home extension that enables the DOTS server to initiate a secure connection to the DOTS client. The DOTS client then conveys the attack traffic information to the DOTS server. 

In a typical deployment scenario, the DOTS server is enabled on a CPE, whilst a client resides in an ISP network. In this case the DOTS server in the home network initiates the Call Home during peace time, and subsequently the DOTS client in the ISP environment can initiate a mitigation request whenever the ISP detects there is an attack from a compromised device in the DOTS server’s domain. Subsequently, the DOTS server would use the DDoS attack traffic information to identify the compromised device in its domain launching the DDoS attack, notify the network administrator, and take appropriate mitigation action such as quarantining the compromised device or block its traffic to the attack target until the mitigation request is withdrawn.

The meeting in Prague is certainly going to be interesting regarding Internet infrastructure security and resilience, and will hopefully have a positive impact in this area.

Relevant Working Groups at IETF 104

SIDROPS (SIDR Operations) WG
Agenda: https://datatracker.ietf.org/meeting/104/materials/agenda-104-sidrops
Charter: https://datatracker.ietf.org/wg/sidrops/charter/
GROW (Global Routing Operations) WG
Agenda: https://datatracker.ietf.org/meeting/104/materials/agenda-104-grow
Charter: https://datatracker.ietf.org/wg/grow/charter/
IDR (Inter-Domain Routing) WG
Agenda: https://datatracker.ietf.org/meeting/104/materials/agenda-104-idr
Charter: https://datatracker.ietf.org/wg/idr/charter/
DOTS (DDoS Open Threat Signaling) WG
Agenda: https://datatracker.ietf.org/meeting/104/materials/agenda-104-dots
Charter: https://datatracker.ietf.org/wg/dots/charter/
Categories
Deploy360 Improving Technical Security Mutually Agreed Norms for Routing Security (MANRS) Securing Border Gateway Protocol (BGP)

RIRs Enhance Support for Routing Security

BGP hijacking and route leaks represent significant problems in the global Internet routing systems, along with source address spoofing. BGP hijacks are where allocated or unallocated address space is announced by entities who are not holders and are not authorized to use it.

The announcement of allocated address space often creates big news, such as when 53 route prefixes of Amazon were hijacked, but the announcement of unallocated address space (whether IPv4, IPv6 or AS numbers) which are also known as ‘bogons’ often does not generate much publicity as it does not cause immediate disruptions to service or business. With depletion of the IPv4 address space though, the announcement of bogons are on the rise with miscreants scraping the unallocated address space from all RIRs and abusing it.

Resource Public Key Infrastructure (RPKI) was therefore developed to try to solve these problems, and APNIC (the Routing Internet Registry for the Asia-Pacific region) recently announced it will honour the creation of AS0 ROA objects. They join ARIN, AfriNIC and the RIPE NCC in supporting AS0 ROA objects, with only LACNIC yet to implement this.

APNIC members can create AS0 ROAs for the prefixes they manage using the MyAPNIC platform.

So, what is the significance of AS0 ROAs? A quick overview of ROA is in order before explaining the importance of AS0 ROA. According to RFC6483:

A “route” is a unit of information that associates a set of destinations described by an IP address prefix with a set of attributes of a path to those destinations.

The Border Gateway Protocol (BGP) relies on the assumption that the Autonomous System (AS) that originates routes for a particular prefix, is authorized to do so by the holder of that prefix. A Route Origination Authorization (ROA) is used to verifiably assert that the holder of IP address space is authorized to originate routes from a given set of prefixes.

A ROA identifies a single AS that has been authorized by the address space holder to originate routes, and provides a list of one or more IP address prefixes that will be advertised. If the address space holder needs to authorize multiple ASes to advertise the same set of address prefixes, the holder issues multiple ROAs, one per AS number.

The information in the ROAs can be used by networks using BGPto perform Route Origin Validation (ROV) on incoming BGP advertisements. ROV allows BGP speakers to determine if they should accept the routes being advertised to them as real, and is based on the state of a received announcement which can be Valid, NotFound, or Invalid.

  • Valid – The announcement is covered by at least one ROA
  • NotFound – The announcement is not covered by any ROA
  • Invalid – Announcement that contradicts ROA information. It can be an AS of unauthorised origin AS, or that the announcement is more specific than is allowed by the maximum length set even if it originates from a valid AS number.

What must be remembered is that RPKI validation relies on the availability of RPKI data, and therefore RPKI caches should be located close to routers that require this data (we are not going to discuss Relying Party-RP or RTR Protocol here).

Up until September 2012, AS0 was listed in the IANA Autonomous System Number Registry as “Reserved – May be used to identifying non-routed networks”. This status was updated with RFC7607 which redefined AS0 in line with RFC6491 “Resource Public Key Infrastructure (RPKI) Objects Issued by IANA”:

AS0 ROA: A ROA containing a value of 0 in the ASID field. “Validation of Route Origination Using the Resource Certificate Public Key Infrastructure (PKI) and Route Origination Authorizations (ROAs)”

Whereas, RFC6483 defines the term “Disavowal of Routing Origination”.

“A ROA is a positive attestation that a prefix holder has authorized an AS to originate a route for this prefix into the inter-domain routing system.  It is possible for a prefix holder to construct an authorization where no valid AS has been granted any such authority to originate a route for an address prefix.  This is achieved by using a ROA where the ROA’s subject AS is one that must not be used in any routing context.  Specifically, AS0 is reserved by the IANA such that it may be used to identify non-routed networks

A ROA with a subject of AS0 (AS0 ROA) is an attestation by the holder of a prefix that the prefix described in the ROA, and any more specific prefix, should not be used in a routing context. The route validation procedure will provide a “valid” outcome if any ROA matches the address prefix and origin AS even if other valid ROAs would provide an “invalid” validation outcome if used in isolation.  Consequently, an AS0 ROA has a lower relative preference than any other ROA that has a routable AS, as its subject.  This allows a prefix holder to use an AS0 ROA to declare a default condition that any route that is equal to or more specific than the prefix to be considered “invalid”, while also allowing other concurrently issued ROAs to describe valid origination authorizations for more specific prefixes.”

This means that AS0 in a ROA can be used to mark a prefix and all its more specific prefixes as Invalid and not to be used in a routing context. By publishing a ROA that lists AS0 as the only origin, it allows a resource holder to signal that a prefix (including its more specific prefixes) should not be routed. In other words, a BGP speaker should not accept or propagate routes containing AS0.

RFC7607 codifies the BGP speaker behaviour to handle AS0.

“A BGP speaker MUST NOT originate or propagate a route with an AS number of zero in the AS_PATH, AS4_PATH, AGGREGATOR, or AS4_AGGREGATOR attributes. 

An UPDATE message that contains the AS number of zero in the AS_PATH or AGGREGATOR attribute MUST be considered as malformed and be handled by the procedures specified in RFC7606 “treat-as-withdraw”

An UPDATE message that contains the AS number of zero in the AS4_PATH or AS4_AGGREGATOR attribute MUST be considered as malformed and be handled by the procedures specified in RFC6793 “attribute discard”

If a BGP speaker receives zero as the peer AS in an OPEN message, it MUST abort the connection and send a NOTIFICATION with Error Code “OPEN Message Error” and subcode “Bad Peer AS” (see Section 6 of RFC4271).  A router MUST NOT initiate a connection claiming to be AS0.”

Returning to RFC6491, this ‘Recommends’ that IANA issue an AS0 ROA for all reserved IPv4 and IPv6 resources not intended to be routed, to all Unallocated Resources – namely Resources that have not yet been allocated for special purposes to Regional Internet Registries (RIRs) – or to other resources that are not intended to be globally routed.

This measure can greatly enhance the effectiveness of RPKI and routing security in general, but network operators should also take a look at the MANRS initiative – which is supported by the Internet Society. This specifies additional actions including filtering, anti-spoofing, coordination, as well as support for global validation mechanisms such as RPKI and currently encompasses over 200 Autonomous Systems around the world, including some of the largest ISPs.

If you’re a network operator or IXP, then please see how you can help contribute towards improving the security and resilience of the global routing system.

Further Information

Categories
Improving Technical Security Mutually Agreed Norms for Routing Security (MANRS) Securing Border Gateway Protocol (BGP)

Developing Good BGP Neighbour Relationships @ APRICOT 2019

Routing Security is featuring heavily on the APRICOT 2019 programme, which is being held on 23-28 February 2019 in Daejeon, South Korea. This helps build on the MANRS initiative being supported by the Internet Society,

On Wednesday, 27 February (09.30-13.00 UTC+9) there will be a Routing Security session that will discuss the latest problems, developments, and how routing security measures can be implemented. Speakers include Job Snijders (NTT) who’ll be discussing changes to BGP in the coming 18 months; Töma Gavrichenkov (Qrator Labs) on how BGP hijacks can be used to compromise the digital certificates used to secure online transactions; and from Anurag Bhatia (Hurricane Electric) who’ll analyse the top misused ASNs.

During the second part of the session, Tashi Puntsho (APNIC) will cover the practical issues and implications of deploying your own RPKI Certificate Authority; Tim Bruijnzeels (NLnet Labs) will discuss the use of route servers at Internet Exchange Points; whilst Ed Lewis (ICANN) will discuss the issues with using the RIR Whois databases.

Following on from this, our colleague Andrei Robachevsky will be raising awareness of the MANRS Initiative during the FIRST Technical Colloquium (16.30-18.00 UTC+9).

FIRST is the global organisation of Computer Security and Incident Teams (CSIRTs) which are often in the front line when network security incidents occur, but are also involved in implementing preventative measures and capacity building. MANRS therefore considers CSIRTs to be important partners in improving the security and resilience of the global routing system, as well as providing input and feedback on the MANRS Observatory that is being developed to provide analysis of the state of the security and resilience of the routing system.

The Asia Pacific Regional Internet Conference on Operational Technologies (APRICOT) is the largest international Internet conference in the region, drawing network engineers, operators, researchers, service providers, users and policy communities from over 50 countries to teach, present, and develop relationships. Other Asia-Pacific networking organisations also use the opportunity to meet, in order to share knowledge required to operate the Internet.

If you’re interested in attending then it’s still possible to register at https://2019.apricot.net/register/register/

Alternatively, if you’re unable to make it in person, then the sessions can be followed via webcast.

Further Information

Categories
Improving Technical Security Mutually Agreed Norms for Routing Security (MANRS) Securing Border Gateway Protocol (BGP)

Route Leak Causes Major Google Outage

Google recently faced a major outage in many parts of the world thanks to a BGP leak. This incident that was caused by a Nigerian ISP – Mainone – occurred on 12 November 2018 between 21.10 and 22.35 UTC, and was identified in tweets from the BGP monitoring service BGPMon, as well as the network monitoring provider Thousand Eyes.

Google also announced the problem through their status page:

We’ve received a report of an issue with Google Cloud Networking as of Monday, 2018-11-12 14:16 US/Pacific. We have reports of Google Cloud IP addresses being erroneously advertised by internet service providers other than Google. We will provide more information by Monday, 2018-11-12 15:00 US/Pacific.

In order to understand this issue, MainOne Inc (AS37282) is peering at IXPN (Internet Exchange Point of Nigeria) in Lagos where Google (AS151169) and China Telecom (AS4809) are also members.

Google (AS15169) advertise their prefixes (more than 500) through the IXPN Route Server, where PCH (Packet Clearing House) collects a daily snapshot of BGP announcements of IXPN. Unfortunately, 212 prefixes (aggregates of those 500+ announcements) from Google were leaked, which was recorded by BGPMon and RIPEstat.

Looking at the RIPE stats it is evident that the first announcement via MainOne Inc (AS37282) was recorded at 21:12 UTC and the issue lasted for more than an hour:

As per the tweet from BGPMon, the issues lasted for 74 minutes:

Looking at the circumstances around this incident, it’s likely this was an inadvertent leak from MainOne caused by a configuration mistake. A Google representative is quoted in ArsTechnical as saying “officials suspect the leak was accidental and not a malicious hijack”, and also added that the affected traffic was encrypted which limited the harm that could result from malicious hijackings.

Later in the same day, the MainOne Twitter account posted on the BGPMon analysis thread, accepting the mistake and assuring the world that corrective measures are now in place:

So this was a configuration mistake that was quickly rectified and didn’t cause any reported financial damage (even though service outages do cause financial and reputational damage to the service provider and its users), but it does demonstrate the problems that can be caused by accidental mistakes, and especially how an actor with bad intent could do a great deal of damage  as with the Amazon Route 53 hijack. It therefore illustrates why greater efforts need to be made towards improving the security and resilience of the Internet.

This BGP leak could have been easily avoided if proper prefix filtering had been undertaken by MainOne (AS37282) or China Telecom (AS4809). It is very difficult for the networks in the middle to block such leaks, because the prefixes are still legitimately originating from the correct AS number (in this scenario AS15169 – Google).

As mentioned in many previous blogs, Mutually Agreed Norms for Routing Security (MANRS) can be part of the solution here. It calls for four simple but concrete actions that ALL network operators should implement to reduce the most common routing threats, including filtering which prevents the propagation of incorrect routing information (the other three are anti-spoofing, address validation, and global coordination).

Network operators have a responsibility to ensure a globally robust and secure routing infrastructure, and your network’s safety depends on a routing infrastructure that weeds out accidental misconfigurations and bad actors. The more network operators who work together, the fewer incidents there will be, and the less damage they can do. It’s time to implement the MANRS actions now!

Categories
Improving Technical Security Mutually Agreed Norms for Routing Security (MANRS) Securing Border Gateway Protocol (BGP)

U.S. R&E Community Embraces Routing Security

The Internet Society participated in a Routing Security Workshop that was held during the Internet2 Technology Exchange 2018 on 15 October 2018 in Orlando, United States. The research and education networking community has been one of the key targets of the MANRS initiative that is promoting adoption of best practices to reduce threats to the global routing system, and this community workshop followed on from a previous engagement we had with Internet2 and a number of other R&E networks in the US earlier in the year.

Internet2 interconnects R&E institutes across the United States in conjunction with regional and state networks, so we see them as a key partner in raising awareness of the routing security issues, as well as encouraging the adoption of the four MANRS principles. Indeed, one of the aims of MANRS is for network operator communities to take ownership of this process by generating awareness and disseminating best practices, along with making recommendations for improvement. So this workshop was a fantastic step in this direction.

Another positive step was Internet2 formally becoming a MANRS participant shortly before the workshop, follow in the footsteps of ESnet, CAAREN, KanREN, George Washington University, Indiana University, and DePaul University. WiscNet subsequently also joined, which brings the total number of R&E networks participating in MANRS to nearly 30.

Around 50 participants attended the workshop, where the opening presentation was provided by myself (Kevin Meynell). This highlighted how the global routing system is constantly under attack, and provided some statistics on who the outages were affected, and who were the potential culprits. It also made the point that whilst more than 60,000 Autonomous Systems make up the Internet, only about 10,000 are considered part of the core, which means routing security can be greatly improved even if only a relatively small percentage of these adopt the MANRS principles.

The remainder of the workshop covered how to implement some of the routing security best practices, including the importance of Internet Routing Registry (IRR) updates, implementation of RPKI and uRPF, as well as how to implement BGP FlowSpec to implement packet filtering in order to mitigate Distributed Denial of Service (DDoS) attacks. There was also an interesting presentation on the Legal Barriers to Securing the Routing Architecture, followed by a discussion on what routing security means to Internet2 members implementing the next generation Internet.

Our colleague Ryan Polk assisted by Fabio Erdos also took the opportunity to interview the representatives of several MANRS participants attending the Internet2 Technology Exchange, to get their views on the routing issues they had encountered, how they were supporting routing security best practices, and why supported the MANRS initiative.

We would like to thank all those who agreed to be interviewed, Paul Howell, Anita Nikolich and Grover Browning who organised the workshop, and Internet2 for hosting it.

Further Information

Categories
Events IETF Internet of Things (IoT) IPv6 Securing Border Gateway Protocol (BGP)

IETF 103, Day 2: IPv6, NTP, Routing Security & IoT

This week is IETF 103 in Bangkok, Thailand, and we’re bringing you daily blog posts highlighting the topics of interest to us in the ISOC Internet Technology Team. And following on from the previous day, Tuesday also features a packed agenda.

LPWAN will be discussing whether to move to a Working Group Last Call on the Static Context Header Compression (SCHC) framework for IPv6 and UDP, that provides both header compression and fragmentation functionalities. Three other drafts describe similar schemes for SigFox,LoRaWAN and IEEE 802.15.4 type networks.


NOTE: If you are unable to attend IETF 103 in person, there are multiple ways to participate remotely.


Then at 11.20 UTC+7, IPWAVE will be focusing on updates to the specification for transmitting IPv6 Packets over IEEE 802.11 Networks in Vehicular communications, and the use cases for IP-based vehicular networks. There have also been a couple of updates to DNS Name Autoconfiguration for Internet of Things Devices and IPv6 Neighbor Discovery for Prefix and Service Discovery in Vehicular Networks, so these may also be discussed.

6MAN will be meeting at 13.50 UTC+7 and has nine drafts up for discussion. The couple of working group sponsored drafts relate to specifying a IPv6 Segment Routing Header (SRH) and how this can be used by Segment Routing capable nodes, and specifying a Router Advertisement flag to indicate to hosts that a link is IPv6-only. There are also a couple of new drafts that specify how IOAM (In-situ Operations, Administration and Maintenance) records are encapsulated in IPv6, and defining the building blocks that can be used for OAM in Segment Routing with IPv6.

The other drafts being discussed cover communicating NAT64 prefixes to clients with Router Advertisements, Updates to Requirements for IPv6 Options, Path MTU Discovery solutions, a new Path MTU Hop-by-Hop Option to record minimum Path MTU from source to destination, and IPv6 Packet Truncation procedures.

Running in parallel is SIDROPS that is discussing five drafts. RPKI Validation State Unverified proposes to introduced a new ‘Unverified’ validation state for route prefixes, whilst BGPsec Validation State Unverified proposes a similar validation states for BGPsec routes. Two other drafts introduce and define a digitally signed object into an RPKI  that provides a means of verifying that a Customer Autonomous System holder has authorised a Provider Autonomous System to be its upstream provider. That leaves a draft considering policy for dropping invalid routes – including hijacked and missing or erroneously created ROAs for route prefixes.

To conclude the day, there’s a choice of two sessions at 16.10 UTC+7.

NTP is a working group we’ve decided to cover as (amongst other things), it’s working to improve the security of the Network Time Protocol. There’s no less than 20 drafts on the agenda, although Network Time Security for the NTP specifies a mechanism for using TLS and Authenticated Encryption with Associated Data (AEAD) to provide cryptographic security for the client-server mode of NTP. Following on from this will be a review of the NTS implementations and interoperability testing.

T2TRG researches the issues of turning the IoT into reality, and will continue to discuss the State-of-the-Art and Challenges for the Internet of Things Security, the guidance for designing IoT systems using the REST architectural style, and a new data and interaction model called CoRAL (The Constrained RESTful Application Language).

For more background, please read the Rough Guide to IETF 103 from Olaf, DanSteve, and myself.

Relevant Working Groups

Categories
Artificial Intelligence Deploy360 Internet Exchange Points (IXPs) Mutually Agreed Norms for Routing Security (MANRS) Securing Border Gateway Protocol (BGP)

GLIF 2018 Held at the Home of Hamlet

The 18th Annual Global LambaGrid Workshop (GLIF 2018) was held on 18-21 September 2018 at the Kulturværftet in Helsingør (Elsinore), Denmark. Kronberg Castle, located next to the venue, was immortalised as Elsinore in the William Shakespeare play Hamlet, but there proved to be nothing rotten with the state of high-bandwidth networking as 50 participants from 19 countries came to hear how these networks are facilitating exascale computing in support of biological, medical, physics, energy production and environmental research, and to discuss the latest infrastructure developments.

This event was organised by myself with support from NORDUnet who hosted the event in conjunction with the 30th NORDUnet Conference (NDN18), and where I also took the opportunity to raise awareness of the MANRS initiative.

The keynote was provided by Steven Newhouse (EBI) who presented the ELIXIR Compute Platform which was being used for analysing life science data. In common with high-energy physics, genomics research produces a lot of data, but this is more complex and variable, requires sequencing and imqging on shorter timescales, and of course has privacy issues. The European Molecular Biology Laboratory is based across six countries and employs over 1,600 people, but also collaborates with thousands of other scientists and requires access to existing national repositories as well. High-bandwidth networks are therefore necessary to interconnect their on-site computer and storage clusters, but will increasingly be necessary to facilitate connectivity with other research and commercial cloud resources such as EGI.eu and HelixNebula.

David Martin (Argonne National Labs) continued this theme, by presenting on the US Department of Energy’s Exascale Computing Initiative. This aims to develop and operate the next generation of supercomputers at the Argonne, Lawrence Livermore, Los Alamos and Oak Ridge National Labs by 2021, along with a software stack that will present a common computing platform for supporting advanced research applications and neural networks. The Argonne Labs Computing Facility will be based around an Intel Aurora supercomputer with over 1,000 petaflops of processing, 8 PB of memory, and 10 TB/s of input/output capability that will require future network connections in the petabit-per-second range.

Joe Mambretti (Northwestern University) then discussed the Open Science Cloud (OSDC) which is an open-source cloud-based infrastructure that allows scientists to manage, share and analyse large datasets. The aim is to have 1-2 PB of storage at each participating campus, interconnected with 100 Gb/s+ links, but presented and managed as a common namespace with uniform interfaces and policies.

The rest of the day was devoted to how network automation can integrate compute and storage facilities, particularly across multiple domains. Migiel de Vos (SURFnet) presented the work being undertaken for SURFnet 7, and explained the distinction between automation and orchestration whereby the former is considered task and domain specific, whilst the latter is developing intelligent processes that consist of multiple automated tasks across multiple domains. This required the development of new information models, standardised interfaces, automated administration, and then predetermined service delivery agreements.

Gerben van Malenstein (SURFnet) then discussed LHCONE Point-to-Point Service that allowed Layer 2 circuits to be dynamically established between Data Transfer Nodes for exchanging data from the Large Hadron Collider. This was built on the AutoGOLE work which was now enabled on 21 open exchange points. Nevertheless, whilst AutoGOLE was a functional and proven multi-domain system, there was still limited uptake by network services and end-users, which was necessary to completely remove human configuration of network equipment and create a truly global research platform.

Most of the following day was devoted to technical discussions chaired by Lars Fischer (NORDUnet) and Eric Boyd (University of Michigan). These focused around some practical examples of network automation being used at the University of Michigan, a passive network measurement system with programmable querying at 100 Gb/s line rates that was being developed by the IRNC AMIS Project, as well as discussions on how to automate the generation of network topology maps.

Topology maps are useful for users to show how they can reach counterparts in other parts of the world, and where particular services are available. They are also useful as a marketing tool to show investors and stakeholders how they contribute towards creating a truly global infrastructure, and demonstrate how the NREN model is accepted around the world, and for example, the GLIF map has become a somewhat iconic piece of artwork.

Other developments were the establishment of a new exchange point called South Atlantic Crossroads (SAX) based in Fortaleza, Brazil that was expected to interconnect with new cable systems to Angola (SACS) and Portugal (EllaLink), as well as to AMPATH and SouthernLight over the existing MONET connection. There were also plans to build procure a new 100 Gb/s connection from Europe to the Asia-Pacific, from Geneva to Singapore via the Indian Ocean to supplement the existing link from Amsterdam to Tokyo via Russia.

There were further updates on the new KREOnet network which supported 100 Gb/s links between five major Korean cities and Chicago (StarLight) via KRLight, as well as multiple 10 Gb/s links to 11 other Korean cities, Hong Kong and Seattle. The KREOnet-S infrastructure further offered SDN capabilities permitting dynamic and on-demand virtual network slicing, whilst a Science DMZ provided high-performance computing facilities for KISTI’s new 25.5 petaflop supercomputer.

SURFnet is transitioning its network to SURFnet 8 and would be upgrading its core network and international links, whilst StarLight was developing a Trans-Pacific SDN testbed, as well as an SDX for the GENI initiative.

The closing plenary session focused on how high-bandwidth research connections and exchange points can be better planned and coordinated, and whether a new entity should be created to support this. The GLIF Co-Chairs Jim Ghadbane (CANARIE) and David Wilde (AARNet) outlined some ideas around this, and then hosted a discussion on how things should be progressed.

Further Information

Categories
Deploy360 Internet Exchange Points (IXPs) IPv6 Securing Border Gateway Protocol (BGP)

Training the next generation of network engineers in Kyrgyzstan

The Internet Society in conjunction with Packet Clearing House (PCH), our Kyrgyzstan Chapter (ISOC-KG) and the CAREN Project organised a BGP and Peering capacity building workshop on 3-7 September 2018 in Bishkek, Kyrgyzstan. This five-day workshop was aimed at training engineers for the existing KG-IX Internet Exchange in the capital Bishkek, but also for the prospective Ferghana Valley Internet Exchange being established in the southern city of Osh.

The workshop was led by Nishal Goburdhan who’s an Internet Analyst at PCH, a non-profit organisation that builds and support IXPs around the world. He was assisted by myself (Kevin Meynell), with the workshop being hosted by the National Academy of Sciences of the Republic of Kyrgyzstan.

The workshop was comprised of a mix of lectures and hands-on lab work to teach the skills required for interconnecting networks on the Internet, and participating in an Internet Exchange. It commenced with Internet address planning using both IPv4 and IPv6, followed by setting-up OSPF on different internal networks, then interconnecting those using BGP and applying routing policy and filtering. The workshop concluded with how to set-up an IXP and discuss current best practices for peering.

Twelve participants attended the workshop, drawn from the incumbent telco Kyrgyz Telecom, KG-IX, commercial ISPs, universities, and KRENA (the National Research and Education Network). Despite limited previous experience and some difficulties in communicating between English, Russian and Kyrgyz languages (although we had an excellent translator), the group proved very adept at picking-up what needed to be done, cooperating as a team, and completing the tasks. It was also extremely encouraging that although none of the participants had any previous IPv6 experience, they were keen to learn how to set-up and manage IPv6 networks which will be critical for the future development of the Internet in Kyrgyzstan.

KG-IX has greatly improved performance and reduced the cost of Internet access in Kyrgyzstan, but this has mostly benefitted Internet users in the capital Bishkek and northern part of Kyrgyzstan. The Ferghana Valley in the south part of the country also has a substantial population, yet has poor access to communication services and users typically pay more than five times for the same bandwidth as in Bishkek.

Establishment of the Ferghana Valley Internet Exchange Point (FVIXP) is therefore extremely important for improving connectivity in the region, particularly with respect to reducing costs. This open and neutral exchange, supported by the Internet Society, is planned to be built in Osh, but will also need network engineers to support it which was one of the motivations for organising a capacity building workshop to develop the necessary skills.

The Internet Society would like to thank the Internet Society Kyrgyzstan Chapter, the National Academy of Sciences of the Republic of Kyrgyzstan, Nishal Goburdhan and Packet Clearing House, and the EU-funded CAREN Project for their support of this workshop.

Categories
Deploy360 Mutually Agreed Norms for Routing Security (MANRS) Securing Border Gateway Protocol (BGP)

Routing Security boosted by major network operators

There have been some important developments towards improving routing security over the past few weeks, with announcements at NLNOG and AusNOG, as well as from Cloudflare about commitments to validate IP prefixes and reduce route leaks and hijacks. This supports the work we’ve being doing with the MANRS initiative to raise awareness of this issue, and to persuade network operators to take collaborative responsibility for this critical aspect of the Internet.

Cloudflare to deploy RPKI

Cloudflare has been a long-time advocate of routing security, and during their recent Crypto Week, they announced that they’ll be deploying RPKI on their networks. Resource Public Key Infrastructure (RPKI) allows IP address prefixes and AS numbers to be cryptographically verified (using Route Origin Authorization), and therefore provides some assertion that the holders of these have the right to announce them. The use of RPKI is included as one of the four MANRS actions “Global Validation – facilitating validation of routing information on global scale” which includes the creation of ROAs and the maintenance of accurate data in Internet Routing Registries (IRRs).

Cloudflare also announced GoRTR, which is an open-source implementation of the RPKI to Router (RTR) protocol (see RFC 6810). This is written in the Go Programming language, and is able to retrieve and validate RPKI (see RFC 6480) prefix origin data from a trusted cache. All major router vendors such as Cisco, Juniper, Huawei, Arista, Quagga, FRR are able to support RTR.

Lists of valid routes are now being securely distributed (via HTTPS) via Cloudflare’s Content Delivery Network (CDN) using a lightweight local RTR server. This free service is being offered in order to encourage adoption of route origin validation on the Internet.

NLnet Labs announce Routinator

This follows on from the announcement of Routinator from NLnet Labs back in July. This is an open source RPKI toolset that offers a Certificate Authority (CA) package to allow network operators to run RPKI on their own systems instead of using the hosted platforms offered by the five Regional Internet Registries; a Publication Server, to let network operators publish the certificates and ROAs generated by the CA; and finally Relying Party software that allows network operators to download the global RPKI dataset, validate it and use it their BGP decision making process.

NTT IRRdb to provide ROA validation

Job Snijders (NTT) also announced during the recent NLNOG 2018 event, that the NTT IRR database will now provide ROA validation status as well. NTT operates one of the major Internet Routing Registries, but until recently virtually anyone could create any route/route6 object and sneak those into the prefix-filters. But perhaps one of the most important points of his presentation was that he believed only about 20 major network operators needed to start doing Route Origin Validation in order to greatly improve routing security and achieve big benefits.

AusNOG & Hurricane Electric

AusNOG 2018 was held on 30-31 August 2018 in Sydney, Australia. This is one of the most important and influential national Network Operator Groups, and this year I did a lightning talk on possible route hijacks, possible route leaks and bogon announcements, based on data from bgpstream.com.

One incident of a possible route hijack occurred in Australia where a network operator started advertising a handful of prefixes that they didn’t own. Unfortunately, one of their peers Hurricane Electric accepted those advertisements and it ended up reaching the global routing table.

However, Walt Wollny (Hurricane Electric) was up next, and was able to announce they had already started to implement strict filtering with its direct peers . They also have a website showing the AS numbers of all their direct peers, what prefixes they were receiving, and what policy was being being implemented, along with a documented algorithm for prefix filtering.

Cloudflare and MANRS

Last but not least, Martin Levy (Cloudflare) mentioned the MANRS initiative in his blog and re-iterated Cloudflare’s willingness to collaborate, although expressing the view that MANRS does not go far enough or have sufficient ability to weed out bad actors. Whilst not disagreeing with this statement, I would point out that MANRS is primarily intended to be an awareness raising initiative to persuasively bring about behavioural change amongst network operators, of whom many are simply unaware that routing security is a substantial issue. As the MANRS community grows, not only will awareness increase, but it will become easier for the good actors to identify and take actions against those operators where prefix hijacks, route leaks and bosons are prevalent.

Get Involved

It’s great to see so many good things starting to happen around routing security, and more network providers getting involved in implementing the solutions. If you are running a network infrastructure then be part of the solution and help protect the core. Join MANRS.

Further Information

Categories
Deploy360 Mutually Agreed Norms for Routing Security (MANRS) Securing Border Gateway Protocol (BGP)

Even Better MANRS During August

We already discussed the MANRS activities during SANOG 32 where we organised a Network Security Workshop and signed an MoU with the ISP Association of Bangladesh (ISPAB), but the Internet Society was also involved with three other events during the month of August. This included the Symposium on Internet Routing Security and RPKI, VNIX-NOG 2018 and the inaugural INNOG 1.

Symposium on Internet Routing Security and RPKI

ZDNS along with CNCERT organised a symposium on 17th August at Crowne Plaza Beijing to discuss routing security issues and how RPKI can help address this problem. There were many prominent participants representing local, regional and international entities including Baidu, Tencent, Alibaba, Huawei, ZTE, the Chinese Academy of Sciences, APNIC, ICANN, along with the Internet Society.

Dr Stephen Kent (BBN) was the keynote speaker, having played an important role in the SIDR (Secure Internet Domain Routing) Working Group at the IETF (Internet Engineering Task Force) and also co-authored many RFCs (Request for Comments) on RPKI. He discussed the ideas behind RPKI and Route Origin Authorization/Validation.

George Michaelson (APNIC) who along with his colleague Geoff Huston co-authored RFC 6483 – Validation of Route Origination Using the Resource Certificate Public Key Infrastructure (PKI) and Route Origin Authorizations (ROAs) – followed this with a presentation on the status of RPKI deployment in the region with specific focus on China. CNNIC, the National Internet Registry (NIR) of China, only started offering ROA services in November 2017, so there is currently limited uptake, which is why the symposium is hoping to raise awareness of the importance of implementing RPKI.

I then outlined some of practical issues that network operators need to consider when deploying Route Origin Validation, as well as highlighting how the issues related to unallocated IPv4/v6 address space (aka Bogons) can be addressed.

VNIX-NOG 2018

The VNIX Network Operators Group Conference (VNIX-NOG) was held on 24 August 2018 in Danang City, Vietnam. This annual event is organised by the Vietnam Internet Network Information Centre (VNNIC), a National Internet Registry, and has become an important focus for local community to improve understanding and knowledge, and offers a platform to discuss information security issues relevant to Vietnam.

The theme of this year’s event was Routing Security, and Srinivas Chendi (APNIC) provided some background on RPKI, whilst Hiroki Kawabata (JPNIC) shared JPNIC’s experience of deploying RPKI for.

I then presented the ‘Routing Security landscape‘ that highlights the issues of BGP hijacks and leaks and how it would be beneficial for ISPs to join MANRS initiative. Very interestingly, not a single service provider in Vietnam advertises any IPv4/v6 Bogon.

INNOG 1

The ISP Association of India (ISPAI) organised the inaugural Indian NOG (INNOG) in the capital city of New Delhi. The event comprised three days of workshops (27-29 August 2018) and a one-day conference (30 August).

The Internet Society supported the Network Security workshop with our community trainers Moinur Rehman and Anirban Data, and turned to be fully booked. Our colleague Subhashish Panigrahi presented the MANRS initiative and highlighted the routing issues impacting Indian ISPs and enterprises, as well as the simple steps can be taken to address the problems.

Following this on 3 September 2018, the Internet Society signed a MoU with the ISP Association of India (ISPAI) in order to promote the MANRS principles and encourage member ISPs to sign-up as MANRS participants. The MoU was signed by Rajnesh Singh (ISOC Regional Bureau Director for the Asia-Pacific region) and Rajesh Chhariya (President ISPAI) and will give a great boost to improving the routing security situation in India.

AusNOG 2018

I also participated in AusNOG 2018, the Australian Network Operators’ Group, on 30-31 August 2018 in Sydney, Australia. A number of interesting announcements were made here, which will be the subject of a separate forthcoming blog.

Further Information

Categories
Deploy360 Events IETF IPv6 Open Internet Standards Securing Border Gateway Protocol (BGP)

IETF 102, Day 5: Au revoir Montréal

There’s just the couple of sessions to highlight on the last day of IETF 102 before we wrap up for the week.

V6OPS continues at 09.30 EDT/UTC-4 where it left off on Thursday afternoon. On the agenda are drafts relating to Multi-Addressing Considerations for IPv6 Prefix Delegation which considers prefix delegation considerations for both classic routing and various multi-addressing use cases; whilst IP over Ethernet (IPoE) Session Health Checking describes a mechanism for IP over Ethernet clients to achieve connectivity validation using PPP-style keepalives such as BFD Echo, or ARP and Neighbor Discovery functions.

The remaining draft proposes a method for Discovering Provisioning Domain Names and Data, which describes a way for hosts accessing the Internet via multiple interfaces and with possible multiple IPv6 prefixes, to identify themselves using Fully Qualified Domains as Provisioning Domain identifiers.


NOTE: If you are unable to attend IETF 102 in person, there are multiple ways to participate remotely.


The final session starting 11.50 EDT/UTC-4 includes IDR. This has been working on (amongst other things) the issue of route leaks, and is trying to pull together different conflicting approaches towards mitigation of these in favour of a more complementary approach. This work includes two drafts on Methods for Detection and Mitigation of BGP Route Leaks, and on Design Discussion of Route Leaks Solution Methods.

With that, IETF 102 draws to a close and we say au revoir to Montréal. Many thanks for reading along this week… please do read our other IETF 102-related posts … and we’ll see you at IETF 103 on 3-9 November 2018 which for the first time is being held in Bangkok, Thailand!

Relevant Working Groups