Dear network operators, please use the existing tools to fix security

Internet routing may well be a screaming car wreck, but a deployathon by the Asia Pacific Network Information Centre (APNIC) has shown how short, focused efforts can make a difference.

Routers use the Border Gateway Protocol (BGP) to tell each other the current best ways to route internet traffic, but the system relies on everyone telling the truth.

The BGP standard includes so-called Resource Public Key Infrastructure (RPKI) Route Origin Authorisations (ROAs) to certify the truth of routing messages, but they’re not deployed as widely as they might be.

As APNIC’s chief scientist Geoff Huston says, internet routing is therefore a “system that relies on the propagation of rumours”.

False rumours can be mistakes that cause routing failures — sometimes on a massive scale. They can also be deliberate attempts to engineer malicious traffic hijacks.

This month’s APNIC conference in Chiang Mai included a full-day workshop on advanced BGP. The 26 participants used a virtual environment to learn how to deploy RPKI, sign ROAs, and set up Route Object Validation (ROV) on a variety of routers.

“Some of them have registered ROAs for their organisations and they’re going to deploy when they go back home, they say,” said APNIC consultant Dr Philip Smith.

Some didn’t wait that long. Later in the conference, a few enthusiastic participants got together to sign RPKI ROAs and publish them.

Over the following ten hours, the total number of Validated ROA Payloads in the APNIC RPKI repository jumped from 25,844 to 25,897.


The growth in Validated ROA Payloads in the APNIC RPKI repository following the conference session earlier this month.

Screenshot: Alexander Band

The APNIC sessions to sign ROAs and set up ROV on routers have been “incredibly valuable”, according to Alexander Band, head of product development at NLnet Labs, which makes free, open source software for domain name system (DNS) and routing infrastructure.

“It provides networks immediate protection against the most common form of BGP hijacking,” he told ZDNet.

New research by London-based systems engineer Ben Cartwright-Cox shows that more than 600 networks worldwide now drop RPKI invalid routes. These include smaller networks, as well as large Tier 1 carriers such as AT&T [PDF].

One of the most recent additions to the list has been Swedish network provider Telia Carrier, which has operations in Sweden, Finland, Norway, Denmark, Lithuania, Latvia, and Estonia.

Telia announced on Monday that it had implemented RPKI across its entire global internet backbone.

Telia’s network, autonomous system number AS1299, is currently the world’s number one according to Dyn Research’s global backbone rankings. Its directly connected customer base accounts for nearly 60% of global internet routes.

“As the leading global internet backbone, route stability is paramount and we encourage our network customers, peers, and the internet community, in general, to support the RPKI initiative by implementing it in their own networks,” said Jorg Dekker, Telia’s head of internet services.

Better tools would lead to better routing

The APNIC workshop also highlighted the unfortunate fact that many of the tools for setting up RPKI aren’t the best.

NLnet Labs’ route validator Routinator worked straight out of the box, but according to Smith, the RIPE NCC Validator and Cloudflare’s OktoRPKI had real problems that could be made worse by poor documentation.

“Routinator is the only validator fit for purpose,” Smith said.

“Many netops have no Linux experience, and the other two need a lot of figuring out things to make the installation work. If you have not done Linux, you haven’t a hope. So that all needs to be sorted.”

Telia’s AS1299 is only one of more than 65,000 autonomous networks comprising the internet. Poor tools may well produce poorly configured validators in at least some of these systems and that can pose problems.

“If everybody turns this on, none of it is going to work, so this needs to be sorted before we go any further,” Smith said.

Let’s also fix DNS, email authentication, and website encryption

It’s tempting to point the finger at network operators for failing to deploy RPKI. But another finger needs to be pointed at the software vendors for providing shoddy documentation.

Routing security isn’t the only system where deploying existing tools can make a big difference.

Huston said in 2017 that failing to secure the DNS with DNSSEC is savage ignorance. Network operators should get onto that before fingers are pointed at them.

Network operators should also avoid being the recipient of pointing fingers by deploying DMARC message authentication to prevent spammers from spoofing their domains for email.

The UK’s National Cyber Security Centre (NCSC) has used DMARC to significantly reduce that risk for government domains.

“That’s how you stop people clicking on the link, because they never get the crap in the first place. Simple things done at scale can have a difference,” said Dr Ian Levy, the NCSC’s technical director in 2018.

The Australian government has also been deploying DMARC on its domains, though its efforts have lagged behind the UK.

Then there’s website encryption.

Huston says that every website should be running TLS encryption, forcing all users to connect via HTTPS.

“They should. Fool if you don’t,” he told ZDNet.

“When I’m going to somewhere, even if the routing system is lying, that somewhere has to demonstrate that they’re the party I wanted to get to. And that’s really important.”

But many web hosting providers still price TLS encryption as a premium service. SSL certificates are a profitable up-sell. They don’t install the simple tools that would enable website operators to use free Let’s Encrypt certificates.

Making it harder to do things securely is, in the view of your writer, irresponsible and perhaps even reckless.

As for securing things like routing, DNS, and email authentication, it may not be sexy new work, but it’s work that really needs to be done. Do it.

Disclosure: Stilgherrian travelled to Chiang Mai, Thailand, as a guest of APNIC.