Categories
Deploy360 Domain Name System (DNS) Domain Name System Security Extensions (DNSSEC) Improving Technical Security

Are you ready? How to prepare for the DNSSEC Root KSK Rollover on October 11, 2018

Are you ready? Are your systems prepared so that DNS will keep functioning for your networks?  One week from today, on Thursday, October 11, 2018, at 16:00 UTC ICANN will change the cryptographic key that is at the center of the DNS security system – what we call DNSSEC. The current key has been in place since July 15, 2010. This is a long-planned replacement.

If everything goes fine, you should not notice and your systems will all work as normal. However, if your DNS resolvers are not ready to use the new key, your users may not be able to reach many websites, send email, use social media or engage in other Internet activities!

This change of this central security key for DNS is known as the “Root Key Signing Key (KSK) Rollover”. It has been in discussion and planning since 2013. We’ve written many articles about it and spoken about it at many conferences, as have many others in the industry. ICANN has a page with many links and articles at:

But here we are, with only a few days left and you may be wondering – how can I know if my systems are ready?

The good news is that since the Root KSK Rollover was delayed 1 year, most all of the DNS resolver software has been shipping for quite some time with the new key. If you, or your DNS server administrators, have been keeping up with recent updates, you should be all set.

1. Test if you are doing DNSSEC validation

Before you do anything else, you should first check if you are doing DNSSEC validation on your network.  As noted in ICANN’s guidance document, go to a command-line / terminal / shell window and type:

dig @<IP of your DNS resolver> dnssec-failed.org a +dnssec

For example, using Google’s Public DNS Server, the command would be:

dig @8.8.8.8 dnssec-failed.org a +dnssec

If the response includes this text:

;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL

then you ARE doing DNSSEC validation and should read the rest of this article.

If the response instead includes:

;; ->>HEADER<<- opcode: QUERY, status: NOERROR

… well, you are NOT doing DNSSEC validation. You can skip the rest of this article, go have a beverage, and not have to worry about the Root KSK Rollover on October 11.  However, you should also read up on DNSSEC and understand why you start validating to raise the level of security and trust on your network. (But, at this point, you might as well wait until October 12 to deploy it.)

If you are doing DNSSEC validation, read on. 

Two notes:

  • Unfortunately if you are not an administrator of your DNS resolvers, there are limited mechanisms to check if you have the new key. There are a couple of possibilities (see #2 and #3a below), but otherwise you will need to contact your DNS administrators / IT team and point them to this blog post and other resources.
  • In DNS / DNSSEC circles the root key is also referred to as a “trust anchor”.

2. Try the Sentinel KSK Test

For a small percentage of you reading this, you might be able to use the “sentinel test” that is based on an Internet draft that is in development. You can do so at either of these sites:

Right now there is only one DNS resolver (Unbound) that implements this sentinel test. Hopefully by the time we do the next Root KSK Rollover, some years from now, this will be more widely deployed so that regular users can see if they are protected.

[UPDATE: the Knot DNS resolver  also supports the Sentinel Test in its version 3.0.0 release – see the release notes.]

However, for most of us, myself included, we need to go on to other methods…

3a. Check if your DNS resolvers have the new Root KSK installed – via various tools

There are several tests you may be able to perform on your system. ICANN has published a list at:

That document lists the steps for the following DNS resolvers:

  • BIND
  • Unbound
  • PowerDNS Recursor
  • Knot Resolver
  • Windows Server 2012RS and 2016
  • Akamai DNSi Cacheserve
  • Infoblox NIOS

For BIND users, ISC2 also provides a focused document: Root KSK Rollover in BIND.

3b. Check if your DNS resolvers have the new Root KSK installed – via specific files

If you have command-line access to your DNS servers, you can look in specific files to see if the new key is installed.  The current key (“KSK 2010”) has an ID of 19036. The new key has an ID of 20326. As Paul Wouters wrote in a Red Hat blog post today, these keys can be found in these locations in Red Hat Linux:

  • bind – see /etc/named.root.key
  • unbound / libunbound – see /var/lib/unbound/root.key
  • dnsmasq – see /usr/share/dnsmasq/trust-anchors.conf
  • knot-resolver – see /etc/knot-resolver/root.keys

Look in there for a record with an ID of 20326. If so, you are all set. If not, you need to figure out how to get the new key installed.

Note – these locations here are for Red Hat Linux, CentOS, and Fedora. Other Linux distributions may use slightly different file locations – the point is that there should be a file somewhere on your system with these keys.

4. Have a backup plan in case there are problems

As Paul notes in his post today, it would be good to have a backup plan in case there are unexpected DNS problems on your network on October 11 and users are not able to resolve addresses via DNS. One suggestion is to temporarily change your systems to give out one of the various sets of “public” DNS servers that are operated by different companies. Some of these include:

IPv4 IPv6 Vendor
1.1.1.1 2606:4700:4700::1111 Cloudflare
8.8.8.8 2001:4860:4860::8888 Google DNS
9.9.9.9 2620:fe::fe Quad9
64.6.64.6 2620:74:1b::1:1 Verisign

You can switch to one of these resolvers while you sort out the issues with your own systems. Then, once you have your systems correctly configured, you can switch back so that the DNSSEC validation is happening as close to your users as possible (thereby minimizing the potential areas of the network where an attacker could inject malicious DNS traffic).

5. Plan to be around on 11 October 2018 at 16:00 UTC

Finally, don’t schedule a day off on October 11th – you might want to be around and able to monitor your DNS activity on that day.  This Root KSK Rollover has been in the works for many years now. It should be a “non-event” in that it will be “just another day on the Internet”. But many of us will be watching whatever statistics we can. And you’ll probably find status updates using the #KeyRoll hashtag on Twitter and other social networks.

The end result of all of this will be the demonstration that we can safely and securely change the cryptographic key at the center of DNS – which allows us to continue improving the level of security and trust we can have in this vital part of the public core of the Internet!


Image credit: Lindsey Turner on Flickr. CC BY 2.0

P.S. This is NOT what the “Root key” looks like!

Acknowledgements:  Thanks to Ed Lewis, Paul Hoffman, Paul Wouters, Victoria Risk, Tony Finch, Bert Hubert, Benno Overeinder, Hugo Salgado-Hernández, Carlos Martinez and other members of the dnssec-coord discussion list for the discussion that informed this post.

Categories
Deploy360 Domain Name System (DNS) Domain Name System Security Extensions (DNSSEC)

ICANN seeking public comment on Root KSK rollover process for DNSSEC

On 11 October 2018, should ICANN roll the Root Key Signing Key (KSK) that is at the heart of DNSSEC? ICANN is planning to restart the rollover process for the Root KSK and is therefore seeking public review of their new plan.  It includes more publicity about the need to be prepared for the rollover, and analysis of data indicating the level of preparedness.

The Plan for Continuing the Root KSK Rollover describes how ICANN intends to roll the root key signing key (KSK), and is based on input from the technical community following their decision to postpone the rollover last year.

Further input is requested by 2 April 2018. This will be used to prepare a final plan that will be presented to the ICANN Board for approval. ICANN is seeking public comments and we encourage you to read the plan and submit your views.

Learn how to submit your comments to ICANN

The Root KSK was originally planned to be rolled over on 11 October 2017, but ICANN postponed the rollover due to collected data that showed that a significant number of resolvers used by network operators were not ready for this. This meant that significant sections of the Internet could experience issues with resolving DNSSEC-signed domains following the rollover, so it was considered prudent to wait and reach out to affected network operators.

ICANN manages the Root Key Signing Key (KSK) that acts as the trust anchor for DNSSEC in the global Domain Name System. This key is used to sign the VeriSign-managed Root Zone Signing Key (ZSK) that validates the Top-Level Domains (TLDs). The Root KSK needs to be configured in DNSSEC-aware resolvers to allow validation of the chain-of-trust, and by extension all cryptographically-secured records in the DNS.

The current Root KSK has been used since the DNS Root Zone was first signed in 2010, and it’s good practice to change keys periodically. ICANN wanted to attempt this rollover under normal rather than comprised conditions, so it was not imperative that the rollover happened as planned in 2017, and clearly sufficient DNSSEC resolvers need to have the new trust anchor configured if this process is to be a smooth undertaking.

RFC 8145 (“Signaling Trust Anchor Knowledge”) was published in April 2017, and specifies how recursive name servers can signal to authoritative servers, the trust anchors that they have configured for their DNSSEC validation. This was implemented by both Unbound and BIND shortly afterwards, and as organisations began to deploy the new software versions, some of this “key tag data” started appearing in queries to the root name servers. This is useful information for the KSK rollovers, especially for the root, but it would seem that the number of recursive name servers providing this data was not as high as one might like for the planned root KSK rollover last year.

Further Information

For more information on DNSSEC and how to deploy it, please see our Start Here page for more information!

Categories
Deploy360 Domain Name System (DNS) Domain Name System Security Extensions (DNSSEC)

ICANN Postpones DNSSEC Root KSK Rollover – October 11 will NOT be the big day

People involved with DNS security no longer have to be focused on October 11. News broke yesterday that ICANN has decided to postpone the Root KSK Rollover to an unspecified future date.
To be clear:

The Root KSK Rollover will NOT happen on October 11, 2017.

ICANN’s announcement states the the KSK rollover is being delayed…

…because some recently obtained data shows that a significant number of resolvers used by Internet Service Providers (ISPs) and Network Operators are not yet ready for the Key Rollover. The availability of this new data is due to a very recent DNS protocol feature that adds the ability for a resolver to report back to the root servers which keys it has configured.

Getting More Information

UPDATE – 4 Oct 2017 – 

Discussion on the public DNSSEC-coord mailing list indicates more info may be available in a talk Duane Wessels is giving at the DNS-OARC meeting tomorrow (Friday, September 29). The abstract of his session is:


A Look at RFC 8145 Trust Anchor Signaling for the 2017 KSK Rollover

RFC 8145 (“Signaling Trust Anchor Knowledge”) was published in April 2017. This RFC describes how recursive name servers can signal, to authoritative servers, the trust anchors that they have configured for Domain Name System Security Extensions (DNSSEC) validation. Shortly after its publication, both Unbound and BIND implemented the specification. As organizations begin to deploy the new software versions, some of this “key tag data” is now appearing in queries to the root name servers.

This is useful data for Key Signing Key (KSK) rollovers, and especially for the root. Since the feature is very new, the number of recursive name servers providing data is not as significant as one might like for the upcoming root KSK rollover. Even so, it will be interesting to look at the data. By examining this data we can understand whether or not the technique works and hopefully inspire further adoption in advance of future KSK rollovers.


If you, like me, will not be in San Jose for this session, there will be a webcast / live stream. The link should be available tomorrow morning on the DNS-OARC event page. Or you can follow the #oarc27 hashtag or @dnsoarc onTwitter.

Per the OARC 27 timetable, Duane’s talk begins at 9:40am PDT (UTC-7). (Side note: for those involved with DNS, there are many other excellent sessions on the timetable!)

Apparently whatever data ICANN received through this research convinced them that not enough ISPs were ready to go with the new KSK and so a postponement was necessary.

Understandable caution

I do understand why ICANN would step back and delay the KSK roll. If there are significant sections of the Internet that will experience issues with resolving DNSSEC-signed domains on October 11, it is prudent to wait to assess the data and potentially reach out to affected ISPs and other network operators. Particularly when, as we noted in our State of DNSSEC Deployment 2016 report last year, the number of domains signed with DNSSEC continues to grow around the world.

I look forward to working with ICANN and the rest of the DNSSEC community to set a new date. As I wrote (along with my colleague Andrei Robachevsky) in our comments back in April 2013, we believe that the Root KSK should be rolled soon – and rolled often – so that we gain operational experience and make Root KSK rollovers just a standard part of operations.  (Note: our CITO Olaf Kolkman submitted similar comments, although at the time he was with NLnet Labs.)

Updating the DNS infrastructure is hard

The challenge ICANN faces is that updating the global DNS infrastructure is hard to do. The reality is that DNS resolvers and servers are massively DE-centralized and controlled by millions of individual people. You probably have one or more DNS resolvers in your home in your WiFi router and other devices.

The success of DNS is that generally it “just works” – and so IT teams often set up DNS servers and then don’t pay much attention to them. At a talk I gave yesterday to about 180 security professionals at the ISC2 Security Congress in Austin, TX, I asked how many people had updated the software on their DNS resolvers within the past year – only a few hands were raised.

All of the latest versions of the major DNS resolvers support the new Root KSK. Recent versions all generally support the automated rollover mechanism (RFC 5011). But… people need to upgrade.

And in the example of a home WiFi router, the vendor typically needs to upgrade the software, then the service provider has to push that out to devices… which can all take a while.

A group of us looking to expand the use of elliptic curve cryptography in DNSSEC wrote an Internet Draft recording our observations on deploying new crypto algorithms. Updating the root KSK as a trust anchor faces a similar set of issues – although a bit easier because the focus is primarily on all the DNS resolvers performing DNSSEC validation.

The critical point is – upgrading the global DNS infrastructure can take some time. ICANN and members and of the DNSSEC community (including us here at the Internet Society) have been working on this for several years now, but clearly the new data indicates there is still work to do.

Next Steps

The good news is that companies now have more time to ensure that their systems will work with the new key.  The new Root KSK is published in the global DNS, so that step has at least been done. More information is available on ICANN’s site:

https://www.icann.org/kskroll

I would recommend two specific pages:

The time to do this is NOW to be ready for the Root KSK Roll when it does happen.

For more information about DNSSEC in general, please see our Deploy360 DNSSEC page.


Image credit: Lindsey Turner on Flickr. CC BY 2.0

P.S. And no, that is NOT what the “Root key” looks like!

Categories
Deploy360 Domain Name System Security Extensions (DNSSEC) To archive

Watch Live TODAY – DNSSEC Root KSK Ceremony at 17:00 UTC

DNSSEC Key Ceremony 25

Today a critical part of DNS security – DNSSEC – will receive a major update, and you can watch it all live at starting at 17:00 UTC (1:00pm US EDT – local time) streaming out of ICANN’s data center in Virginia:

https://www.iana.org/dnssec/ceremonies/27

Olaf Kolkman, our CITO, will be in attendance as a “Crypto Officer” (key holder). Olaf wrote a post with info about the 25th key ceremony back in May 2016 and shared some of his photos.

The important step today is that this key ceremony will involve the creation of a new Key Signing Key (KSK) for the root of DNS. This begins what will be a year-long process of “rolling over” the cryptographic key at the heart of the DNSSEC system. ICANN has a page dedicated to the “Root KSK Rollover” explaining the details – and this “at-a-glance” PDF provides the key facts and dates.

This is a great step in making DNSSEC even more secure.

If you’re interested, ICANN posts the “script” that will be used to go through today’s key ceremony. All of the key ceremonies are streamed live and archived for later viewing.

If you want to learn more about DNSSEC in general, please visit our Start Here page to find resources to help!


Image credit – Olaf Kolkman on Flickr. Used with permission.

Categories
Deploy360 Domain Name System Security Extensions (DNSSEC) IETF

New RFC 7958 – DNSSEC Trust Anchor Publication for the Root Zone

RFC 7958 text

How can you trust the root of the “global chain of trust” that is used in DNSSEC? How can you be sure as you are validating DNSSEC signatures that this global chain works?

To provide this chain of validation, DNSSEC relies on what is called a “trust anchor”. When you check the signature for DNS records for “internetsociety.org”, for instance, you go through a process along the lines of this (a simplified version):

  1. Your validating recursive resolver gets the DNS records (such as “A” or “AAAA”) for “INTERNETSOCIETY.ORG” along with the DNSSEC signature in a RRSIG record and the public key used for the signing in a DNSKEY record.
  2. It then retrieves the DS record for “INTERNETSOCIETY.ORG” from .ORG to verify that this is the correct DNSKEY.  It also retrieves a RRSIG record for the DS record and the DNSKEY record from .ORG.
  3. Next it retrieves the DS record for “.ORG” from the root of DNS, along with the associated RRSIG for the DS record and the DNSKEY for the root.
  4. HERE IS THE CHALLENGE – How does your recursive resolver know that the DNSKEY it retrieved for the root of DNS is the correct one?

This is where there is a need for a “trust anchor” that the recursive resolver can trust to know that this is indeed the correct DNSKEY it should be using.

The DNSSEC protocol can be used with any trust anchor, but in practice we all use the DNSSEC trust anchors published by IANA (with ICANN doing the actual publishing as part of their contract to perform the IANA functions).

A new informational (non-standard) RFC 7958 out this week explains the formats IANA uses to publish the root key trust anchors and how those trust anchors can be retrieved.  It also outlines additional steps that can be taken during the retrieval to ensure the trust anchors aren’t modified during the retrieval.

In 2017 we will see a change in the Root Key Signing Key (KSK) in 2017, which will mean a change in the root trust anchor. This RFC 7958 is a good reference to have out there so that everyone can understand exactly how to retrieve and use the trust anchors at the heart of DNSSEC.

Please do read this new RFC and share it widely with anyone involved in developing applications or services that perform DNS resolution and validation.

And if you know very little about DNSSEC but want to learn more, please visit our Start Here page to find resources to help you get started!

Categories
Deploy360 Domain Name System Security Extensions (DNSSEC)

5 Hours Left To Submit Comments on ICANN Design Team Review of Plan for DNS Root Zone KSK Change

ICANN.jpgDo you have any comments on the findings of the ICANN Design Team regarding the changing of the root zone key-signing key (KSK) for DNSSEC?  If so, you have about five hours left to submit your comments as the comment period ends at 23:59 UTC today, 5 October 2015. You can read the Design Team report and submit your own comments at:

https://www.icann.org/public-comments/root-ksk-2015-08-06-en

The comment period has been open since August 6, 2015, and the word has been distributed through multiple online mailing lists and other forums in the time since.  To date there have only been a few comments, although I’m seeing several (including my own) coming in today:

http://forum.icann.org/lists/comments-root-ksk-06aug15/index.html

You may recall that ICANN announced the members of this design team back in February 2015 and this was after a comprehensive public comment period back in 2013.  Here are some links that can provide some context:

As you will see in my own response, I am generally pleased with the findings of the Design Team but have a few points I wish to add.

NOW IS THE TIME TO SUBMIT YOUR COMMENTS… you have about five hours left!

P.S. And if you just want to learn what DNSSEC is all about, please visit our Start Here page to learn more!

Categories
Deploy360 Domain Name System Security Extensions (DNSSEC)

ICANN Announces DNSSEC Root KSK Rollover Design Team

ICANN.jpgAfter soliciting statements of interest back in December, ICANN announced this week the people who had been selected for the DNSSEC Root Key Signing Key (KSK) Rollover Design Team. They are:

  • Joe Abley, Snake Hill Labs/DyN, CA
  • Jaap Akkerhuis, NLNetLabs, NL
  • John Dickinson, Sinodun Internet Technologies, UK
  • Geoff Huston, APNIC, AU
  • Ondrej Sury, CZ.NIC, CZ
  • Paul Wouters, No Hats/Red Hat, NL
  • Yoshiro Yoneya, JPRS, JP

We’ve written before about how important we believe the rollover of the Root KSK of DNSSEC is, and we are pleased to see this next step in the process.  All of the people selected have been extremely involved in the DNS / DNSSEC community for many years and have contributed in many ways to the ongoing deployment of DNSSEC.

We look forward to hearing the next steps taken by this team to move forward on rolling the Root KSK.  I suspect there will be some discussion at ICANN 52 next week in Singapore, but I also expect much more to happen after that event in the months ahead.

P.S. If you want to get started with DNSSEC, please visit our Start Here page to begin!

Categories
Deploy360 Domain Name System Security Extensions (DNSSEC)

ICANN Seeking Volunteers For DNSSEC Root KSK Rollover Plan Design Team

ICANN.jpgDo you want to help ICANN plan the best was to roll the root key used for DNSSEC?  Are you interested in being considered as a volunteer member of ICANN’s Root KSK Rollover Plan Design Team?  Recently ICANN staff sent a message to the public dnssec-coord mailing list and other various mailing lists asking for volunteers.  The “Solicitation of Statement of Internet for Membership in the Root Zone Key Signing Key Rollover Plan Design Team” (say that 10 times fast!) begins:

ICANN, as the IANA functions operator, in cooperation with Verisign as the Root Zone Maintainer and the National Telecommunications Information Administration (NTIA) as the Root Zone Administrator, together known as the Root Zone Management (RZM) partners, seek to develop a plan for rolling the root zone keysigning key (KSK). The KSK is used to sign the root zone zone-signing key (ZSK), which in turn is used to DNSSEC-sign the Internet’s root zone. The Root Zone Partners are soliciting five to seven volunteers from the community to participate in a Design Team to develop the Root Zone KSK Rollover Plan (“The Plan”). These volunteers along with the RZM partners will form the Design Team to develop The Plan.

The document goes on to list the requirements and the process.  Essentially, if you meet the requirements you need to send a message with the requested information to ksk-rollover-soi@icann.org by the end of the day on Friday, January 16, 2015.  The Root Zone Management partners will then choose from among the applicants to form the Design Team.

We’ve written here before about how incredibly important it is to get the Root KSK Rollover right, and so we commend ICANN for going through this process to create an appropriate Design Team.  We would encourage people with operational knowledge of DNSSEC and DNS in general to definitely read over the document and consider applying!

P.S. And if you don’t know about DNSSEC, or want more information, please visit our Start Here page to find out how to begin!