Did you see all the tweets and mentions on the web last week about Comcast apparently blocking NASA’s website on their network? While it made for great headlines, the truth was that there was an error with the DNSSEC signing of the NASA.gov domain and now that Comcast has deployed DNSSEC-validating DNS servers those DNS servers were correctly responding with a failure and blocking access to the site.
Comcast alerted the administrators of the NASA.gov site and worked with them to resolve the issue. Comcast’s DNS Engineering team has now issued a very helpful analysis of the NASA.gov DNSSEC validation failure. As they outline in part of the executive summary:
On January 18, 2012, the NASA.GOV domain had a DNS Security Extensions (DNSSEC) signing error that blocked access to all NASA.GOV sites when using DNS recursive resolvers performing DNSSEC validation. As one of the largest ISPs in the world utilizing DNSSEC validation, users of Comcast noticed a problem when attempting to connect to the website. This caused some people to incorrectly interpret this as Comcast purposely blocking access to NASA.GOV and recommending users switch from Comcast security-aware DNS resolvers to resolvers not performing DNSSEC validation … Instead, the administrators of the NASA.GOV domain had enabled DNSSEC signing for their domain, and the security signatures in their domain were no longer valid. The Comcast DNS resolvers correctly identified the DNSSEC signature errors and responded with a failure to Comcast customers. This is the expected result when a domain can no longer be validated, and this protects users from a potential security threat.
The document then goes on at some length to explain how Comcast monitors for DNSSEC validation failures, how the NASA.gov domain failed, how the issue was resolved and how users responded to the issue. The document includes a number of charts from DNSViz showing the precise error in the DNSSEC chain-of-trust and then interestingly includes examples of the web and Twitter reaction to the problem.
In a message to the dnssec-deployment mailing list announcing the release of this document, Comcast’s Jason Livingood stated:
Since we feel that the entire Internet community has room for improvement on signing processes (not to single out NASA), we decided to start doing failure analyses here and there – and share them with the community in the hope that it will help bring greater operational scrutiny and maturity to DNSSEC signing processes.
Kudos to the Comcast DNS Engineering team for releasing this analysis as it is indeed quite helpful to understand how DNSSEC validation failed, what the user experience was and how the issue was resolved. We look forward to seeing more of these types of analysis documents as the global DNSSEC rollout continues. Sharing this level of detail will definitely help others who encounter similar situations and will only make the DNSSEC system that much stronger.
UPDATE: Dark Reading also put up a good post today on this topic: DNSSEC Error Caused NASA Website To Be Blocked