Deploy360 IPv6

New Apache SpamAssassin Release 3.4.0 Includes Native IPv6 Support!

Apache Spam Assassin logoVery cool news out of the Apache SpamAssassin project that the new release 3.4.0 has full native IPv6 support!  This is important because SpamAssassin is widely used as an anti-spam tool for email servers.  This update nows means that it can be used in mail servers running on IPv6, including on mail servers running in an IPv6-only environment.  From the release notes:

Improved support for IPv6

The rules-updating program sa-update and its infrastructure is now usable over either IPv4 or IPv6, including from an IPv6-only hosts (bug 6654).

SpamAssassin is now usable on an IPv6-only host: affects installation, self-tests, rule updates, client, server, and a command-line spamassassin.

Command line options -4 and -6 were added to prefer/choose/force IPv4 or IPv6 in programs spamassassin, spamd, spamc, and sa-update.

Command line options –listen and –allowed-ips in spamd can now accept IPv6 addresses.

Preferably a perl module IO::Socket::IP is used (if it is available) for network communication regardless of a protocol family – for DNS queries, by spamd server side, and by a client code in Mail::SpamAssassin::Client. As a fallback when the module IO::Socket::IP is unavailable, an older module IO::Socket::INET6 is used, or eventually the IO::Socket::INET is used as last resort.

If spamd fails to start with an ‘Address already in use’ message, please install perl module IO::Socket::IP, or deintall IO::Socket::INET6, or specify a socket bind address explicitly with a spamd –listen option. See bug 6953 for details.

The spamd server can now simultaneously listen on multiple sockets, possibly in different protocol domains (Unix sockets, INET or INET6 protocol families.

DnsResolver was updated allowing it to work on an IPv6-only host (bug 6653)

A plugin RelayCountry now uses module Geo::IP and its database of IPv6 addresses GEOIP_COUNTRY_EDITION_V6 when available.

The following configuration options were extended to accept IPv6 addresses: dns_server, trusted_networks, internal_networks, msa_networks, (but not yet the whitelist_from_rcvd), and their defaults were adjusted accordingly.

The parser code of Received header fields can now deal with IPv6 addresses in a mail header section.

The AutoWhitelist plugin was updated and can now deal with IPv6 addresses.

Installation unit tests were updated to prevent them from failing on an IPv6-only host.

This is excellent news and we congratulate the Apache SpamAssassin team on making this happen.  If you are a SpamAssassin user, you can get release 3.4.0 now to be able to fight spam on email connections over IPv6!

Deploy360 Domain Name System Security Extensions (DNSSEC)

NLnet Labs Releases Helpful DNSSEC Infrastructure Audit Framework

NLNet Labs DNSSEC Infrastructure Audit FrameworkHow secure is your DNSSEC infrastructure? If you operate a registry for a top-level domain (TLD) or if you are a DNS operator providing DNSSEC signing services, how secure are your operations?  And how secure are your mechanisms for communicating DNSSEC information with registrars and other entities?  Or, if you are a security auditor or researcher, how can you best assess the security of your client’s DNSSEC infrastructure?

To help assess DNSSEC infrastructure and answer questions like these, the great folks at NLnet Labs recently released a “DNSSEC Infrastructure Audit Framework” available publicly for anyone to use.  You can download the document and use it as a checklist to audit your own infrastructure or that of someone else.

As noted in the introduction, this document is not intended to be any kind of formal standard or assessment, but rather a guide and checklist to help people looking to understand how secure their DNSSEC infrastructure is:

A DNSSEC audit is the process of structural examination of a DNSSEC infrastructure. The purpose of this process is to evaluate the level of assurance of the system. This is achieved by reviewing the implementation and operation of the system controls and whether they are in compliance with the corresponding policy requirements or, in absence of formal policies, with best current industry practices.

A key document for performing an audit is a review checklist. The review checklist provides structure of the actual work and gives confidence that the audit scope is adequately covered. This document is a generic checklist for a DNSSEC review and provides a framework that assists auditors to perform an actual DNSSEC audit. However, the actions herein do not conform any formal audit standards and are merely intended to provide directions of how an audit might look like.

This document is neither standard nor best practice and is not suitable for any form of formal certification. Its intention is to offer a basis for a structured review of a DNSSEC environment.

The authors welcome feedback on this document so that it can mature. The licensing terms of the document are such that any entity may modify and publish the document on their own terms as long as NLnet Labs is being acknowledged. Incorporation in other documents, including standards is encouraged.

This is great contribution to the larger work of DNSSEC deployment and we thank Matthijs Mekking and Olaf Kolkman for both writing this document and then also making it public under a lenient license.

We hope many of you will find it helpful and do encourage you to provide feedback to Matthijs and Olaf. Using documents like this we can make the Internet more secure!

Deploy360 Events IPv6

Successful Live Video Streaming Over IPv6 Using Google+ Hangouts On Air and YouTube

SUCCESS!  We did prove this week that you can do live streaming of video from an event out over IPv6. As we wrote about earlier this week, one of our objectives at ION Krakow was to prove that we could to a live stream / webcast out of an event over IPv6.  Well, to be more precise, we wanted people to be able to receive the livestream over IPv6.

As it turned out, the network at PLNOG 11 (where ION Krakow occurred) was IPv4-only so we were streaming our video signal from ION Krakow only over IPv4 out to Google’s servers which were then streaming the video out over both IPv4 and IPv6.

We did have a few challenges with the actual broadcast (which I mention at the end), but the key point is…

it worked!

People were able to watch the live stream on both Google+ Hangouts On Air (HOA) and YouTube over IPv6 connections.  In our article we’d asked if some people watching could send us screen shots and several people did exactly that – as shown below.

Live Streaming To A Dual-Stack Computer

Longtime IPv6 advocate Shumon Huque tweeted out a screenshot showing that he was watching the live stream on our YouTube channel and was getting most of the video stream connections coming in over IPv6 (click/tap the image for a larger view):

Livestreaming of video over IPv6

He’s using the very cool IPvFoo/IPvFox browser extension to show the connections from the web page  and whether they are over IPv4 or IPv6.  Being on a dual-stack computer, Shumon’s web browser is going to use a “Happy Eyeballs” algorithm to determine for each requested connection whether IPv4 or IPv6 is fastest and so you will see situations like this where parts of the connection are still over IPv4.

Live Streaming To An IPv6-ONLY Computer

Lee Howard at Time Warner Cable took it a step further and turned IPv4 completely OFF on his Macbook Air. He then sent us this screenshot showing he was watching the video streaming over Google+ Hangouts on Air and also showing the output of a terminal window showing that his wireless interface had only IPv6 running (click/tap the image for a larger view – and yes, we blanked out part of his IPv6 addresses for his own privacy):

live streaming over IPv6

This proved to us rather definitively that our live stream was fully available to people over IPv6!

Challenges Unrelated to IPv6

We did have a couple of challenges with the actual broadcast content that were unrelated to IPv6. First, I missed a key setting in Google+ HOA where you specified the audio connection separate from the video connection. As a result for the first hour and 45 minutes until we figured out the problem we were streaming audio from my laptop’s microphone instead of from the event a/v system!  (Oops!)  The good news is that I was also running a separate audio recording directly from the mixer and so now I can go back and upload a new video that merges the camera video with the full audio stream.

The second issue was that we unfortunately discovered that Google+ Hangouts On Air have a 4-hour maximum time limit when the HOA stopped broadcasting right in the middle of the IPv6 panel! We had to restart the HOA which restarted the YouTube stream and required all viewers to go to new URLs to watch. The good news here is that I was separately recording the video stream to disk so even though we weren’t broadcasting the stream I have a local copy that I can now cut up and upload to YouTube.

There were a host of other “lessons learned” for this experiment that we’ve captured for the next time we do a live stream using Google+ HOA.  Thank you to everyone who participated in our experiment!

Conclusion – It Worked!

The great news out of all of this is that we proved that you can run a livestream for an event in such a way that people can watch the video stream over IPv6 – including on an IPv6-only network.  This is excellent to see and a good step for the continued IPv6 deployment.

Kudos to the teams at Google for making both Google+ Hangouts / HOA and YouTube all work over IPv6. Given the large size and ease of use of those services, this is great to have available.

The great thing is, of course, that livestream producers don’t have to do anything to make their live streams available over IPv6.  Simply by using Google+ HOA or YouTube the livestream becomes available over both IPv4 or IPv6.

This is how it should be!

And notice again that the live stream goes out over IPv6 even if you are broadcasting from an IPv4-only network. (Again, as it should be.)

We look forward to learning that other livestreaming services provide a similar functionality and allow viewers to watch a live stream over IPv6.  We’ll probably look to use this Google+ HOA setup again for our next livestream, although we may also try out “YouTube Live” to see if that works any better for us.  (And if there are other livestreaming services out there that will be supporting IPv6, please do let us know and we’ll be glad to look at your service.)

Many thanks to everyone who joined in to help us prove that live video streaming could be done over IPv6 using readily available services[1], and thanks in particular to both Lee and Shumon for sending in these screenshots to confirm availability over IPv6!   We’ll be uploading the individual sessions to our YouTube channel and I also intend to write up some general guidelines for live streaming over IPv6.

Great news!

[1] Yes, we could have installed one of the various live streaming servers on our own infrastructure and run it over IPv6, but not every company or organization has the ability to do this.  We wanted to try out the existing public live streaming services that anyone can easily use.

UPDATE: In speaking with someone today about this test and the livestreaming, it occurred to me that this really is NOT a milestone, i.e. “the first live streaming over IPv6”, because the reality is that everyone using Google+ HOA or YouTube for live streaming has been getting IPv6 connectivity for those viewers with IPv6 networks. This “live streaming over IPv6” has just been part of what Google has been offering for some time now.   We may have been one of the first ones to actively try to measure and demonstrate that we were live streaming over IPv6… but lots of other people have been doing it for quite some time now… but they just haven’t necessarily known about it.

And really, why should they care?  Live streaming over IPv6 should “just work” without the users on either the broadcasting or viewing end ever noticing.

Deploy360 Domain Name System Security Extensions (DNSSEC)

Google Clarifies DNSSEC Support – Opt In Now, Full Validation Coming Soon

Google logoAfter Google’s announcement earlier this week of DNSSEC validation support in their Public DNS service, there was some concern and discussion in various DNSSEC mailing lists about the fact that DNSSEC validation was not being performed by default and required a client to request validation.  Folks at Google clarified that this was just part of their initial rollout and that providing full validation is in their plans.

They have now also updated their FAQ about DNSSEC support in Google Public DNS and most importantly updated these two questions (my emphasis added):

Does Google Public DNS support the DNSSEC protocol?
Yes. Google Public DNS is a validating, security-aware resolver. Currently this is an opt-in feature: for queries coming from clients requesting validation (the AD and/or DO flag is set), Google Public DNS verifies that response records are correctly authenticated. Validation by default (i.e. for all queries) will be enabled soon.

Which client resolvers currently enable DNSSEC?
Unfortunately, most standard client stub resolvers do not enable full DNSSEC checking and cannot be easily reconfigured to do so. We have decided to make our initial launch only cover resolvers that explicitly ask for DNSSEC checking so that we become aware of any problems before exposing our users to possible large-scale DNS failures due to DNSSEC misconfigurations or outages. Once we are happy that we can safely enable DNSSEC for all users except those who explicitly opt out, we will do so.

It’s great to see Google responding to questions and adding these clarifications – and from the point of view of advocacy for DNSSEC deployment, it is great to have Google out there endorsing and promoting DNSSEC as a way to increase Internet security.

(And you can easily get started with DNSSEC if you haven’t already.)

For those of you who enjoy listening to audio, I recorded some audio commentary on our SoundCloud channel about why I view this news from Google as incredibly important:

Deploy360 Domain Name System (DNS) Domain Name System Security Extensions (DNSSEC)

Huge News For Internet Security – Google Public DNS Is Now Performing DNSSEC Validation!

Google logoIn a huge step forward for Internet security today, Google announced that Google’s “Public DNS” service is now performing DNSSEC validation. What this means is that anyone using Google’s DNS servers (and anyone can do so – see below) can now get the increased security that comes with DNSSEC.  (Learn more about the value of DNSSEC on our DNSSEC Basics page.)

It also means that if you want the added security of DNSSEC, but your Internet Service Provider and local operating system don’t validate with DNSSEC,  you can simply change your operating system to point to the following DNS servers operated by Google for either (or both) IPv4 and IPv6:


Once configured, all future DNS queries will be resolved using these DNS servers and DNSSEC validation will be performed by Google’s servers.  You will then benefit from the added protection of DNSSEC validation.  (Our resource page about Google Public DNS offers a few more pointers about configuration.)

Note that there is one important caveat here – you have to request DNSSEC validation when you send the DNS query to Google’s Public DNS servers, i.e. they will only validate the DNS query if you request it.  To do that you need an application that supports DNSSEC.  For web browsers, there are add-ons and extensions for both Google Chrome and Mozilla Firefox:

If you are an application developer, there are DNS developer libraries that support DNSSEC available in a wide range of programming languages so that you can add DNSSEC support to your application.

In the announcement, Google’s Yunhong Gu noted that Google Public DNS is currently “serving more than 130 billion DNS queries on average (peaking at 150 billion) from more than 70 million unique IP addresses each day.”  As the article further notes:

“Effective deployment of DNSSEC requires action from both DNS resolvers and authoritative name servers. Resolvers, especially those of ISPs and other public resolvers, need to start validating DNS responses. Meanwhile, domain owners have to sign their domains. Today, about 1/3 of top-level domains have been signed, but most second-level domains remain unsigned. We encourage all involved parties to push DNSSEC deployment and further protect Internet users from DNS-based network intrusions.”

To that end, if you have domains registered, we strongly encourage you to learn about how your can sign your domains with DNSSEC using domain name registrars.  You can learn more about which top-level domains support DNSSEC on our DNSSEC Statistics page.

Google provides the following information about using their Public DNS service:

This move by Google to provide this DNSSEC validation is a great addition to the support for DNSSEC validation offered by large US ISPs such as Comcast (making DNSSEC validation available to their 18 million customers) as well as ISPs in a wide range of countries including Sweden, the Czech Republic and Brazil.

We look forward to seeing more public DNS providers and more ISPs turn on DNSSEC validation in their networks.  If you want to know more about what is involved with enabling DNSSEC validation on your network, including home and enterprise networks, this SURFnet white paper provides easy instructions for common DNS servers.

And in the meantime, if you don’t want to wait for your ISP and want to start getting the value in DNSSEC validation today, you now have the option of using Google’s public DNS servers!


Deploy360 Domain Name System Security Extensions (DNSSEC)

PowerDNS Releases Version 3.2 With Increased DNSSEC Support

Congratulations to Bert Hubert and the rest of the team at PowerDNS for their release 3.2 last Thursday that, if you scroll down through the release announcement and changelog is pretty much mostly about improvements to their already strong DNSSEC support!  The list of changes and improvements is rather impressive.

In speaking with Bert last week, he said the team there views DNSSEC as basically “done” now for the authoritative server end and is now moving to focus on what they can do to make DNSSEC easier for deployment in DNS resolvers.  We’re looking forward to seeing what the team does there.

Meanwhile, if you are a PowerDNS user, the new release will give you even more DNSSEC power… time to upgrade!

Deploy360 Domain Name System Security Extensions (DNSSEC)

New Release 1.14 of DNSSEC-Tools – Get It Now!

Recently at the ICANN 45 DNSSEC Deployment Workshop, we learned that the great folks over at the DNSSEC Tools project had just released a new version of their great package of DNSSEC-related tools.  The new version 1.14 is available in several forms from:

Some of the changes include:

  • dnssec-nodes – many new features and graphing capabilities
  • libval – support for the TLSA recorded needed for the DANE protocol
  • dnssec-check – increased stability

As an advocate for the powerful capabilities of DANE, I’m particularly pleased to see that support added for TLSA records.

You can find out more information on the main web page.

I know from speaking with Sparta’s Russ Mundy at the ICANN 45 workshop that he and the others involved with the DNSSEC-Tools project are definitely looking for user feedback – and also looking to understand what other DNSSEC-related tools people might find useful.  Please do give this new release a try and let the team there know how it works for you.

Deploy360 Domain Name System Security Extensions (DNSSEC)

Code Examples: Checking the DNSSEC Status Of A Large Number of Domains

SIDN LabsDo you want to check the DNSSEC status of a large number of domains?  To know whether they are signed or unsigned? Or perhaps if any of the domains are failing validation?

Yesterday at the DNSSEC Deployment Workshop at ICANN 45 in Toronto I learned that the good folks at SIDN Labs in the Netherlands have created a service that allows you to do just that… and they are offering it for free public usage.

They provide two ways to use the service: 1) a web interface where you upload a file; or 2) a RESTful API you can query.  The web interface is in Dutch, but for non-Dutch-speakers it’s not hard to figure out (or translate via browsers):

You just upload a file and the service will give you back the results of whether the domains are secure, insecure or failing validation (‘bogus’).

What was more interesting to me, though, was the RESTful API allowing you to query the status of a domain by simply connecting to:

as in:

The comma-separated results that come back are:,"",secure,""

with the third field being either “secure”, “insecure” or “bogus”.

My immediate thought was how I could use this to create a simple little program to help me remember which of my domains I have signed and which ones I still need to sign.  After playing around with it for a few minutes in python, I decided that others might find my experiments useful or interesting, so I uploaded them to a Github repository at:

I included one very simple example that does no error checking and simply issues queries based on a list in the program.  I then added a second example that you could use from a command line to query for one or more domains:


(Omitting the ‘python’, of course, if you change ‘’ to be executable.)  An obvious extension would be to make the program accept the name of a file containing domain names.  You could also change it so that “bogus” entries come out on top or have big “Danger! Danger!” warnings of some type. I may make a web page that when I go to it shows me visually which of my domains are signed and which aren’t.  There’s a hundred other things you could do with it.  My purpose was just to try it out and see how the API worked.

Feel free to use those examples in whatever way you want… and thanks to SIDN Labs for making this service available for any of us to use!

Deploy360 IPv6 Tutorials

Video Tutorial: Using FTP over IPv6

The folks at RhinoSoft recently published a video on YouTube showing how their “Serv-U” FTP server and “FTP Voyager” FTP client all work with IPv6. While obviously focused on one vendor’s implementation, it provides an interesting view into how IPv6 can work with FTP. Kudos to the team at RhinoSoft for making this video available.

As with any reference we make to commercial products, we at the Internet Society Deploy360 Programme are not explicitly endorsing this product but rather providing a view of what this vendor is doing with FTP and IPv6. If we find other similar vendors providing services over IPv6 we are glad to consider posting about their videos, too. (And suggestions are always welcome.)

Deploy360 Domain Name System Security Extensions (DNSSEC)

How To Hack OpenSSH To Add DNSSEC Validation

OpenSSH logoWould you like to have SSH just automagically use DNSSEC to verify the authenticity of the SSH keys you are using to connect to another system?

Over on his blog, Jan-Piet Mens lays out the steps to add exactly this, demonstrating how to add ldns support into OpenSSH. Essentially all you need to do is recompile OpenSSH with the “--with-ldns” option.

To back up a moment and explain a bit more, RFC 4255 defines how to store SSH keys in DNS as SSHFP resource records. With DNSSEC signing all the resource records for a domain, you can now verify the authenticity of those SSH keys with the use of a DNSSEC-validating resolver. This provides a more secure alternative than requiring you to in theory confirm an RSA fingerprint when you are connecting to a server.

So for this all to work, you need to:

  1. Have SSH keys for the target machine stored in DNS as SSHFP resource records.
  2. Have the domain for the target machine signed with DNSSEC.
  3. Compile and install OpenSSH with the ldns option.
  4. Have access to a DNSSEC-validating DNS resolver. (Which could be accomplished by installing DNSSEC-Trigger, for instance, or using a DNSSEC-validating DNS resolver from your ISP if they offer one.)

Once you have done those steps, the beauty of the process is that you are no longer prompted with the message “The authenticity of host ‘____’ can’t be established” with the RSA key and the question about do you really want to connect.

Right now you have to recompile OpenSSH to add the ldns support, but hopefully as DNSSEC becomes increasingly deployed more widely this will just be one of the standard compilation options so that you’ll be able to just go to the command-line and type “ssh” and let it automatically do the DNSSEC validation.

Thanks, Jan-Piet, for writing up these steps!

Deploy360 Domain Name System Security Extensions (DNSSEC)

Warning! DNSSEC-Trigger Installation Issue After Mountain Lion Upgrade

Dnssec TriggerIf you are a Mac OS X user looking to upgrade to the brand new Mountain Lion release – and you also have installed DNSSEC-Trigger to have a local DNSSEC-validating DNS resolver, it seems there may be an issue during the upgrade process that you need to deal with.

[UPDATE: This issue apparently only affects new installations of DNSSEC-Trigger.  If you already have DNSSEC-Trigger installed, the upgrade to Mountain Lion works.  It is when you go to install DNSSEC-Trigger on Mountain Lion that the issue appears.]

Over on the dnssec-trigger mailing list, Olaf Kolkman of NLnet Labs writes about the problem with Mountain Lion and provides instructions for how to address the problem.  If you notice unbound not starting after  the Mountain Lion upgrade, you will need to follow Olaf’s instructions:

If the command
$ id unbound
returns “no such user”, you know that you have been bitten by this problem.

To fix:
Allocate yourself a free id. You can see the allocated ids using the following:
dscl localhost -list /Local/Default/Groups PrimaryGroupID
dscl localhost -list /Local/Default/Users UniqueID

Then assign the ids to the unbound user.
sudo dscl localhost -create /Local/Default/Users/unbound PrimaryGroupID
sudo dscl localhost -create /Local/Default/Users/unbound UniqueID

In his email message, Olaf also provides a “use-at-your-own-risk” shell script for performing this fix.  He also indicates that the DNSSEC-Trigger team will be including a fix in a new release sometime in the next few weeks.

Deploy360 IPv6

IPv6 Friday: Tutorial on How To Set Up An IPv6 Tunnel From SIXXS

How easy is it to “IPv6-enable” your network using a tunnel from SIXXS?  What are the steps you need to go through to do so?

Over on his “IPv6 Friday” site, Olle Johansson has a post up today about “Eating my own dog food: IPv6-enabling my training class room” where he walks through the steps involved with setting up a tunnel from on an Ubuntu server.  He goes on to explain how to then set up your local network to receive IPv6 addresses via router advertisements.

Kudos to Olle for continuing to publish great tutorials like this and I hope many of you will try this out as a way to get IPv6 connectivity into your current networks!