The past few years have seen significant improvements in online privacy and security through stronger encryption and clearer user security indicators. For the most part, consumers are in a better position than ever to make informed decisions about protecting their activity and preventing profiling by third parties.

Repeatedly, however, ISPs have demonstrated their willingness to use their privileged position to collect and sell information about user activity, oblige invasive authorities, and restrict services from their competitors. Network operators, governmental bodies, and advertisers can still easily ascertain which websites or applications a user is visiting and how frequently they choose to do so. User profiling, bandwidth management, traffic inspection, and censorship are rife.

While there are reliable methods of bypassing these issues using a VPN or Tor, the burden of action and understanding is on the user, only a small minority of whom can afford to pay the extra monetary, performance, and setup costs required to do so.

In a nutshell, our online activity is exposed and exploited by default thanks to systemic gaps in current internet standards. Network operators can enforce a number of restrictive or intrusive policies leveraging this gap. More than ever, consumers need technology fixes to limit the visibility of their web activity by default.

Improvements to encryption standards such as TLS 1.3 have removed some vulnerabilities that stemmed from outdated ciphers and algorithms, but fundamental gaps still exist. Data that is transmitted in the process of locating a server (DNS lookup) and forming a connection with it (TLS with SNI) can still leak, immediately exposing the websites you’re visiting.

This poses a foundational privacy problem that is especially concerning for vulnerable populations under authoritarian rule. It is also a common window through which technology providers in ‘free’ societies profile and monitor traffic to facilitate censorship, bandwidth discrimination, and surveillance.

Two standards have gained momentum to tackle these weaknesses in TLS and DNS. These proposals are termed Encrypted SNI (ESNI) and DNS over HTTPS (DoH).

Over the past 12 months, leading technology providers including Cloudflare, Google, and Mozilla have revealed their intentions to push forward these new standards in an attempt to close the privacy gaps that expose network requests to potential scrutiny. So far, few people have noted or even realized the potential these new technologies have to utterly shift the way network-wide policies are defined and enforced.

Combined, these specifications have the potential to make our online activity more opaque and illegible by default. At a very basic level, ISPs, public WiFi providers, and anyone else privy to your network packets could be prevented from snooping on your web accesses. This is a game-changer in the dynamic between operators and application providers, and potentially a huge win for online privacy and anti-censorship at large.

What is DNS over HTTPS?

Domain Name Resolution refers to the process through which an easily readable internet domain (such as example.com) is matched to the underlying IP address that machines can understand. It operates every time you enter a URL into your web browser, and usually happens so quickly that nobody notices it.

Unfortunately, these DNS queries are currently sent out in plaintext, which means they can be tracked, read, and altered by Man in the Middle (MitM) attacks. In addition, every website you visit can be logged by your ISP or VPN provider, through which they can be gathered and analysed by advertisers and governments.

DNS over HTTPS (DoH) is a proposed protocol for performing this remote DNS resolution via the HTTPS protocol. The technology aims to improve privacy and security by preventing the observation and amendment of DNS data. Google and the Mozilla Foundation have been testing versions of DoH since March 2018.

The concept is simple: rather than sending DNS requests and responses out unencrypted, they’ll be sent wrapped in an HTTPS GET or POST request. Given HTTPS is encrypted and authenticated using TLS, it will be much more difficult for an attacker to view or modify them.

DoH is considered to be in development, but is still somewhat available: Firefox’s Nightly build already includes DoH functionality using Cloudflare’s 1.1.1.1 DNS server. This provides individuals and organizations the opportunity to test out the technology before it goes mainstream.

What is ESNI?

Originally standardized in 2003, the TLS Server Name indication (SNI) extension allows a server to host multiple HTTPS websites on the same IP address. The SNI extension requires clients to specify which website they want to connect to during the initial TLS handshake so that a server hosting multiple domains can send back the certificate for the correct domain.

Even if your DNS lookups are protected by DoH, the domains you’re connecting to can still leak thanks to SNI, because the extension transmits the domain name in plaintext. Unless a VPN is being used, ISPs can see this.

As the name might suggest, ESNI is a version of the SNI extension that encrypts this gap through the exchange of symmetric keys. Once exchanged, the DNS server and client are the only entities capable of decrypting the extension and viewing the domain name. This can only work with TLS 1.3 and above.

Cloudflare’s announcement explains this process in depth. The main takeaway, however, is that ESNI prevents third parties like ISPs, network administrators, and firewalls from intercepting the TLS SNI extension and using it to see which websites and applications users are visiting.

Why are DoH and ESNI important?

Widespread adoption of these technologies has the potential to fundamentally change the ‘default privacy settings’ of the internet. It will disrupt the work of ISPs, affect technology providers, and make it much harder to censor content and surveil users en masse.

In terms of privacy, both DoH and ESNI have the capacity to heavily impede current bulk surveillance standards. In the UK for example, the 2016 Investigatory Powers Act forces ISPs to store a record of the websites citizens visit for 12 months, which is done by collecting DNS requests. ISPs elsewhere often monitor DNS queries to sell to advertisers or to facilitate censorship. Under DoH, this data will be encrypted and essentially worthless.

It’s important to note, however, that while using DoH will hide lookups from both your ISP and VPN provider, it will not hide them from your DNS provider, who may pass them on to authorities if obligated to do so.

Of course, it’s also possible to defeat ISP surveillance and censorship by using a VPN (Virtual Private Network) to encrypt your traffic. This will secure your connection, but not without some limitations. Your VPN provider can still see your DNS queries, and may pass this data on to authorities if they participate in logging and are asked to do so. Poor quality VPNs can also leak DNS data if developed or configured improperly.

In addition to these privacy benefits, both ESNI and DoH have the capacity to significantly limit state censorship capabilities.

DNS blocking is one of the most granular tools used by ISPs to implement governmental and regulatory blocking orders. Presently, the majority of connected devices use the default ISP DNS capabilities set on their operating system. By contrast, early adoption of DoH is likely to be driven through centralised third-party cloud providers like Cloudflare, Google, or Mozilla.

Changing to a third party DoH provider will remove ISPs from the DNS path and therefore bypass any existing domain-specific blocks. Governments looking to censor specific websites may instead be forced to approach the DoH providers themselves, who may be based out of the inquiring government’s jurisdiction.

In addition, these protocols can all operate on port 443 – the standard upon which most of the web runs. If DoH and ESNI are adopted across entire cloud providers like Google, Cloudflare, and Amazon using port 443, repressive authorities will be unable to block specific websites without blocking these platforms entirely, or half of the web with it. This is known as collateral freedom.

The future of online privacy?

Of course, with such broad potential for disruption, these developments have not been without opposition. International authorities including Britain’s National Cyber Security Centre (NCSC) have expressed concerns that the unexpectedly rapid deployment of ESNI and DoH may imperil the monitoring of terrorism and other illegal content online.

The issue has piqued in recent months as Google announced the implementation of DoH as part of its public DNS system (8.8.8.8/8.8.4.4). This will soon be supported in Chrome and is already available in Android 9. The mere fact that it is in Android at all is a powerful signal to the industry that DNS privacy now has a powerful backer.

Not everyone is happy with these developments, not least because they place a great deal of trust in providers like Google and Cloudflare, who may find themselves playing an even more critical role in the online ecosystem. This is rightly concerning for those who feel the tech giants are already ‘too big to fail’.

Despite these concerns, it seems the widespread adoption of these standards is simply a matter of time. Though it won’t happen overnight, we may witness a transition from a world where everyone can implicitly infer your online activity to one where context is securely exchanged between trusted parties.

For this approach to replace today’s status quo it will need wide support in popular browsers and mobile network stacks. Privacy-minded users must press for their adoption in as many places as possible and educate broader communities on the fundamental security risks that necessitate their development. To this end, courses like the EC-Council Certified Ethical Hacker Certification are available online to help consumers gain an in-depth understanding of online attack vectors and their respective countermeasures.

Google undoubtedly changed the world with its drive for HTTPS. We need to do the same with other protocols and officially plug the gaps that linger in our current model. The 20th Century internet must be switched off and replaced with one that truly respects the rights of our citizens to privacy and online freedom.