VPN Encryption, Protocols & Ciphers: The Ultimate Guide for Beginners

JP Jones - CTO @ Top10VPN

JP is our CTO and with over 25 years of software engineering and networking experience oversees all technical aspects of our VPN testing process. Read full bio

VPN encryption is the very heart of VPN technology. In this guide we explain how VPN encryption works, what the main VPN protocols are, and how they keep you protected.

illustration of a man putting a message into a secure network of pipes, protecting it from the unprotected data outside of the pipes

Encryption is the process of encoding data so that only authorized individuals are able to access it. When you encrypt a message you convert it from normal, readable text into an unintelligible code.

Virtual Private Network (VPN) services use encryption to create a secure connection between your device and a VPN server. This connection is known as the VPN tunnel and it helps to keep your internet data hidden from ISPs, hackers, and any other snooping third parties.

Every VPN uses a particular set of rules to establish this secure tunnel. This set of rules is called the VPN protocol. Some VPN protocols are more csecure than others, and some won’t protect your data at all.

A VPN is only as secure as the encryption it uses to protect your data.

In this guide you’ll learn how VPN encryption works, which VPN protocols are best, and all about commonly-used ciphers.

We’ll break it down into plain English and explain all of the jargon in simple terms so that you can get a better understanding of VPN encryption and all that it entails.

Our Encryption Glossary is a useful resource to help wade through the technical language, and we’ll link to it regularly throughout the guide.

What Is VPN Encryption?

Encryption is the process of converting data into an unintelligible code so that unwanted parties cannot access it.

VPN services use encryption to secure your data as it travels between the VPN software on your device and the VPN server you’re connecting to.

When you connect to a VPN, your traffic gets encrypted before it is sent to the VPN server. Once it gets to the server it is decrypted and forwarded onto the internet.

Diagram showing how a VPN works.

Your data travels through a secure tunnel which prevents internet service providers (ISPs), governments, and other third parties from spying on your internet activity.

VPN encryption protocols are the set of rules that determine how this secure tunnel gets formed. Depending on the protocol in use, your VPN might have different speeds, capabilities, or even vulnerabilities. We’ll look at the pros and cons of the most common VPN protocols in the next chapter.

Every VPN protocol uses an algorithm to actually encrypt (and decrypt) the data. This algorithm is known as a cipher. An example of a simple cipher might be substituting every letter in a message with the letter that precedes it in the alphabet. So, privacy would become oqhuzbx.

In short:

  • VPN protocols are used to create a secure tunnel between your device and a remote server.
  • Ciphers are used to encrypt the data that flows through the tunnel.

Ciphers usually operate in conjunction with a piece of information known as a key. It is extremely difficult to decrypt a given set of data without knowledge of the key that was used to encrypt it.

When reviewing a VPN provider’s encryption, we generally refer to a mixture of cipher type and key length, which refers to the number of bits in a given key.

Blowfish-128, for example, is the Blowfish cipher with a key length of 128 bits.

Generally speaking, longer keys are more secure. A short key length often means poor security as it is more susceptible to brute-force attacks.

The current gold standard for online encryption protocols is AES-256. AES is the cipher, and 256 is the key-length.

256-bit keys cannot be brute-forced because it would take billions of years to run through all of the possible combinations using even the most sophisticated modern-day computers.

Our chapter on ciphers will cover the most common ciphers on the market and evaluate their advantages and disadvantages.

What Types of VPN Protocol Are There?

VPN protocols are the instructions a VPN client follows in order to establish a secure connection between a device and a chosen VPN server.

At their core, VPN protocols are basically a combination of encryption standards and transmission protocols. They ensure your data is safe both at rest and in transit.

Most VPN providers offer access to the following VPN protocols:

  1. OpenVPN TCP
  2. OpenVPN UDP
  3. PPTP
  4. L2TP/IPsec
  5. SSTP
  6. SoftEther
  7. IKEv2/IPsec
  8. WireGuard

Each of these protocols come with their own strengths and weaknesses. A good VPN provider will allow users to select exactly which protocol they want to use when connecting to a server.

In this section, we’ll cover the seven main VPN protocols that you should be aware of. Their strengths and weaknesses are briefly summarized in the table below:

1. OpenVPN: Very Secure & Fast

screenshot image of the OpenVPN protocol's logo

  • The most secure VPN protocol currently on the market
  • Available as OpenVPN TCP or OpenVPN UDP
  • Fast and open-source

OpenVPN is the safest and most secure VPN protocol available.

Importantly, OpenVPN is open-source, meaning it is completely transparent and continues to be improved through public testing.

OpenVPN is also highly-configurable. While it is not supported natively by any platform, most VPN providers offer free apps that support OpenVPN. These apps are available across most major platforms including Microsoft Windows, Apple MacOS, Android, Linux, and iOS.

Some providers even offer OpenVPN configuration files, meaning you can visit the OpenVPN website, download the original OpenVPN client, and use it to connect to your chosen VPN service.

OpenVPN is the industry’s leading VPN protocol and we recommend you use it whenever you can.

OpenVPN TCP vs. OpenVPN UDP

OpenVPN can work on two different types of communication protocol: TCP and UDP. These determine how your data gets sent across the connection.

Data is typically transferred across the internet in chunks, known as packets. Transmission Control Protocol (TCP) is designed to ensure that packets get delivered to the OpenVPN client in the same order they were sent from the OpenVPN server.

To achieve this, TCP:

  • Delays delivering packets to the OpenVPN client until it has received all of the expected packets.
  • Re-arranges any out-of-order packets back into place.
  • Re-requests and waits for any packets that may have been lost in transmission.

User Datagram Protocol (UDP), on the other hand, focuses on delivering data to the VPN client as efficiently as possible.

UDP is designed to:

  • Transmit data packets without requiring a confirmation of arrival.
  • Send data in smaller packet sizes than TCP.

The smaller packet size and lack of organizational checks means OpenVPN UDP has faster connection speeds than OpenVPN TCP, where the additional processing steps add latency to the connection.

UDP is faster than TCP so is a better option if you’re using a VPN to game, stream, or use VoIP services.

OpenVPN TCP offers a more reliable connection. When using UDP, you may experience some lost packets — for example, on a VoIP call you might hear the voice of the person you’re talking to cut out for a fraction of a second. This isn’t an issue for TCP, which is the most used connection protocol on the internet.

OpenVPN TCP is also very useful for bypassing censorship because you can use TCP port 443, the default port for HTTPS traffic.

Strict censorship regimes are unlikely to block TCP port 443 as it would wreak havoc for banks, online retailers, and other security-sensitive online resources.

TCP UDP
Connection Speed Loser ✗ Winner ✓
Connection Reliability Winner ✓ Loser ✗
Bypassing Censorship Winner ✓ Loser ✗

How Does OpenVPN Encryption Work?

There are several elements that make up OpenVPN encryption: server authentication, the data channel, the control channel, and HMAC authentication:

  • Server authentication verifies that the server you are talking to is trustworthy. OpenVPN authenticates servers in the same way as HTTPS, using TLS certificates.
  • The control channel secures the connection between your computer and the VPN server. It is used at the start of the connection to agree the rules for securely sending data and authenticating the client to the server. This protects your data as it travels. OpenVPN uses TLS encryption for the control channel.
  • The data channel is used to secure the data itself. This layer is encrypted using a cipher and a symmetric encryption scheme, the key for which is obtained via the control channel.
  • HMAC authentication is used to ensure that packets being sent have not been altered in transit by an attacker with the ability to read or alter the data in real time.

It’s important to be aware that some VPN services don’t use the same encryption strength on both the data and control channel. Using weaker encryption on the data channel can be a cheap shortcut to faster connections speeds, as it uses less processing power.

Unfortunately, a VPN is only as secure as its weakest element, so you should look for a VPN that’s as strong as possible in its encryption of both channels.

We’ll go into more detail about this in the sections below on ciphers and handshakes.

2. PPTP: Outdated & Weak Security

  • Very fast, but has severe security vulnerabilities
  • Best avoided unless absolutely necessary

The Point-to-Point Tunneling Protocol (PPTP) is one of the oldest VPN protocols in use today. It was developed by a Microsoft-funded team and published in 1999.

PPTP used to be the standard for corporate VPN networks, and still retains some popularity today. There are three main reasons:

  1. PPTP is faster than most other protocols.
  2. PPTP is natively supported on almost all platform devices.
  3. PPTP is easy to set-up.

That said, as a privacy and security technology, PPTP should be considered obsolete.

The fast connection speeds that PPTP boasts are largely due to its use of 128-bit encryption keys. Secure alternatives like OpenVPN use 256-bit encryption, which is much safer.

PPTP trades off security for speed in a way that leaves it with serious vulnerabilities. A skilled attacker can compromise a PPTP-encrypted VPN connection in a matter of minutes.

It is widely believed that the NSA has collected huge amounts of data by exploiting the insecurities of PPTP.

PPTP is also very easy to block with censorship systems like the Great Firewall of China. It relies on the GRE protocol and uses the TCP port 1723, both of which are easily firewalled.

You should avoid using PPTP unless it is absolutely necessary. If unavoidable, it is crucial you only use it for non-sensitive data.

3. L2TP/IPsec: Generally Secure, Can Be Slow

  • Uses double encapsulation which increases security and reduces speed
  • There is speculation that IPsec has been compromised by the NSA

Layer 2 Tunneling Protocol (LT2P) takes the best features of Microsoft’s PPTP and Cisco’s L2F to create a network tunnel between your device and your chosen server.

L2TP can handle authentication, but does not provide any encryption capabilities.

L2TP on its own provides no encryption, so it is typically combined with Internet Protocol Security (IPsec) — a suite of several security protocols that work to authenticate and encrypt data between two devices. This combination is commonly known as L2TP/IPsec.

When connecting to a VPN server with L2TP/IPsec, IPsec is used to create a secure control channel between client and server, typically using the AES cipher.

Packets of data from your web browser are encapsulated by L2TP. IPsec then encrypts this L2TP data and sends it to the server, which in turn decrypts and decapsulates the data.

While this double encapsulation makes L2TP/IPsec more secure, it can also slow it down.

L2TP/IPsec is normally easy to configure, and it’s already built into many existing platforms so you’ll see many VPN providers offering access to it.

Is LT2P/IPsec Secure?

Leading security experts including Edward Snowden have suggested that IPsec was deliberately weakened during its creation. Leaked documents have also revealed that similar VPN connections may have already been compromised by the NSA, though this has not been confirmed.

Outside of this speculation, security researchers have been able to use vulnerabilities within the protocol to break IPsec VPN connections.

The main security issue with L2TP/IPsec lies with VPN services that use pre-shared keys that are available for download online.

While these keys are only used to authenticate client-server connections, they open the door to potential man-in-the-middle attacks. Using this method, an attacker could impersonate a VPN server in order to decrypt the traffic and eavesdrop on the connection.

L2TP/IPsec also uses a limited number of fixed ports, which makes it relatively easy to block.

For those willing to overlook these issues, LT2P/IPsec is a good choice for non-sensitive use given that it is supported natively by many platforms — just be sure to avoid pre-shared keys.

4. SSTP: Closed Source with Potential Risks

  • Closed-source with worries about Microsoft’s commitment to user privacy
  • Good for bypassing firewalls and unblocking websites

Secure Socket Tunneling Protocol (SSTP) is a Microsoft-owned proprietary protocol that’s based on the SSL 3.0 encryption standard. This means that, like OpenVPN, it can use TCP port 443 — the port that is rarely blocked by firewalls.

SSTP is not open-source. It is therefore impossible to disprove suggestions that Microsoft have built backdoors or other vulnerabilities into the protocol.

In our opinion, the risk of SSTP’s close integration with Windows far outweighs any benefits, especially given Microsoft’s past cooperations with the NSA.

Another red flag is that SSL 3.0 is vulnerable to a particular man-in-the-middle attack known as POODLE. It has not been confirmed whether SSTP is also affected by this vulnerability, but in our view it’s not worth the risk.

5. SoftEther: A Promising Newcomer

screenshot image of SoftEther protocol's logo

  • Secure, fast, and good at bypassing censorship
  • Not supported by many VPN providers
  • Requires manual configuration

SoftEther is an open-source VPN protocol, initially developed as part of a Master’s thesis at the University of Tsukuba.

Having been released in 2014, SoftEther is one of the newer VPN protocols. Despite this, it is known for having good security without compromising on speed.

SoftEther supports strong encryption ciphers like AES-256 and RSA-4096, but also boasts speeds that are reportedly 13x faster than OpenVPN.

It is also well-designed to bypass censorship regimes. SoftEther bases its encryption and authentication protocols on OpenSSL, enabling the traffic it tunnels to be almost indistinguishable from HTTPS traffic. This means it uses TCP Port 433, which is very difficult for firewalls to block.

In 2018, SoftEther received an 80-hour security audit which revealed 11 security vulnerabilities. These were patched in a subsequent update, however researchers at Aalto University have recently found that SoftEther is sometimes vulnerable to man-in-the-middle attacks. This is because the default configuration is for client’s not to verify the server’s certificate. Attackers can therefore impersonate a VPN server and gain access to user credentials and online activity.

When using SoftEther, be sure to tick the Always Verify Server Certificate box in the New VPN Connection settings.

screenshot of the server certificate verification options on the SoftEther new VPN connection settings

SoftEther’s default settings do not include server certificate verification

SoftEther is not supported natively on any operating system and very few VPN providers support its use. Of those we’ve tested, only Hide.me and CactusVPN support the SoftEther protocol.

In short, SoftEther is a very fast and reasonably secure protocol. It is particularly good for bypassing censorship, but users should be wary of its default configuration settings and lack of mainstream VPN compatibility.

6. IKEv2/IPsec: Secure, Stable & Very Fast

  • Very fast and great for mobile use
  • There is speculation that IPsec has been compromised by the NSA

Internet Key Exchange version 2 (IKEv2) is another closed-source VPN protocol developed in a collaboration between Microsoft and Cisco.

IKEv2 is natively supported by iOS, BlackBerry, and Windows versions 7 and later.

As we discussed with SSTP, closed-source protocols like IKEv2 should be treated with skepticism, as there is no way to verify they have not been compromised.

Fortunately for IKEv2/IPsec users, there are open-source versions of IKEv2 developed for Linux which don’t carry the same trust issues as the proprietary version.

As with L2TP, IKEv2 is combined with IPsec when used as part of a VPN solution. For this reason, speculation about the security of IPsec — Snowden revelations included — should be taken into account when choosing this VPN protocol.

That said, IKEv2/IPsec offers better functionality and greater performance when compared to L2TP:

  • IKEv2/IPsec can seamlessly handle the changing of networks through something called the MOBIKE protocol. This makes it great for mobile users who regularly switch between mobile data and WiFi.
  • IKEv2/IPsec is faster than L2TP/IPsec as it is programmed to make better use of bandwidth.
  • IKEv2/IPsec supports a wider range of encryption ciphers than L2TP/IPsec, including AES, Blowfish and Camellia.

IKEv2 won’t allow you to bypass strict censorship though. The protocol uses specified ports that are very easy for censorship systems to block.

Overall, given its connection flexibility, speed, and relative security, IKEv2/IPsec is a good option — especially for mobile users.

7. WireGuard: The Future of VPN Protocols?

screenshot image of WireGuard protocol's logo

  • Still in testing, but has the potential to be the best VPN protocol available
  • Open-source, fast, and has a small codebase

WireGuard is a new, open-source tunneling protocol that aims to be faster and more efficient than the current most popular VPN protocol: OpenVPN.

While it’s still early days in WireGuard’s development, it was recently announced that WireGuard will be included in the Linux kernel at its next release. Some are predicting this to be a turning-point for VPN technology.

WireGuard aims to tackle the issues often associated with OpenVPN and IPsec. These include complicated setup, disconnections, and long reconnection times. Its codebase stands at under 4,000 lines in size — 100x smaller than counterparts like OpenVPN and L2TP/IPsec. This makes finding and fixing bugs much easier.

For encryption, WireGuard uses the ChaCha20 cipher which utilizes a 256-bit key.

While ChaCha20 is not common amongst VPN protocols, it has the potential to challenge AES as the industry gold-standard. It is up to three times faster and appears to be equally secure.

With both ChaCha20, and WireGuard more generally, immaturity is the main factor working against it — it has not been around long enough to be publicly tested.

Benchmarks show that WireGuard is already very fast, and we should expect these speeds to increase further with its inclusion in the kernel.

It is exciting times for WireGuard. If it continues on its current trajectory, we should expect to see it challenging OpenVPN as the best available VPN protocol very soon.

Which VPNs Support WireGuard?

In April 2020, NordVPN announced the release of its new custom protocol, NordLynx — available across all platforms. The protocol is built entirely around WireGuard, and is reported to provide users with connection speeds that are up to twice as fast as before.

NordVPN chose to create a custom protocol, combining WireGuard with their proprietary double NAT (Network Address Translation) system. This was because WireGuard is still in development and there remain worries about its ability to provide reliable privacy and security to VPN users.

Despite this, an increasing number of VPN providers now support WireGuard in one way or another:

We expect this number to continue to rise as WireGuard achieves mainstream acceptance.

Screenshot of a iVPN advertisement explaining that you can use WireGuard with iVPN

It should also be noted that ExpressVPN has recently announced the development of its own proprietary protocol Lightway. While Lightway isn’t based on WireGuard, it is reported to offer many of the same benefits (i.e. faster speeds and “fewer lines of code than other major protocols”).

How Do I Change VPN Protocol?

Some VPN services, like ExpressVPN, allow you to change the VPN protocol within the VPN app’s settings menu.

If this is the case, simply open up the settings menu and select the VPN protocol you wish to use.

screenshot of the protocol settings menu on ExpressVPN's desktop app

ExpressVPN allows users to select which protocol to use

If there is no option to select protocol within the custom app, you may be able to install alternative protocols using manual configuration.

NordVPN is one example of a VPN service that runs on OpenVPN but allows for manual installation of IKEv2.

If your VPN service supports alternative protocol configuration be sure to carefully follow the instructions given on its website.

Remember that even a VPN using the most secure protocols and ciphers on the market may put your personal data at risk by logging your online activities.

You should look for a VPN with a privacy-friendly logging policy, as well as one that supports the safest VPN protocols and ciphers.

To make things easier for you, why not take a look at our highest recommended VPNs – they are the safest ones around.

In the next section, we’ll look at the encryption ciphers that VPN protocols use to actually encrypt data.

VPN Encryption Ciphers

A cipher is an algorithm for securing data on both control and data channels. Often ciphers are spoken about in conjunction with a key length (e.g. AES-256).

While VPN protocols lay the framework for a secure encryption tunnel to be generated, ciphers are what actually encrypt your data.

In practice, the primary ciphers used by commercial VPN providers are AES, Blowfish, and Camellia.

1Advanced Encryption Standard (AES)

The Advanced Encryption Standard (AES) is considered one of the safest ciphers to use. It is the gold standard for online encryption protocols, and is used commonly in the VPN industry.

AES was established by the US National Institute of Standards and Technology (NIST) in 2001 and is occasionally also known as the Rijndael algorithm. Its large block size of 128 bits means it can handle larger files than other ciphers with smaller block sizes, like Blowfish.

AES is usually available in 128-bit and 256-bit key lengths. While AES-128 is considered secure, our knowledge of the NSA’s efforts to undermine encryption means AES-256 is preferred as it’s likely to offer much greater protection.

When you read about ‘military-grade’ or ‘bank-grade’ encryption on a VPN provider’s website, it generally refers to the use of AES-256. The US government even uses AES-256 encryption to secure its own sensitive data.

2Blowfish

Blowfish is a cipher designed by American cryptographer Bruce Schneier in 1993.

Blowfish used to be the default cipher used in OpenVPN, but has now been largely replaced by AES-256.

You’ll typically see Blowfish used with a 128-bit key length, although it can range from 32 bits to 448 bits.

There are some weaknesses with Blowfish. Most well-known is its vulnerability to a cryptographic attack known as a ‘birthday attack’. For this reason, Blowfish should only be used as a fallback to AES-256.

3Camellia

Camellia is a cipher that is very similar to AES in terms of security and speed.

Camellia is considered to be a safe cipher. Even using the smaller key length option (128 bits), it is thought to be infeasible to break with a brute-force attack given current technology. There are no known successful attacks that effectively weaken the Camellia cipher.

The main difference between Camellia and AES is that it is not certified by NIST, the US organization that created AES.

While there is certainly an argument for using a cipher that is not associated with the US government, Camellia is rarely available in VPN software, nor has it been as thoroughly tested as AES.

Symmetric & Asymmetric Encryption

So far, we’ve looked at the protocols and ciphers involved in VPN encryption. In this section, we’ll take a step back to consider encryption more generally.

There are two main types of encryption: symmetric and asymmetric.

Symmetric Encryption

diagram showing the same symmetric key encrypting plaintext into ciphertext and then decrypting it back into plaintextSymmetric encryption (or symmetric-key encryption) is where only one key is used to handle both the encryption and the decryption of data.

With symmetric encryption, both parties require the same key to communicate.

VPN technology uses symmetric encryption: both the VPN server and VPN client use the same encryption key to encrypt and decrypt your connection.

AES, Blowfish, and Camellia are all symmetric-key ciphers.

Asymmetric Encryption

diagram showing a public key encrypting plaintext into ciphertext and then a different private key decrypting it back into plaintextWith asymmetric encryption (or public-key cryptography), individuals looking to communicate securely use software to generate both a public key and a corresponding private key.

When Person A wants to send a secure message to Person B, they’ll use Person B’s public encryption key to convert a plain text message into an encrypted message.

While the encrypted message might travel from A to B through various other parties, anyone who tries to read the message will only see the encrypted text.

When Person B receives the encrypted message, they use their private key to decode it back into plain text.

This system is often used by journalists who publish their public key on their social media profile so that sources can send them secure messages.

The most well-known of these systems is Pretty Good Privacy (PGP), though lots of different tools use its open source variant, OpenPGP.

Symmetric vs Asymmetric Encryption

  • Symmetric-key encryption is faster than asymmetric encryption as it only requires the generation of one key and is therefore much simpler mathematically.
  • Asymmetric encryption tends to be more secure than symmetric encryption because there is not the requirement to share just a single key.

VPN encryption actually takes a hybrid approach. It uses asymmetric encryption to create a secure VPN client-server connection through which symmetric keys can then be safely exchanged.

The next chapter will explore this hybrid approach in greater detail, looking at VPN Handshakes. We’ll also introduce Hash Authentication and Perfect Forward Secrecy, two encryption features that are important for VPN security.

VPN Handshake & Hash Authentication

In addition to protocols and ciphers, VPNs also use processes known as handshakes and hash authentications to further secure and authenticate your connection.

A handshake refers to the initial connection between two computers. It’s a greeting in which both parties authenticate one another and the rules for communication are established.

During a VPN Handshake, asymmetric encryption is used to form a secure connection between VPN client and server. Once the connection has been secured, a symmetric key is shared that is used for the remainder of the session.

The VPN handshake secures the connection and creates the VPN tunnel. Symmetric-key ciphers then secure your actual data for its journey through the tunnel.

diagram showing the step-by-step process of a VPN handshake between VPN client and VPN serverVPN handshakes tend to use the RSA (Rivest-Shamir-Adleman) algorithm. RSA has been the basis for internet security for the last two decades.

While there is not yet hard evidence of RSA-1024 being cracked, it is generally considered a security risk given the processing power available today.

RSA-2048 is a far more secure alternative and comes with relatively little computational slowdown. As such, most VPN services have moved away from using RSA-1024.

You should only trust VPN services that use RSA-2048 or RSA-4096.

Although this process works well and generates secure encryption, every session that is generated remains decryptable with the private key used in the RSA handshake. In this sense, it is a sort of ‘master key’.

If the master key were ever to be compromised, it could be used to decrypt every secure session on that VPN server, past or present. An attacker could then gain access to the symmetric key and decrypt all data flowing through the VPN tunnel.

To avoid that, we recommend using VPN services that are set-up with Perfect Forward Secrecy.

Perfect Forward Secrecy

illustration of a VPN client and VPN server in separate rooms both generate the same temporary key to encrypt their sessionPerfect Forward Secrecy is a protocol feature which utilizes an algorithm known as the Diffie-Hellman (DH) or Elliptic Curve Diffie-Hellman (ECDH) key exchange in order to generate temporary, symmetric, session keys.

Perfect Forward Secrecy ensures that the symmetric key is never exchanged across the connection.

Instead, both the VPN server and the VPN client independently generate the key themselves using the DH or ECDH algorithm.

It is a mathematically complex process, but Perfect Forward Secrecy essentially removes the threat of there being a single private key that, if compromised, exposes every secure session ever hosted on the server.

Instead, because the keys are temporary, if they are hacked it only reveals one specific session, and nothing more.

It should be noted that RSA alone cannot provide Perfect Forward Secrecy. DH or ECDH must be included in RSA’s cipher suite for it to be implemented.

ECDH can actually be used on its own – instead of RSA – to generate a secure VPN handshake with Perfect Forward Secrecy. However, be wary of VPN services using DH alone, as it is vulnerable to being cracked. This is not an issue when used with RSA.

Our recommended VPN protocol, OpenVPN, is set up with Perfect Forward Secrecy.

SHA Hash Authentication

Secure Hash Algorithms (SHA) are used to authenticate the integrity of transmitted data and client-server connections. They ensure that information has not been altered in transit between source and destination.

SHAs work by editing source data using what is known as a hash function. The original source message is run through an algorithm and the result is a fixed-length string of characters that looks nothing like the original. This is known as the “hash value”.

It is a one way function – you cannot run a de-hash process to determine the original message from the hash value.

Hashing is useful because changing just one character of the input source data will totally change the hash value that is output from the hash function.

A VPN client will run the data received from the server, combined with the secret key, through a hash function agreed during the VPN handshake.

If the hash value the client generates differs from the hash value in the message, the data will be discarded as the message has been tampered with.

SHA hash authentication prevents man-in-the-middle attacks as it is able to detect any tampering with a valid certificate.

Without it a hacker could impersonate a legitimate VPN server and trick you into connecting to an unsafe one, where your activity could be monitored.

To ensure maximum security, we recommend using VPN services, like ExpressVPN, that utilize SHA-2 or higher. SHA-1 has proven weaknesses that can compromise security.

HTTPS Encryption: SSL & TLS

While not specifically VPN encryption, it’s worth quickly mentioning the SSL and TLS protocols. These form the encryption that most of us have experienced when visiting a secure website which uses HTTPS.

Hyper Text Transfer Protocol Secure (or HTTPS) is a secure version of the original HTTP protocol that formed the web’s foundation.

You’ll know when a website is using HTTPS by the padlock symbol in the browser’s address bar.

Screenshot of an HTTPS address bar.
HTTPS uses the security protocol TLS (Transport Layer Security), which is based on its predecessor Secure Sockets Layer (SSL Version 3.0).

TLS and SSL are also widely used within VPN protocols for authentication and the creation of secure VPN tunnels.

OpenVPN and SoftEther, for example, rely on the OpenSSL library for securing and authenticating connections. OpenSSL is a collection of open-source implementations of the SSL/TLS protocols, including the AES, Blowfish, and Camellia ciphers.

The handshake used in these VPN protocols is also known as a TLS Handshake. In it, TLS certificates are shared between device and server in order to verify one another’s identity and protect against man-in-the-middle attacks. HTTPS sessions are initiated in the same way.

VPNs based on SSL/TLS encryption differ from those based on IPsec, such as L2TP and IKEv2. There are pros and cons to using either, but SSL-based protocols have a particular advantage when it comes to bypassing censorship. This is because the traffic blends in with HTTPS traffic, which governments are typically reluctant to block.

What Is the Best VPN Protocol?

OpenVPN used with AES-256 cipher is generally considered the best and safest VPN protocol. OpenVPN is open-source and has been publicly tested for weaknesses. It provides a great balance between privacy and performance, and is compatible with many popular platforms. Most commercial VPN services use OpenVPN as default.

Wireguard is a new contender that promises to challenge OpenVPN for the top spot in the very near future. We’ll be monitoring its forthcoming release within the Linux kernel with keen interest.

Another good protocol is IKEv2. It’s an especially good option for mobile devices as it handles network changes very well, automatically restoring dropped connections with ease and speed.

For maximum security, you should also ensure that the VPN encryption you choose:

  • Uses reliable key exchange protocols, either ECDH or RSA-2048.
  • Implements Perfect Forward Secrecy.
  • Uses SHA-2 (or higher) for HMAC Authentication.

Encryption Glossary

Term Definition

Algorithm

A set of instructions that need to be followed (by software) in order to solve a problem. A cipher is an algorithm for performing encryption or decryption.
Authentication A process to ensure and confirm a user’s identity.

Bit

Short for ‘binary digit’, it’s the smallest unit of data in a computer.
Block A group of bits of fixed length (e.g. 64 bits)

Block Size

The maximum length of a group of bits.

Brute-Force Attack

A method to try and guess a secret by trying every possible combination of characters. It’s a crude, exhaustive effort rather than a more complicated or strategic one. Imagine you can’t open the padlock on your locker – rather than trying to remember the code, by brute forcing it you simply try every combination from 000 through to 999 until it opens up.
Cipher An algorithm for performing encryption and decryption.

Cryptography

The art of writing and solving codes to create secure messaging, designed so that only the intended parties can process and read it.
Encryption The process of converting data into a code in order to prevent unauthorized access to it.

Firewall

Software which monitors and controls packets of data incoming or outgoing from a network. You’ll likely be familiar with the one on your desktop or laptop computer, which is designed to watch out for hackers and viruses.

Handshake

A process that initializes the connection between two computers – like its name suggests, it’s a greeting that establishes the rules for communication.

HTTPS

It stands for ‘Hypertext Transfer Protocol Secure’, but no one ever calls it that. It’s a secure version of HTTP, the protocol that is the foundation of the web, and is used to ensure communication between devices and websites stays authenticated, private, and secure.

IPsec

Short for Internet Protocol Security, IPsec is a suite of security protocols used to authenticate and encrypt the data that travels between two computers. In VPN protocols, it is commonly combined with L2TP or IKEv2. It has been suggested, by a number of security experts including Snowden, that IPsec was deliberately weakened during its creation and has therefore been compromised by the NSA.

Kernel

The core of an operating system. In a computer it controls the operation of the computer and its hardware.

Key

A string of bits used by a cipher to convert ‘plain text’ (unencrypted information) into ciphertext (encrypted information), and vice versa. A key can vary in length – generally, the longer it is, the longer it would take to crack it.

Packet

A packet is a unit of data routed between an origin and destination on a network, and made up of a header, the payload (your data), and the trailer. It is normally of a specific set size, which matches the network MTU (Maximum Transmission Unit) size. When data needs to be sent over a network, if it’s larger than a packet it may need to broken down into multiple data packets, which are then sent individually and the data reassembled at the destination.

Port

Just as a boat docks at a physical port, computing ports represent the ‘endpoint’ of communication. Any information reaching your device does so via a port.
Protocol A set of rules used to negotiate a connection between a VPN client and a VPN server. Some are more complex or secure than others – OpenVPN and IKEv2 are some popular choices.

Encryption Glossary

About the Author


  • JP Jones - CTO @ Top10VPN

    JP Jones

    JP is our CTO and with over 25 years of software engineering and networking experience oversees all technical aspects of our VPN testing process. Read full bio