Blog article

IPsec VPN Explained | How IPsec works | IPsec vs SSL

IPsec is integral to many VPN services. Learn more about what is IPsec, how IPsec VPN works, its usage and differences between IPsec VPN and SSL VPN.

Lukas Dolnicek


Min read

Get a business VPN for your team

IPsec (Internet Protocol Security) has earned a huge popularity amongst the VPN services such as GoodAccess. So let's explain how IPsec VPN works and what are the capabilities that make it so popular.

Before we take a dive into the tech stuff, it's important to notice that IPsec has quite a history. It is interlinked with the origins of the Internet and is the result of efforts to develop IP-layer encryption methods in the early 90s.

As an open protocol backed by continuous development, it has proved its qualities over the years and even though challenger protocols such as Wireguard have arisen, IPsec keeps its position as the most widely used VPN protocol together with OpenVPN.

Table of contents

  1. What is IPsec?
  2. What protocols does IPsec use?
  3. Tunneling vs transport mode of IPsec
  4. What is IPsec VPN
  5. IPsec VPN vs SSL VPN. Which one to choose?
  6. Summary

What is IPsec?

The Internet Protocol Security (IPsec) is a network protocol suite that enables secure communications between two devices over IP networks, mostly used on public internet today.

It is a network protocol suite that ensures both packet encryption and source authentication.

IPsec is perfectly suited for ensuring privacy of IP network communications so it is often used to establish a VPN connection by IPsec VPN tools. Today, IPsec is considered a security standard due to the use of strong (unbroken) ciphers and algorithms, TLS authentication, MitM protection, Perfect Forward Secrecy, etc., for a variety of applications such as:

  • securing private network communications,
  • protecting web traffic from snooping or interception,
  • ensuring the integrity of IP packets.

Many routers have at least some implementation of IPsec protocols, as do most operating systems (or there are clients for the particular OS).

What protocols does IPsec use?

It is important to understand that IPsec is not a single protocol.

It uses a group of authentication and encryption protocols to perform specific tasks. Let's check the most important ones.

The Security Authentication Header (AH)

It serves for packet authentication only (origin, integrity), not for encryption.

The Authentication Header encapsulates the packet, securing packet integrity via MD5/SHAxxx, and after that data is sent to the destination router. Once received by the destination router, the packet is decapsulated and checked for potential integrity violations.

There is no payload encryption over the process which limits usage of this protocol. AH is usually used in IPsec transport mode (see below).

Encapsulating Security Payload (ESP)

Similarly to Security Authentication Header, ESP is a part of IPsec protocol suite responsible for data integrity (only for payload) and additionally payload encryption.

The IP header of ESP packet is not encrypted and its integrity is not protected so it can be changed during transit, which enables successful NAT traversal. ESP is usually used in tunneling mode.

Internet Security Association and Key Management Protocol (ISAKMP)

ISAKMP is a protocol used for establishing Security Association (SA). This procedure involves two steps:

  • Phase 1 establishes the IKE SA tunnel, a two-way management tunnel for key exchange. Once the communication is established, IPSEC SA channels for secure data transfer are established in phase 2. Attributes of this one-way IPsec VPN tunnel, such as which cipher, method or key will be used, were pre-agreed by both hosts (in case of IPsec VPN, this is a connection between a gateway and computer).
  • For each IPsec VPN tunnel in phase 2, two separate IPSEC SAs must be established, one for IN, the other for OUT. The most used ISAKMP configuration is manual (pre-shared keys, PSK) and dynamic (IKEv1, IKEv2).
Fig 1: IKEv2 settings in GoodAccess business VPN (admin view).

Tunneling vs. Transport mode of IPsec

There are two modes in which IPsec can be configured to run:

IPsec tunneling mode encrypts and authenticates the entire data packet. The packet is encapsulated into another one so it is eligible for changing an IP header.

Such a procedure implies the possibility of changes in routing, NAT traversal and successful transit of data from a computer behind the router through the public internet to its destination (e.g. another computer behind a different router).

An IPsec VPN tunnel enables creating virtual private networks (both site-to-site VPN and remote access VPN) and is used far more frequently than transit mode.

In IPsec VPN clients, tunneling mode is used as a default option.

Fig 2: With a professional business VPN, users always have a choice to switch between protocols (IKEv2, OpenVPN) based on their preference (personal or technical preference, company guidelines, etc.).

IPsec transport mode only encrypts the data packet payload. The IP header is not subject to change so no changes in routing are possible.

This limitation determinates IPsec transport mode to be used for end-to-end communication only.

Both ends must see each other, so it can be used for an encryption within an already established GRE tunnel.

Fig 3: IPsec allows creating a virtual private tunnel through the public internet.

What is IPsec VPN

A VPN (Virtual Private Network) ensures secure private communications over public networks such as the Internet.

VPNs are a common tool in the armory of every responsible administrator.

You can find different types of VPN on the market. In the age of remote work, distributed IT assets, and omnipresent connectivity, VPN is the way to access files, applications and other resources that would be otherwise accessible only from a local network (see our previous article on how business cloud VPN works).

It also allows secure connection from unprotected public networks (café, airport and generally from anywhere outside the protected company perimeter).

VPNs use a variety of security protocols, or rules, to encrypt data traveling between devices, ensure data integrity and authentication of sender/receiver.

An IPsec VPN is a VPN that uses the IPsec protocol suite to establish and maintain the  privacy of communication between devices, apps or networks over the public internet.

IPsec VPN uses a technique called "tunneling" to encrypt the data that is being sent between the device and the VPN server. The data is first encapsulated in an IPsec packet, which is then encrypted using a cipher. The encrypted packet is then sent over the internet to the VPN server, where it is decrypted and forwarded to the destination.

IPsec VPNs are widely used for several reasons such as:

  • High speed
  • Very strong ciphers
  • High speed of establishing the connection
  • Broad adoption by operating systems, routers and other network devices

Of course, not all VPNs use IPsec. There are alternative choices out there such as OpenVPN, Wireguard and others (see the list of essential VPN protocols on our blog).

Also, not every VPN encrypts on the network layer (L3), a common counterpart is considered SSL VPNs (see the difference later in this blog).

What are the IPsec VPN ports?

When establishing an IKEv2 connection, IPsec uses UDP/500 and UDP/4500 ports by default.

By standard, the connection is established on UDP/500, but if it appears during the IKE establishment that the source/destination is behind the NAT, the port is switched to UDP/4500 (for information about a technique called port forwarding, check the article VPN Port Forwarding: Good or Bad?).

IPsec VPN vs. SSL VPN: Which One Should Your Business Use?

When talking about IPsec VPNs, it's also important to draw a comparison to their collateral - SSL VPNs (Secure Sockets Layer).

There are several differences in terms of  technology, usage, advantages, and disadvantages.

SSL VPNs today mostly use TLS (Transport Layer Security) to encrypt HTTPS traffic.

The purpose of HTTPS is to protect the content of communication between the sender and recipient.

This ensures that anyone who wants to intercept communication will not be able to discover usernames, passwords, banking information, or other sensitive data.

However, because this encryption protects communication between the internet browser and the server, it does not encrypt other data that can be used to trace user behavior on the internet such as IP addresses, physical location, browser and operating system used by the host and connected sites.

All this information can be seen and monitored by the ISP, government, or misused by corporations and attackers. To eliminate such risks, IPsec VPN is a go-to solution.

IPsec VPN and SSL use different technology

  • IPsec VPN works on a different network layer than SSL VPN. IPsec VPN operates on the network layer (L3) while SSL VPN operates on the application layer.
  • IPsec VPN uses the Internet Key Exchange (IKE) protocol for key management and authentication. IKE uses the Diffie-Hellman algorithm to generate a shared secret key that is used to encrypt traffic between two hosts. SSL VPN uses Transport Layer Security (TLS) to encrypt traffic. TLS uses Public Key Infrastructure (PKI) for key management.

IPsec VPNs’ usage differs from SSL VPN

  • IPsec VPN securely interconnects entire networks (site-to-site VPN) OR remote users with a particular protected area such as a local network, application, or the cloud. SSL VPN creates a secure tunnel from the host's web browser to a particular application.

Pros and cons of IPsec and SSL VPN

  • When security is the primary concern, modern cloud IPsec VPN should be chosen over SSL since it encrypts all traffic from the host to the application/network/cloud. SSL VPN secures traffic from the web browser to the web server only.
  • IPsec VPN protects any traffic between two points identified by IP addresses. SSL VPN is best suitable for protecting filesharing over the public Internet, communication between email client - email server, web browser and web server
  • SSL VPN is considered easier to set up and manage since it usually doesn't require the installation (sometimes it does) of a client as IPsec VPN does. However, modern cloud VPNs such as GoodAccess automates the vast majority of settings and can be up and running in less than 10 minutes.
  • IPsec has better performance results due to the usage of UPD (SSL VPN uses TCP)
  • Modern IPsec VPN can be empowered with additional features such as DNS filtering and other security measures. SSL VPN has only one usage.

The problem of choosing between IPsec VPN vs SSL VPN is closely related to the topic "Do You Need a VPN When Most Online Traffic Is Encrypted?" which we have covered in our recent blog.

Wrapping Up IPsec and SSL VPNs

Some may think that VPNs are hardly necessary with the rise of in-built encryption directly in email, browsers, applications and cloud storage.

In reality VPNs still provide vitally needed remote access protection and management. It is especially important with such high proportions of work-from-home employees, increasingly more IT resources and infrastructure in the public cloud, and also increasing pressure on compliance with regulations such as GDPR and the NIS2 Directive.

With its time-proven technology and constant development, the IPsec protocol suite is the go-to for securing business communication on the public internet. And until its competitors, such as Wireguard protocol, mature, it will stay a cornerstone of modern cloud VPNs.

If you are wondering where to get an IPsec VPN, give GoodAccess a try. Just create a free account and take your personal test drive.

Let’s get started

See why your peers choose GoodAccess. Create your free account today and enjoy all premium features for 14 days, hassle-free.
Trusted by 1300+ customers