Articles‎ > ‎Network Security‎ > ‎

Attacking the IPsec Standards in Encryption-only Configurations

Attacking the IPsec Standards in Encryption-only Configurations

 Chintan I. Patel

1. Introduction

IPsec is a framework of open standards for ensuring private communications over public networks [2]. It is an important network layer security control protocol, especially used for Virtual Private Networks (VPN). A VPN is a virtual network built on top of existing physical networks that can provide a secure communications mechanism for data and control information transmitted between networks [2]. VPNs are used most often to protect communications carried over public networks such as the Internet [2].

The use of integrity protection with encryption to prevent active attacks is needed. Lack of strong integrity checks allows space for the attacks to take place despite the past examples where security had been a major concern. Despite knowing about the Bellovin’s classic attacks encryption-only configurations are still allowed by the IPsec standards. Instead of eliminating configurations containing weaknesses, the IPsec Requests For Comments (RFCs) have generated warnings regarding the encryption-only configurations. According to the warnings generated by the IPsec RFCs, correctness of encryption padding must be performed during in-bound processing and checked when implementations are done. The Bellovin’s attacks can be prevented by checking the encryption padding.

The attacks mentioned in the paper are efficient and realistic. These are performed by eavesdropping on Encapsulating Security Payload (ESP)-encrypted traffic and by injecting the traffic on the network. These attacks are cipher-text only.


2. Issues addressed in the paper

The main attacks are applicable only if an IPsec implementation performs a full padding check, in accordance with the recommendations of the IPsec RFCs [1]. Thus, the attacks prevented in Bellovin’s works for the attacks mentioned in [1]. The various attacks show that IPsec in encryption-only configurations is vulnerable, whether or not implementers follow the RFCs and carry out proper padding checks [1].


  • Padding/de-padding and CBC mode Encryption/decryption in tunnel mode ESP.
  • Chosen-plaintext attacks.
  • Ciphertext-only attacks against tunnel-mode ESP based on Options processing.
  • Attacks based on the Protocol field.
  • Attacking IPsec’s traffic flow confidentiality mechanisms.
  • Padding Oracle attacks.
  • Implementing attacks in Linux and OpenSolaris.


3. Core ideas of the paper

The main idea to perform the attacks is to manipulate selected header fields of inner datagrams protected using ESP, the encryption protocol of IPsec, so that they consistently lead to the creation of ICMP messages [1]. The ICMP message will be generated only if the padding and other fields present in the ESP trailer are formatted correctly. ICMP messages will only be available to the attacker in encrypted form [1].


3.1 IPsec summary

The ESP protocol provides a confidentiality service and uses the block cipher algorithm operating in Cipher-Blocking Chaining (CBC) mode. The algorithms, mode, keys, Initialization vectors (IVs) and other associated data to be used in encrypting network traffic are stored in an object called a Security Association (SA) [1].

In tunnel mode ESP, the protected IP datagram is called the inner IP datagram. It is encrypted and forms part of the payload of the outer IP datagram [1].

A Message Authentication Code (MAC) algorithm can also be used to configure ESP, to provide integrity protection.


3.2 IP datagram Headers summary

The attacks depend on the structure of the headers of IP datagram. The fields of the inner datagram are implemented after the completion of IPsec processing. The generations of Internet Control Message Protocol (ICMP) errors are also determined by the attacks performed on the network.

Figure 1: Structure of IP header [1]


The above figure shows the structure of the IP datagram header. The Internet Header Length (IHL) field is 4 bits long and has a value between 5 and 15 [1]. This field indicates the length of the header in 32-bit words [1]. If the value of IHL is greater than 5 then additional options bytes are assumed to be present in the Options field.

If the Options field format is not followed then the datagram is dropped and IP generates an ICMP “parameter problem” message which is routed to the host indicated in the Source Address field [1].

The Protocol field is used to determine the upper layer protocol to which the datagram payload should be passed [1]. If this field value is not supported by the host, then the datagram is dropped and ICMP “protocol unreachable” message is sent back to the host with the help of the source address.

The Time to Live (TTL) field is used to track the “age” of datagrams [1]. The TTL value is decreased by 1 every time an intermediate router is visited by a datagram and is set to an initial value set by the host sending the datagram. If the TTL field reaches zero, then the datagram is dropped and ICMP “Time exceeded” error message is generated and sent to the host depending on its host address. If the value does not reach zero, then the datagram is forwarded.

Exploitation of IHL, Protocol fields, TTL and identification fields is done to carry out the attacks on the network using the mentioned parameters by displaying the respective ICMP error messages. In IPsec configurations, the ICMP error messages are not blocked and are sent in an encrypted form across the networks.


3.3 Methods by which attacks can be performed

3.3.1 Padding and CBC mode Encryption in Tunnel mode ESP

Figure 2: Structure of ESP protected datagram [1]


The attacks mentioned in [1] depend on how padding and CBC mode encryption are performed in IPsec. Operation of CBC mode used by ESP in tunnel mode is as follows. The original (inner) datagram that is to be protected is treated as a sequence of bytes. This sequence is padded with a particular pattern of bytes and then a Pad Length (PL) byte and a Next Header (NH) byte are appended [1].

The NH byte present depicts that the decrypting IPsec entity can know to which protocol implementation it should pass the bytes [1]. In tunnel mode, value of NH must be 4 showing IP-in-IP encapsulation and that the bytes are an IP datagram.

Post padding, the data is encrypted in the CBC mode. Assume that the sequence of bytes is ‘q’ blocks, each consisting of ‘n’ bits ( where n = 64 for triple-DES and n = 128 for AES). These blocks are denoted as P1 , P2 , …. , ­ Pq. ‘K’ is used to denote the block cipher algorithm key and ek (.) is encryption and dk (.) is decryption of blocks using ‘K’ respectively. Generation of ciphertext blocks can be indicated by the following:

Co = IV, Ci = ek (Ci-1

Pi), (1 <= I <= q) [1]

3.3.2 De-padding and CBC mode Decryption in Tunnel mode ESP

The payload of the outer datagram can be recovered by the following:

Pi = Ci-1

 dk (Ci), (1 <= i <= q) [1]

Padding with PL and NH bytes can be stripped off to reveal the original inner datagram. If the value of NH byte is 59 as specified by ESP  RFCs then the packet is discarded. At this point, the receiver is able to reconstruct the original inner datagram that was encrypted [1]. The implementations performed should check whether the cryptographic processes used to recover the inner datagram matches the local IPsec policies. If this check fails, then the datagram must be dropped, and if the check passes, then the inner datagram must be passed for further IP implementation. It is important to note that when ESP is employed without integrity protection, the sequence number in the ESP header is not checked by the recipient [1].

3.3.3 Chosen-plaintext attacks

Figure 3: Network Setup [1]


Assumptions [1]:

  • The size of the block cipher used in ESP is 64 bits.
  • Encryption-only ESP is used in tunnel mode between a pair of security gateways GA and GB; these gateways provide security for pairs of communicating hosts HA and HB located behind the gateways as illustrated in Figure 3. This is a typical VPN configuration of IPsec.
  • The key K used to perform ESP encryption for IP traffic flowing from GA to GB is fixed during the attack.
  • The attacker can monitor and capture ESP-protected datagrams that flow between the two gateways; and
  • The attacker can inject modified datagrams into the network between GA and GB.

Also assume that that attacker has obtained ciphertexts Cj  corresponding to a set of 7 chosen plaintexts Pj (0 <= j <= 6). Pj has an IP datagram with source address HA and destination address HB. TTL field is set to 1.

Operation: Cj is inserted as the payload of an outer datagram into the network with destination address as GB due to which the inner datagram will be recovered at GB and passed to the IP implementation at GB. Hence, the TTL field is decremented and equal to zero, thereby, producing ICMP error “time to live exceeded”.

The error message will be sent to the host HA as directed by the source address of HA and hence, will become an ESP-encrypted inner datagram at GB. Padding is done by adding an extra block consisting of NH bytes.

Therefore, the outer datagram carrying the ciphertext will travel in the reverse direction in the tunnel after Cj is injected into the network.


4. Potential Applications of IPsec

  • Used for applications such as email, file transfers, remote logon, client/server etc.
  • Private and secure communications using the tunnel mode, for example, VPN.
  • Encryption and authentication of IP traffic.
  • Useful for e-business.


5. Future Opportunities of using IPsec

As we know that IPsec has been applied to IPv4, it is also considered to be applied for the advanced IPv6. The main objective of its application in IPv6 would be improving the quality of service and taking into considerations loop holes (also mentioned in the paper) in the protocol against the attacks and also providing an efficient and reliable solution for the advanced IPv6.


6. Conclusion

It was shown that IPsec is vulnerable to attacks by using encryption-only configurations. The padding checks as seen that are suggested by the IETF are weak and can be attacked by the techniques presented in the paper (also discussed in this report).

However, just getting padding checks right is not enough to prevent attacks on encryption-only ESP. Ultimately, it seems better to be more defensive from the outset and outlaw configurations that are known to have security weaknesses [1].


Summary of contents 

  • The definition of the IPsec network layer protocol and the use of IPsec in Virtual Private Networks (VPNs) are discussed. The reasons that led to the Bellovin’s attack and how to recover from them.
  • Various issues addressed in [1] are discussed briefly, mainly addressing the ways of different attacks on a network in the tunnel mode with encryption-only configuration.
  • The core ideas include the basic summaries of IPsec and IP datagram headers. The latter is used in the attacks frequently. Methods of attacking such as Padding/de-padding and CBC mode Encryption/decryption in tunnel mode ESP and Chosen-plaintext are discussed in detail.
  • It is concluded that IPsec is vulnerable to attacks using the encryption-only configuration. Check on padding by the IETF cannot resolve the issue of security. Rather the quality of service and defensive techniques must be performed in order to defend against the vulnerable attacks on the network.




[1] “Attacking the IPsec Standards in Encryption-only Configurations”, Degabriele, J.P.; Paterson, K.G.; Security and Privacy, IEEE Symposium on 20 – 23 May, 2007; Digital Object Finder 10.1109/SP.2007.8

[2] “Guide to IPsec VPNs”, Sheila Frankel, Karen Kent, Ryan Lewkowski, Angela D. Orebaugh, Ronald W. Ritchey, Steven R. Sharma; National Institute of Standards and Technology (NIST) special Publication 800-77, December 2005