Question

Network security

1. explain succinctly how a Denial of Service attack may occur on an implementation of the TCP protocol's 3 way handshake?

2. Suggest solutions on your own on how such a Denial of Service attack on TCP 3-way handshake can potentially be prevented or mitigated? Note that you are being asked to think through this answer on your own. You are however allowed to research on the Internet for potential solutions. Please make sure you understand the solution and write your response in your own words.

Answer 1

1. Denial of Service (DoS) is any kind of attack that aims to prevent users from using the computer resources or services. It is usually accomplished by sending huge amount of requests to the target system. Hence it will become difficult for the victim node to provide service to its users. SYN flood is a denial of service attack which exploits the three-way handshake in a TCP connection. It is also called as half-open attack. DoS is achieved by making a server unable to service its clients by consuming its available resources. In a TCP connection, there exists three processes to establish a connection. Firstly, the client will send a SYN message to the server. As an acknowledgement the server sends a SYN/ACK message. Lastly, the client will have to send an ACK message to indicate the receipt of the packet from server. Thus a server will wait for the final ACK message from the client in three-way handshake. An attacker utilizes this loophole. He /She will bombard the server with a high volume of SYN messages. It may be done with spoofed IP addresses. So the server will respond to every request and waits for the ACK message, which will never hit the server. A port is left open for receiving ACK message from client. Meanwhile the attacker sends more connection requests. This requests make the server to keep a port open for some specific time. Thus in effect all the available ports will be occupied and server cannot service its genuine users. Here the server is half-open as it leaves the connection open. Hence it is a half open attack.

2. Denial of Service attack on TCP 3- way handshake can be mitigated using the following techniques:

• There will be a limit set for the number of half open ports. Some backlog memory is always maintained to service the new requests. Thus increasing the Backlog will prevent the DoS.
• Network administrators can either lower the timeout period or carefully drops the connection requests.
• The oldest open port can be reused when the backlog is about to filled up with SYN messages.
• Server can maintain a cookie to register the newly coming connections when the backlog is completely filled. Server will send the SYN/ACK and removes the SYN packet from backlog for making the port ready for new connection. If any legitimate request appears with ACK message, then the SYN message is recreated from cookie data and serviced.