Using IP tables to secure Linux server against common TCP hack attempts

In the innocent communication world of TCP/IP, when using TCP protocol the communication starts with something called a three way handshake. Both parties assume that the other party is as innocent as it is itself. First party/computer sends a message called SYN to any IP address in the world. The computer/server which receives it and even not knowing who has sent this message, sends back a reply message called SYN-ACK and innocently starts waiting for the final reply from the sender (note that it waits, assuming that the sender will send a final reply). The sending computer upon receiving this reply and following the ethics of TCP/IP communication sends back the final reply message called ACK. This completes the three way hand shake which establishes a connection between these two computers and they start exchanging data between each other.

Computer1 -> SYN -> Computer2
Computer2-> SYN-ACK -> Computer1
Computer1 -> ACK -> Computer2

Now if the first computer simply starts sending a lot of SYN messages to a computer or multiple computers, and on receiving back the SYN-ACK replies never reply back with the final ACK message to any of the computers, then these poor computers who are replying back with the SYN-ACK messages will keep waiting for the final ACK message from the sender computer. The purpose of this attack is to flood a server with so many SYN packets that it starts ignoring incoming request from legitimate users.

Computer1 -> SYN -> Computer2
Computer2 -> SYN-ACK -> Computer1
Computer1 -> No rpely. Poor Computer2 waiting for reply
Computer1 -> another SYN -> Computer2
Computer2 -> SYN-ACK -> Computer1
Computer1 -> No rpely. Poor Computer2 now waiting for two replies
and so on

So make sure that on your system all the NEW incoming tcp connections are SYN packets; otherwise we need to drop them:

iptables -A INPUT -p tcp ! --syn -m state --state NEW -j DROP

DROP FRAGMENT PACKETS

Packets with incoming fragments drop them. This attack result into Linux server panic such data loss.

iptables -A INPUT -f -j DROP

XMAS PACKETS

Incoming malformed XMAS packets drop them:

iptables -A INPUT -p tcp --tcp-flags ALL ALL -j DROP

DROP ALL NULL PACKETS

Incoming malformed NULL packets:

iptables -A INPUT -p tcp --tcp-flags ALL NONE -j DROP

Filter ICMP/PING traffic

Pinging public IP addresses is a common way to find out which IP address is live. Pinging is a process in which a computer echoes back whatever data it receives. This is why in technical terms it is called an echo-request and echo-reply mechanism which works over ICMP protocol. An attacker can also send a computer with the largest possible TCP packet, i.e. 64Kb packet and the computer will reply back with a packet of the same size. Its an easy way to clog a server’s bandwidth and processing power. So its a good idea to either block all the ping requests or let them work only for the selected IP addresses. To block all the ping requests:

iptable -A INPUT -p icmp -j DROP

To exclude some IP address, e.g. 74.125.148.10:

iptable -A INPUT -p icmp -s ! 74.125.148.10 -j DROP

If you remove the ‘!’ sign before the IP address, it’ll mean that only this IP address can’t ping this server, but all the rest of the world can. Using the same way you can block the whole blocks of IP addresses.