Stateful packet inspection
As the name suggests, stateful firewalls keep track of connection states. They know at which point the connection has reached (SYN state, SYN-ACK state, ESTABLISHED state, and so on).
In order to understand how it works, let's look at an example:
Let's assume that there is a server with a firewall. Now, it needs to download package updates, so it initiates a request to the CentOS update repository with IP 88.150.173.218. Considering that this is the firewall rule associated with the server, can the reply come back to 88.150.173.218 to the server or will it be blocked at the firewall?
This is illustrated by the following diagram:
The answer is YES if it's a stateful firewall and NO if it's a stateless firewall.
In a stateful firewall, whenever the internal server creates an outbound request, the firewall will keep track of the connection state and the destination IP address. So, if the internal server sends a SYN to 8.8.8.8 IP, then it is expected to receive a reply SYN-ACK from 8.8.8.8 back to the internal server. In this case, since the connection (SYN) has been initiated from the internal server first, SYN-ACK will be allowed irrespective of the firewall rule associated.
Fun fact
This is actually a famous interview question. A firewall has a DENY ALL rule; however, it seems that the server still receives regular updates from the application updates. How is this working? If you answer this in terms of the stateful functionality of a firewall, you will definitely make a good impression on the interviewer.