Stateful Firewalls

Overview

The purpose of this exercise is to introduce you to network access control schemes and the "principle of least privilege" through the use of iptables firewalls.

After this exercise, you will:

1. understand the basics of stateful firewalls
2. be able to apply that knowledge to configure a basic firewall in Linux using iptables.

Firewalls

Stateless Firewalls

In the late 1980s, the Internet was just beginning to grow beyond its early academic and governmental applications into the commercial and personal worlds. The Great Internet Worm in November of 1988 infected around 6,000 hosts (roughly 10% of the Internet) in the first major infection of its kind and helped to focus research and awareness on securing computers from unauthorized access. It was in this environment that the first firewalls were written about and developed at Digital Equipment Corporation (DEC) and Bell Labs (AT&T).

The first functional firewalls inspected individual packet headers without regard for established connections, other packets, or their contents. These kind of firewalls became known as "packet filters" because they literally filtered the packets one by one according to a set of criteria, not unlike a quality control inspector on an assembly line. For TCP and UDP, these criteria could be reduced essentially to the source and destination addresses and ports in the packet header. For example, a packet filter could reject or drop any packets destined for port 23 (telnet) on host 10.10.10.10 from any address other than 10.10.10.11. This kind of filter could rapidly and inexpensively inspect and classify packets without using much space (although they were not very "smart").

Unsurprisingly, simple packet filters are not adequate for many applications, such as the File Transfer Protocol (FTP), because these protocols open additional connections on random ports that can not be anticipated or recognized by the firewall since it does not understand or consider the state of any connection.

This kind of simple "packet filter" ultimately became known as a "stateless firewall".

Stateful Firewalls

"Stateful firewalls" arrived not long after "stateless firewalls". Stateful firewalls keep tables of network connections and states in memory in order to determine if a packet is part of a preexisting network connection, the start of a new and legitimate connection, or an unwanted or unrelated packet. This kind of firewall can recognize, for example, that a new connection on a random high port from a host with a preexisting FTP connection is a related connection and should be allowed. Another difference is that while a stateless firewall will allow all packets from acceptable hosts to an open port, a stateful firewall can be configured to allow packets to that port only if a legitimate TCP connection (or some other protocol) has already been established in some acceptable way. Understanding protocol state essentially gives stateful firewalls vastly more criteria in deciding whether to accept or reject a packet, which translates into finer granularity.

The cutting edge of firewall design today is what is called an "application-layer firewall", which is a firewall that performs "deep packet inspection". This means that the firewall is capable of looking not just at the header of the packets and the state of the connection, but at the payload of the packet in context of what the application processing the packets will do. For example, an application-layer firewall could be used to block Java applets from HTTP traffic by inspecting the packets and stripping Java code or dropping the packets entirely. In order to do this, it must understand what applet code looks like within the payload portion of any HTTP traffic stream. An application-layer firewall essentially has total control over the network stream, although this control comes at a significant expense in terms of CPU time and software complexity.

Most firewalls in use today lie somewhere between the stateful firewall and the application-layer firewall. These firewalls function essentially as a stateful firewall, but may understand enough of a few applications to perform some application-layer tasks. It is also common to couple a primarily stateful firewall (such as netfilter/iptables) with separate application layer firewalls for individual applications.

Firewall Policy Design

People imagine many different things when they hear the term "firewall" in the context of computer networking. Some envision an impenetrable wall of flame [at least I did --ed.]. A Hollywood screenwriter might envision Harrison Ford battling kidnappers. A mechanic might envision the wall between the engine and passenger compartment of a car. Yet mysteriously, every firewall is illustrated as a boring, red brick wall, typically with no fire in sight.

Actually, the brick wall isn't that strange -- the name "firewall" comes from the brick walls in buildings placed to stop the spread of a fire from one area to another. But no matter who you are or what you see in your minds eye, the conventional wisdom is that firewalls are used to "keep the bad stuff out," whether you're protecting your desktop PC at home, your office LAN, or the Pentagon. However, those of us in the field of computer security often see firewalls more as a means of keeping things in rather than keeping them out.

In one sense, these are two sides of the same coin -- but how you design something is (often unconsciously) directly related to how you view the problem, and this can lead to very different design choices when developing a firewall. The goal of "keeping things out" is by definition, exclusively concerned with keeping external attackers "outside" the system, with no regard for what is inside that is worth protecting, and without considering threats (intentional or unintentional) that are already inside, like malicious or foolish employees. This is only half the picture. In contrast, "keeping things in" by definition concerns itself with what is "inside" like sensitive data, privileged access, etc., and encourages the designer to consider all threats -- both internal and external -- against the protected resource.

Practically speaking, these two goals often result in different default policies. The goal of "keeping things out" often results in a policy that by default allows anything not considered to be a threat. This is called a default allow policy, and the classic example of this kind of firewall allows all outbound traffic, but only allows "untrusted" inbound traffic to special services, such as a web server (which is then responsible for its own security). This is better than nothing, but is hardly secure. If an attacker can trick someone inside into opening a trojan horse, the malicious software can exploit the liberal egress policy by making connections to a malicious host on the Internet, which can be used to send messages to the now-compromised system. Incidentally, this is how the firewalls on most home routers are designed.

On the other hand, the "keeping things in" policy usually results in a policy that by default denies everything, and allows only what is necessary for the proper functioning of a system. This embodies the principle of "least privilege" and in the context of a firewall is called a default deny policy. A firewall configured this way allows only the handful of things that are strictly required. This limits inbound traffic as before, but also only allows outbound traffic to carefully chosen targets. For example, this might only allow oubound traffic to a secured mail server, ssh server, and the few web servers required for an employee to accomplish their job. This drastically limits the means by which traffic can enter or leave the network, and if an employee executes a trojan as in the last example, that malicious software will not be able to contact its evil master because the malicious Internet host will almost certainly not be in the list of allowed outbound connections.

The obvious downside to a "default deny" firewall policy is maintenance and inconvenience -- it is harder to install in the first place, and any new network service or traffic type on the network must be explicitly allowed or it will not function. Allowing all outbound traffic significantly cuts down on this kind of maintenance -- at the cost of security.

Firewall and Network Testing Tools

iptables: set and clear rules in netfilter

iptables is actually the user space tool for administering the netfilter functions and tables in the Linux kernel, but the entire netfilter and iptables package is commonly referred to simply as iptables. iptables has several built-in tables of rules (such as filter and nat) , several built-in "chains" (which are sets of network traffic including the built-in INPUT, OUTPUT, and FORWARD for inbound, outbound, and routed traffic), a set of powerful loadable modules of matching stateful filters, the typical set of stateless criteria (such as source, destination, and interface), and a set of targets that represent what to do with a matching packet. These options allow sophisticated firewalls to be defined.

iptables can be intimidating and confusing at first glance even for veteran sysadmins, but especially to users who are not used to configuring firewalls at all or are used to configuring firewalls through a GUI. iptables expressive plugins further complicate the syntax. A typical iptables command looks something like this:

$ iptables -t filter -i ethX -A INPUT -m state --state NEW -p tcp -s 192.168.0.1 --dport 23 -j REJECT

Upon closer inspection, iptables is revealed to be merely a command whose arguments define a single rule for packet filtering based on a number of possible criteria. iptables takes those arguments translates them one command at a time into priority-ordered filter rules in the Linux kernel. Thinking of iptables as a command with arguments can help demystify netfilter and the process of designing firewalls with iptables -- let's break down the above iptables command and translate it into English:

A few other examples:

$ iptables -p tcp --syn --dport 23 -m connlimit --connlimit-above 2 -j REJECT

This rule (from man iptables) allows 2 telnet connections per client host. Note that this rule uses the connlimit matching module, and rejects additional connections.

$ iptables -A INPUT -i lo -j ACCEPT
$ iptables -A OUTPUT -o lo -j ACCEPT

These rules accepts any inbound or outbound traffic on the internal loopback network device (an internal, logical network adapter the kernel uses for network communication internal to the computer) regardless of state, protocol, source, or destination address. The -i lo and -o lo arguments specify the "input interface" and "output interface" the packet arrived on.

$IPTABLES -t filter -A INPUT -m state --state NEW,RELATED,ESTABLISHED -j ACCEPT
$IPTABLES -t filter -A OUTPUT -m state --state NEW,RELATED,ESTABLISHED -j ACCEPT

These rules accept all INBOUND and OUTBOUND traffic regardless of interface, address, port or protocol. They use the state matching module, but accept all NEW, RELATED, and ESTABLISHED packets (which is basically all traffic). This rule is basically like having no firewall at all!

NEW, RELATED, ESTABLISHED

Think of your firewall as a security checkpoint in a big office building. There's usually two lines -- one for people with IDs, and one for people without IDs. If someone already has an ID, they can skip the long line, and go through. If they don't, they have to wait to get an ID card or visitor's pass. This is analogous to the distinction between NEW traffic versus RELATED or ESTABLISHED traffic (which you usually see together). Traffic marked NEW doesn't have an ID badge yet, because it is the first packet of a new stream of traffic. On the other hand, a packet of a RELATED or ESTABLISHED stream is part of something that by definition has already come through the firewall in the past. In other words, the firewall has already given that stream a "badge" (which is really an entry in an internal firewall data structure).

Among other things, this means that firewalls are typically structured so that the first section passes all accepted RELATED,ESTABLISHED traffic first, and then carefully allows only certain kinds of NEW traffic. Why do it in that order?

While this brief introduction to iptables should point you in the right direction, there are other features of iptables not included here that you may want to use for the exercise. There are many HOWTOs, tips, and tutorials online in addition to the iptables manpage; the exercise manual assumes that in order to complete the iptables exercise, you will need to do some research on your own.

nmap: network mapping port scanner

Nmap (homepage) is a very popular "port scanner" that can be used to determine what kind of services are running on a remote or local host, perform OS fingerprinting, and many other tasks. Nmap is capable of performing many tasks in a "stealth mode" designed to not raise the suspicion of the victim, but some tasks require more obvious techniques.

Nmap is incredibly powerful, but the basic functionality of the application is easy to use:

$ sudo nmap yahoo.com

Starting Nmap 4.20 ( http://insecure.org ) at 2007-09-22 21:33 PDT
Warning: Hostname yahoo.com resolves to 2 IPs. Using 216.109.112.135.
Interesting ports on w2.rc.vip.dcn.yahoo.com (216.109.112.135):
Not shown: 1694 filtered ports
PORT    STATE SERVICE
25/tcp  open  smtp
80/tcp  open  http
443/tcp open  https

Nmap finished: 1 IP address (1 host up) scanned in 29.990 seconds

$ sudo nmap www.somehost.edu -P0
Starting Nmap 4.20 ( http://insecure.org ) at 2007-09-22 21:34 PDT
Interesting ports on dns.somehost.edu (33.xx.111.1):
Not shown: 1677 filtered ports
PORT     STATE  SERVICE
53/tcp   open   domain
80/tcp   open   http
443/tcp  open   https
2048/tcp open   dls-monitor
2049/tcp closed nfs
2053/tcp open   knetd
2064/tcp closed dnet-keyproxy
2065/tcp open   dlsrpn
2067/tcp open   dlswpn
2068/tcp open   advocentkvm
2105/tcp open   eklogin
2106/tcp open   ekshell
2108/tcp open   rkinit
2111/tcp open   kx
2112/tcp open   kip
2120/tcp open   kauth
2121/tcp open   ccproxy-ftp
2201/tcp open   ats
2232/tcp open   ivs-video
2241/tcp closed ivsd

Nmap finished: 1 IP address (1 host up) scanned in 32.385 seconds

See the Nmap manpage or online documentation for advanced features.

ifconfig: configure Linux network devices

ifconfig is the network interface configurator in Linux. It is most commonly used by users to see network addresses and statistics, but can also be used to enable and disable interfaces, set configuration options such as network addresses, and more.

For the purposes of this exercise, ifconfig will be used to determine what network addresses are running on what interfaces.

To see the current interface configurations:

$ ifconfig

eth0      Link encap:Ethernet  HWaddr 00:00:5A:00:01:B3
          inet addr:64.81.0.256  Bcast:64.81.40.255  Mask:255.255.255.0
          inet6 addr: fe80::200:5aff:fe00:1b3/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1826346 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1887951 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:691689933 (659.6 MiB)  TX bytes:1037280707 (989.2 MiB)
          Interrupt:58

eth1      Link encap:Ethernet  HWaddr 00:13:D4:04:44:CA
          inet addr:10.10.10.10  Bcast:10.10.10.255  Mask:255.255.0.0
          inet6 addr: fe80::213:d4ff:fe04:44ca/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1165519 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1549057 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:428484191 (408.6 MiB)  TX bytes:1780325755 (1.6 GiB)
          Interrupt:50

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:5808042 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5808042 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:6895907043 (6.4 GiB)  TX bytes:6895907043 (6.4 GiB)

Note that eth0's address is 64.81.0.256, while eth1's address is 10.10.10.10. This means that the two interfaces are on different networks.

telnet: cleartext remote shell

TELNET (TELe-NETwork) is a cleartext remote terminal protocol. On its face, telnet is very simple; the user issues commands over a TCP socket, and the server replies with the results of those commands and waits for more input. In practice, this is complicated with various network and terminal emulation layers. Still, telnet is one of the simplest and oldest network protocols still in use. Due to its cleartext nature and low level access to the system, telnet is incredibly insecure -- it was common in the past for system administrators to log in as root using telnet on a hub network connection that could be sniffed by any sufficiently prepared attacker.

Thanks to the advent of Secure Shell (ssh), active use of telnet servers has died off except for some specialized uses. One place where telnet lives on is debugging ASCII-based network services. For example, web pages can be retrieved by telnetting to HTTP servers, and emails can be sent by telnetting to SMTP servers.

Telnetting to a suspected open port is still one of the fastest ways to see if a service is available or reachable.

Here are a few sample uses of telnet:

$ telnet yahoo.com 80

Trying 66.94.234.13...
Connected to yahoo.com.
Escape character is '^]'.
GET /
...

<html><head> ...[web page data] ...
</body>
</html>

Connection closed by foreign host.

$ telnet mailserver.net 25
Trying 216.0.1.1...
Connected to mailserver.net.

Escape character is '^]'.
220 mailserver.net ESMTP Postfix (Ubuntu)
HELO sender.net
250 sender.net
MAIL FROM: me@sender.net
250 Ok
RCPT TO: me@mailserver.net
250 Ok
DATA
354 End data with <CR><LF>.<CR><LF>
Subject: test mail message
test message
.

250 Ok: queued as 152CFB8802A
^]

telnet> Connection closed.

You have mail in /var/mail/me

netcat: a network swiss army knife

netcat (often nc on some systems) is a Unix utility for creating and using TCP and UDP sockets. In a very simplified way, netcat is like a telnet client and server without any built in protocol or terminal emulation. Another way of putting it is that netcat is the bare essentials for creating a TCP or UDP socket and client, with hooks for using standard in and standard out for IO.

There are too many cool uses of netcat to describe here. For the purposes of this exercise, we'll use netcat to create "fake" TCP or UDP servers that we can use to test firewall configurations.

Creating fake TCP/UDP servers with netcat

Starting a fake listening server (a program that will accept connections on a port) is as simple as running:

$ sudo nc -l 80         # we need sudo because 80 is a privileged port

... to start a listening TCP socket on port 80. Then, from the another host, you can either use telnet or nc to connect to the server you just started on the the first host. You should be able to type in one window and see output in the other if the network pipe is open.

Testing UDP services is exactly the same -- you can use netcat for that, too. You need to start a listening UDP process on the receiving side, and a sending process on the sending side. If you are testing UDP traffic from a client to a server, you can do something like this:

[server]$ nc -u -l 10000        # listen for UDP traffic on port 10000

Then on the client, do something like this:

[client]$ nc -u server 10000    # connect to server via UDP on port 10000

After establishing the connection, enter some data from standard input (probably your keyboard). Input on the sender should appear on the receiving terminal. Hit ^C to close the programs. UDP is of course an unreliable network protocol, so it's possible that there will be errors in the text file.

You can do any of this in reverse to test the connection from a server to a client. If you're able to transmit data, then the firewall is allowing communication.

Introduction

You are Wilbar Memboob, the Security Administrator of FrobozzCo ("You name it, we make it"). You are looking to hire a new system administrator to replace the guy you just fired -- unfortunately, even though his resume looked great on paper and he interviewed well, once you put him at a console he clearly had no idea what he was doing.

To him, proper permissions meant that things worked; if it was secure then that was just icing on the cake. You should have done something when the other admins starting calling him "J-ACCEPT" -- a.k.a. "accept all traffic". But it was when he changed the company firewall policy to -j ACCEPT and the internal LAN got exposed to some black hat grues that he signed his own pink slip.

To keep this from happening again, you've created a little test for your new applicants to take. Unfortunately, before you can grade the applicants, you need to create an answer key.

Assignment Instructions

In the assignment portion of this exercise, you're going to create an answer key for the iptables portion of the "hiring exam." You'll need to swap in your nodes and complete the exercises below.

Setup

1. If you don't have an account, follow the instructions here.

2. Create an instance of this exercise by following the instructions here, using firewall as Lab name. Your topology will look like below:

   .

3. After setting up the lab, access your server node.

Changes to DETER nodes are lost when the nodes are swapped out. This means that you must manually save your work between working on nodes in order to keep it. However, this exercise includes experimental scripts to help you save and restore your work.

Saving your Work

After completing the firewall-related execises on the server node, cd into the /root directory and execute the script /root/submit.sh; like this:

$ ./submit.sh

...it will make a tarball of all the relevant files, including the firewall.sh script you have updated (it will also copy files for the POSIX Permissions lab, but you do not need to bother with them; you should ignore the errors). You must copy or move the created tarball into your group directory, otherwise it will be lost upon swapout. Note: do not run submit.sh with sudo!

Restoring Your Work

The experimental script /root/restore.sh on the server node, takes as input the path to a tarball created by submit.sh described above and restores the files to their proper locations on the system. This includes all the files you are asked to create or change in the first part, as well as the firewall.sh script for the second part. Note: do not run restore.sh with sudo!

To use the restore.sh, copy a tarball created by submit.sh into the /root/ directory and execute this command:

$ ./restore.sh username-permissions-123123123.tar.gz

You will be asked if you want to automatically overwrite files, and if you want to selectively restore some files and not others. The options are self-explanatory.

Finally, if you don't trust the scripts, you can always make your own backups into your group directory and restore them by hand if you prefer.

/root/firewall/firewall.sh

sudo /root/firewall/firewall.sh

/root/firewall/extingui.sh

ETH="eth0"

iptables -A INPUT -i $ETH -j ACCEPT

c$ telnet server 22
Trying 10.0.1.2...
Connected to server.
Escape character is '^]'.
SSH-1.99-OpenSSH_4.2

See well-formed reply from ssh server in Test #1:

‘SSH-1.99-OpenSSH_4.2’

c$ telnet server 80              
Trying 10.0.1.2...
Connected to server.
Escape character is '^]'. 

c$ nc -q 1 server 80 <<< "GET /"

c$ [client]$ telnet server 3306            
Trying 10.0.1.2...
Connected to server.         
Escape character is '^]'.
K#HY000Host 'client-link0' is not allowed to connect to this MySQL serverConnection closed by foreign host.

c$ ping server
PING server-link0 (10.0.1.2) 56(84) bytes of data.
64 bytes from server-link0 (10.0.1.2): icmp_seq=0 ttl=64 time=0.412 ms

s$ ping client
PING client-link0 (10.0.1.1) 56(84) bytes of data.
64 bytes from client-link0 (10.0.1.1): icmp_seq=0 ttl=64 time=0.302 ms

s$ nc -u -l 10000
hello

c$ nc -u server 10000
  (type ‘hello’)

c$ nc -u -l 10006
hello

s$ nc -u client 10006
  (type ‘hello’)

s$ nc -l PORTNUM

c$ telnet server PORTNUM

Straighforward test unknown

Submission Instructions

Submit a tarball created by submit.sh to your instructor. You must use a tarball created by this script so that it will correctly save the permissions of the directories.