flowgrep - basic python IDS/IPS tool
15 december 2004					jose@monkey.org

		"the world's cheapest IPS" -- a friend

flowgrep is a basic IDS/IPS tool written in python as a way to help you
investigate and manage your network. it works by sniffing traffic, 
reassembling TCP streams and UDP/IP fragments into single packets, and 
allowing you to "grep" through their payloads using regular expressions. 
the quality of the regular expression engine is similar to Perl's. think
about it as a marriage of ngrep, tcpkill, and tcpflow.

INSTALLATION

requirements:
	- python 2.2 or later
	- pynids 0.3
	  http://pilcrow.madison.wi.us/pynids/
	  requires libnids http://www.packetfactory.net/projects/libnids/
	  which requires libnet 1.0.2a and libpcap

use "setup.py" or the Makefile to build and install flowgrep.

USAGE

flowgrep's usage is quite simple:

$ flowgrep.py -h
flowgrep.py: TCP stream/UDP payload 'grep' utility
Usage: flowgrep.py OPTIONS [FILTER]

where OPTIONS are any of the following:
   -a [pattern]         match any stream with pattern
   -c [pattern]         match client stream with pattern
   -d [device]          input device 
   -F [file]            obtain server patterns from file, one per line
   -f [file]            obtain client patterns from file, one per line
   -i                   case insensitive match
   -k                   kill matched stream (TCP only)
   -l [dir]             log matched flows relative to dir
   -r [file]            input file (in pcap(3) format)
   -s [pattern]         match server stream with pattern
   -u [username]        run as username (default: nobody)
   -V                   print version information and exit
   -v                   select non-matching input
   -x                   print logged filenames (for use with xargs(1))

   [FILTER]             pcap(3) filter expression


you can specify any string or regular expression to look for and optionally
set the stream to log or be killed (-l or -k, respectively). note that the 
kill option only works on TCP streams. you can look for expressions in
the client or the server (or both) streams. note that for UDP and IP, both 
expressions are tested since no "stream" is established. multiple
expressions can be specified by successive calls to -a, -s or -c. you need 
root permissions to use this effectively since you are sniffing the
network (not needed for -r).

logged streams are written out as two streams for TCP streams or a single
file for UDP and IP payloads that match. the payload for any matched stream
is written out. files are named 'time-source-sport-dest-dport-proto' in the
local directory, with time as a 32 bit integer for seconds since the UNIX
epoch. examples:

-rw-r--r--  1 nobody  jose    40 Oct 20 21:09 1098320995-192.168.10.84-22-192.168.10.190-28318-tcp
-rw-r--r--  1 nobody  jose     7 Oct 20 21:09 1098320995-192.168.10.190-28318-192.168.10.84-22-tcp

note that the files may contain binary data, or malicous data, so handle 
them with care.

flowgrep drops privs after opening the BPF descriptor for capture. it
defaults to running as "nobody" but you can change this with the -u option.

USES OF FLOWGREP

you can do measurements of particular traffic, you can build a very cheap
IPS device (ie for mail-based worm infestations), disrupt spammers, or you 
can just sniff on your coworkers.

see the manpage for some tips and examples.

LICENSE AND AUTHOR
jose nazario <jose @ monkey . org>, BSD 3-clause license

you can always get more flowgrep at http://monkey.org/~jose/software/flowgrep/

CAVEATS

unlike ngrep and grep, flowgrep spits matched data to a logfile. this is
because you could wind up with binary data messing up your terminal, and
it's easy for you to go through the logged files and find what matched.

i'd like to make flowgrep a little more like grep, so any arguments on
how to change it's behavior appreciated.

SIMILAR

ngrep		http://ngrep.sourceforge.net/
tcpflow		http://www.circlemud.org/~jelson/software/tcpflow/
tcpkill		http://monkey.org/~dugsong/dsniff/

REQUIREMENTS FOR 1.0

- support for -A, -B and -C ... would require storing flows ... needs a hard
  limit .. ie +5/-5
- other worthwhile suggestions requested, there are plenty of characters
  left in the alphabet ...
