DragonFly On-Line Manual Pages
flow-extract(1) DragonFly General Commands Manual flow-extract(1)
NAME
flow-extract -- flow-extract - Displays flows from a flow file.
SYNOPSIS
flow-extract [-aDRBbnh] [ -e program ] [ -E program ] [ -f file ] [ -F
file ] [ -d dumpfile ] [-o outfile ] [ -z complevel ]
DESCRIPTION
The flow-extract is used for selecting flows from a binary log file
created by Flow-tools(1). The program uses the same syntax as the
Netlogger extract program to select flows.
OPTIONS
-a Output matching flows in ASCII format (the default).
-B Sepcify the endianness of the input ('big' or 'little').
-b Output matching flows in binary format. This is useful for
selecting data which can then be processed by flow-tools such as
flow-stat(1). This option implies -n.
-n Do not resolve IP address or port information when displaying
the flows in ASCII format.
-D Do not port information, but do resolve IP addresses when
displaying the flows in ASCII format.
-R Do not resolve IP address, but do resolve port information when
displaying the flows in ASCII format.
-n Do not resolve port information when displaying the flows in
ASCII format. IP addresses will be resolved, however.
-h Print an informative help message describing the options and
then exit.
-e program
Specify en extract program on the command line.
-E program
Specify en extract program on the command line.
-f file
Specify a script containing an extract program.
-F file
Specify a script containing an extract program.
Multiple -e, -f, -E, and -F options can be specified. They are
all combined together and treated as a single program. Programs
specified by -e and -f are appended to the program, while
programs specified with -E and -F are prepended. -E and -F can
be used with extract scripts to insert addition conditions at
the top of the script without having to edit it each time.
-d dumpfile
Specify the datafile to read as input. Defaults to standard
input.
-o outfile
Specify the output file. Defaults to standard output.
-z complevel
Specify the compression level. Flow-extract compresses files by
default using zlib. This option specifies how much the files
should be compressed. Defaults to 6, valid numbers are 0 (no
compression) to 9.
USAGE
Programs
program consists of selector-action statements. Statements have the
form
selector { action }
A missing selector means to always select the flow. Flow-extract
processes the entire program for each flow in the log file unless a
matching statement contains a next action.
Actions
There are three actions currently define, print, printall and next.
Print outputs the flow in the format selected using the -a and -b
switches.
Printall outputs the flow with all of the information in it. This
format displays the data that is specific to various flow versions. Its
format is consistant between all versions. Those fields which do not
exist in the flow version that is being processed contain a dash. If -b
is used for binary output there is no difference between .BI print and
printall as the binary version contains all of the flow information
regardless.
Next skips to the next flow and restarts scanning of program.
Selectors
The following selectors are available.
net =, != netname|ipaddr[/netmask]
Select flows if the source or destination network is [not] in the
network specified. The optional netmask allows subnets to be
specified.
srcnet =, != netname|ipaddr[/netmask]
Select flows if the source network is [not] in the network specified.
The optional netmask allows subnets to be specified.
dstnet =, != netname|ipaddr[/netmask]
Select flows if the destination network is [not] in the network
specified. The optional netmask allows subnets to be specified.
host =, !=, <, >, ><, <=, >= hostname|ipaddr
Select flows based on the source or destination host.
srchost =, !=, <, >, ><, <=, >= hostname|ipaddr
Select flows based on the source host.
dsthost =, !=, <, >, ><, <=, >= hostname|ipaddr
Select flows based on the destination host.
iface =, !=, <, >, ><, <=, >= interface
Select flows based on the router interface.
srciface =, !=, <, >, ><, <=, >= interface
Select flows based on the source interface.
dstiface =, !=, <, >, ><, <=, >= interface
Select flows based on the destination interface.
hp =, !=, <, >, ><, <=, >= hostname|ipaddr service-name|port
Select flows based on the source or destination host-port comination.
For example, hp = mail.uchicago.edu smtp will match any flows where the
source host is mail.uchicago.edu and the source port is smtp or any
flows where the destination host is mail.uchicago.edu and the
destination port is smtp.
srchp =, !=, <, >, ><, <=, >= hostname|ipaddr service-name|port
Select flows based on the source host-port combination.
dsthp =, !=, <, >, ><, <=, >= hostname|ipaddr service-name|port
Select flows based on the destination host-port combination. srcport
=, !=, <, >, ><, <=, >= service-name|port
Select flows based on the UDP or TCP source port. Service-name is an
entry from the /etc/services file. Port is an integer number
indicating the port.
dstport =, !=, <, >, ><, <=, >= service-name|port
Select flows based on the UDP or TCP destination port. Service-name is
an entry from the /etc/services file. Port is an integer number
indicating the port.
proto =, !=, <, >, ><, <=, >= number
Selects flows based on the protocol number.
pkts =, !=, <, >, ><, <=, >= number
Selects flows based on the number of packets in the flow.
octets =, !=, <, >, ><, <=, >= number
Selects flows based on the number of octets (bytes) in the flow.
flag FIN|SYN|RST|PUSH|ACK|URG
Selects flows based on the TCP flags set on the flow.
flags safrpu/safrpu
Selects flows based on the TCP flags set on the flow. The flags on the
right of the slash are the mask for the flags on the left. That is, if
the right hand side has saf set and the left hand side has s set, any
flows which have the SYN packet set and the ACK and FIN packets not set
will be selected.
date =, !=, <, >, ><, <=, >= YYYY-MM-DD | YY-MM-DD | MM/DD/YY | MM/DD |
today | yesterday
Select flows based on the date of the flow.
time =, !=, <, >, ><, <=, >= HH:MM:SS | HH:MM
Select flows based on the time of the flow.
before YYYY-MM-DD [HH:MM] | YY-MM-DD [HH:MM] | MM/DD/YY [HH:MM] | today
[HH:MM] | yesterday [HH:MM]
Select flows based on the time of the flow.
since YYYY-MM-DD [HH:MM] | YY-MM-DD [HH:MM] | MM/DD/YY [HH:MM] | today
[HH:MM] | yesterday [HH:MM]
Select flows based on the time of the flow.
Combining Selectors
Selectors can be combined using the following operators.
and, && Short circuited boolean AND.
or, || Short circuited boolean OR.
not, ! Unary negation operators.
Parenthesis can be used to group expressions.
EXAMPLES
Print in ASCII any records showing attempts to ssh to host myhost.
flow-extract -e 'dsthost = myhost && dstport = ssh {print}' -d ft-v06.2002-02-15.200000
Show all traffic on 2001-02-15 to the 128.135.119.0/255.255.255.0
subnet and sent it to flow-stat to find the top bandwidth user by port.
flow-cat ft-v06.2002-02-15* | flow-extract -be 'net = 128.135.119.0/255.255.255.0 {print}' | flow-stat -f7 | sort -nr +2 -3
Show the details of all traffic crossing router interface 43 that is a
successful TCP connection.
flow-extract -e 'iface = 43 && flag ack && ! flags ar/arf {printall}'
Sample script called unusual to print records for unexpected TCP
traffic. This would have to be tailored to a site.
#!/opt/bin/flow-extract -f
# Script 'unusual' for printing unusual traffic
port = ssh || port = smtp || port = http || port = 113 {next}
hp = news.uchicago.edu nntp {next}
# If we made it to here, then print the record
{print}
Run the previous script, but only show records for today.
unusual -E 'before today {next}'
BUGS
This man page needs more examples.
AUTHORS
flow-extract was ported from the TAMU extract program to use flow data
by E. Larry Lidz. Large portions of the code (and this man page) come
directly from there. Many flow specific features have been added to the
program by E. Larry Lidz as well. If you're interested in the changes,
see the ChangeLog with the source, or compare this man page to
extract(8).
SEE ALSO
flow-tools(1), extract(8)
flow-extract(1)