DragonFly On-Line Manual Pages
rwbag(1) SiLK Tool Suite rwbag(1)
NAME
rwbag - Build a binary Bag from SiLK Flow records.
SYNOPSIS
rwbag [--sip-flows=OUTPUTFILE] [--dip-flows=OUTPUTFILE]
[--sport-flows=OUTPUTFILE] [--dport-flows=OUTPUTFILE]
[--proto-flows=OUTPUTFILE] [--sensor-flows=OUTPUTFILE]
[--input-flows=OUTPUTFILE] [--output-flows=OUTPUTFILE]
[--nhip-flows=OUTPUTFILE]
[--sip-packets=OUTPUTFILE] [--dip-packets=OUTPUTFILE]
[--sport-packets=OUTPUTFILE] [--dport-packets=OUTPUTFILE]
[--proto-packets=OUTPUTFILE] [--sensor-packets=OUTPUTFILE]
[--input-packets=OUTPUTFILE] [--output-packets=OUTPUTFILE]
[--nhip-packets=OUTPUTFILE]
[--sip-bytes=OUTPUTFILE] [--dip-bytes=OUTPUTFILE]
[--sport-bytes=OUTPUTFILE] [--dport-bytes=OUTPUTFILE]
[--proto-bytes=OUTPUTFILE] [--sensor-bytes=OUTPUTFILE]
[--input-bytes=OUTPUTFILE] [--output-bytes=OUTPUTFILE]
[--nhip-bytes=OUTPUTFILE]
[--note-add=TEXT] [--note-file-add=FILE]
[--print-filenames] [--copy-input=PATH]
[--compression-method=COMP_METHOD]
[--ipv6-policy={ignore,asv4,mix,force,only}]
[--site-config-file=FILENAME]
{[--xargs] | [--xargs=FILENAME] | [FILE [FILE ...]]}
rwbag --help
rwbag --version
DESCRIPTION
rwbag reads SiLK Flow records and builds a Bag. Source IP address,
destination IP address, next hop IP address, source port, destination
port, protocol, input interface index, output interface index, or
sensor ID may be used as the unique key by which to count volumes.
Flows, packets, or bytes may be used as the counter.
rwbag reads SiLK Flow records from the files named on the command line
or from the standard input when no file names are specified and --xargs
is not present. To read the standard input in addition to the named
files, use "-" or "stdin" as a file name. If an input file name ends
in ".gz", the file will be uncompressed as it is read. When the
--xargs switch is provided, rwbag will read the names of the files to
process from the named text file, or from the standard input if no file
name argument is provided to the switch. The input to --xargs must
contain one file name per line.
If adding a value to a key would cause the value to overflow the
maximum value that Bags support, the key's value will be set to the
maximum and processing will continue. In addition, if this is the
first value to overflow in this Bag, a warning will be printed to the
standard error.
If rwbag runs out of memory, it will exit immediately. The output Bag
files will remain behind, each with a size of 0 bytes.
Use rrwwbbaaggccaatt(1) to see the contents of a bag. To create a bag from
textual input or from an IPset, use rrwwbbaaggbbuuiilldd(1). rrwwbbaaggttooooll(1) allows
you to manipulate binary bag files.
OPTIONS
Option names may be abbreviated if the abbreviation is unique or is an
exact match for an option. A parameter to an option may be specified
as --arg=param or --arg param, though the first form is required for
options that take optional parameters.
At least one of the following output flags must be defined. For each,
OUTPUTFILE is the name of a non-existent file, a named pipe, or the
keyword "stdout" to write the binary Bag to the standard output. Only
one switch may use the standard output as its output stream.
--sip-flows=OUTPUTFILE
Count number of flows by unique source IP.
--sip-packets=OUTPUTFILE
Count number of packets by unique source IP.
--sip-bytes=OUTPUTFILE
Count number of bytes by unique source IP.
--dip-flows=OUTPUTFILE
Count number of flows by unique destination IP.
--dip-packets=OUTPUTFILE
Count number of packets by unique destination IP.
--dip-bytes=OUTPUTFILE
Count number of bytes by unique destination IP.
--sport-flows=OUTPUTFILE
Count number of flows by unique source port.
--sport-packets=OUTPUTFILE
Count number of packets by unique source port.
--sport-bytes=OUTPUTFILE
Count number of bytes by unique source port.
--dport-flows=OUTPUTFILE
Count number of flows by unique destination port.
--dport-packets=OUTPUTFILE
Count number of packets by unique destination port.
--dport-bytes=OUTPUTFILE
Count number of bytes by unique destination port.
--proto-flows=OUTPUTFILE
Count number of flows by unique protocol.
--proto-packets=OUTPUTFILE
Count number of packets by unique protocol.
--proto-bytes=OUTPUTFILE
Count number of bytes by unique protocol.
--sensor-flows=OUTPUTFILE
Count number of flows by unique sensor ID.
--sensor-packets=OUTPUTFILE
Count number of packets by unique sensor ID.
--sensor-bytes=OUTPUTFILE
Count number of bytes by unique sensor ID.
--input-flows=OUTPUTFILE
Count number of flows by unique input interface index.
--input-packets=OUTPUTFILE
Count number of packets by unique input interface index.
--input-bytes=OUTPUTFILE
Count number of bytes by unique input interface index.
--output-flows=OUTPUTFILE
Count number of flows by unique output interface index.
--output-packets=OUTPUTFILE
Count number of packets by unique output interface index.
--output-bytes=OUTPUTFILE
Count number of bytes by unique output interface index.
--nhip-flows=OUTPUTFILE
Count number of flows by unique next hop IP.
--nhip-packets=OUTPUTFILE
Count number of packets by unique next hop IP.
--nhip-bytes=OUTPUTFILE
Count number of bytes by unique next hop IP.
--note-add=TEXT
Add the specified TEXT to the header of every output file as an
annotation. This switch may be repeated to add multiple
annotations to a file. To view the annotations, use the
rrwwffiilleeiinnffoo(1) tool.
--note-file-add=FILENAME
Open FILENAME and add the contents of that file to the header of
every output file as an annotation. This switch may be repeated to
add multiple annotations. Currently the application makes no
effort to ensure that FILENAME contains text; be careful that you
do not attempt to add a SiLK data file as an annotation.
--print-filenames
Prints to the standard error the names of input files as they are
opened.
--copy-input=PATH
Copy all binary input to the specified file or named pipe. PATH
can be "stdout" to print flows to the standard output as long as
the --output-path switch has been used to redirect rwbag's ASCII
output.
--ipv6-policy=POLICY
Determine how IPv4 and IPv6 flows are handled when SiLK has been
compiled with IPv6 support. When the switch is not provided, the
SILK_IPV6_POLICY environment variable is checked for a policy. If
it is also unset or contains an invalid policy, the POLICY is mix.
When SiLK has not been compiled with IPv6 support, IPv6 flows are
always ignored, regardless of the value passed to this switch or in
the SILK_IPV6_POLICY variable. The supported values for POLICY
are:
ignore
Ignore any flow record marked as IPv6, regardless of the IP
addresses it contains. Only IP addresses contained in IPv4
flow records will be added to the bag(s).
asv4
Convert IPv6 flow records that contain addresses in the
::ffff:0:0/96 prefix to IPv4 and ignore all other IPv6 flow
records.
mix Process the input as a mixture of IPv4 and IPv6 flow records.
When creating a bag whose key is an IP address and the input
contains IPv6 addresses outside of the ::ffff:0:0/96 prefix,
this policy is equivalent to force; otherwise it is equivalent
to asv4.
force
Convert IPv4 flow records to IPv6, mapping the IPv4 addresses
into the ::ffff:0:0/96 prefix.
only
Process only flow records that are marked as IPv6. Only IP
addresses contained in IPv6 flow records will be added to the
bag(s).
Regardless of the IPv6 policy, when all IPv6 addresses in the bag
are in the ::ffff:0:0/96 prefix, rwbag treats them as IPv4
addresses and writes an IPv4 bag. When any other IPv6 addresses
are present in the bag, the IPv4 addresses in the bag are mapped
into the ::ffff:0:0/96 prefix and rwbag writes an IPv6 bag.
--compression-method=COMP_METHOD
Specify how to compress the output. When this switch is not given,
output to the standard output or to named pipes is not compressed,
and output to files is compressed using the default chosen when
SiLK was compiled. The valid values for COMP_METHOD are determined
by which external libraries were found when SiLK was compiled. To
see the available compression methods and the default method, use
the --help or --version switch. SiLK can support the following
COMP_METHOD values when the required libraries are available.
none
Do not compress the output using an external library.
zlib
Use the zzlliibb(3) library for compressing the output, and always
compress the output regardless of the destination. Using zlib
produces the smallest output files at the cost of speed.
lzo1x
Use the lzo1x algorithm from the LZO real time compression
library for compression, and always compress the output
regardless of the destination. This compression provides good
compression with less memory and CPU overhead.
best
Use lzo1x if available, otherwise use zlib. Only compress the
output when writing to a file.
--site-config-file=FILENAME
Read the SiLK site configuration from the named file FILENAME.
When this switch is not provided, rwbag searches for the site
configuration file in the locations specified in the "FILES"
section.
--xargs
--xargs=FILENAME
Causes rwbag to read file names from FILENAME or from the standard
input if FILENAME is not provided. The input should have one file
name per line. rwbag will open each file in turn and read records
from it, as if the files had been listed on the command line.
--help
Print the available options and exit.
--version
Print the version number and information about how SiLK was
configured, then exit the application.
EXAMPLES
In the following examples, the dollar sign ("$") represents the shell
prompt. The text after the dollar sign represents the command line.
Lines have been wrapped for improved readability, and the back slash
("\") is used to indicate a wrapped line.
To build both source IP and destination IP Bags of flows:
$ rwfilter ... --pass=stdout \
| rwbag --sip-flow=sf.bag --dip-flow=df.bag
To build a Bag containing the number of bytes seen for each /16 prefix
length of source addresses, use the rrwwnneettmmaasskk(1) tool prior to feeding
the input to rwbag:
$ rwfilter ... --pass=stdout \
| rwnetmask --4sip-prefix=16 \
| rwbag --sip-bytes=sf16.bag
(To print the IP addresses of an existing Bag into /16 prefixes, use
the --network-structure switch of rrwwbbaaggccaatt(1).)
ENVIRONMENT
SILK_CLOBBER
The SiLK tools normally refuse to overwrite existing files.
Setting SILK_CLOBBER to a non-empty value removes this restriction.
SILK_CONFIG_FILE
This environment variable is used as the value for the
--site-config-file when that switch is not provided.
SILK_DATA_ROOTDIR
This environment variable specifies the root directory of data
repository. As described in the "FILES" section, rwbag may use
this environment variable when searching for the SiLK site
configuration file.
SILK_PATH
This environment variable gives the root of the install tree. When
searching for configuration files, rwbag may use this environment
variable. See the "FILES" section for details.
FILES
${SILK_CONFIG_FILE}
${SILK_DATA_ROOTDIR}/silk.conf
/data/silk.conf
${SILK_PATH}/share/silk/silk.conf
${SILK_PATH}/share/silk.conf
/usr/local/share/silk/silk.conf
/usr/local/share/silk.conf
Possible locations for the SiLK site configuration file which are
checked when the --site-config-file switch is not provided.
SEE ALSO
rrwwbbaaggbbuuiilldd(1), rrwwbbaaggccaatt(1), rrwwbbaaggttooooll(1), rrwwffiilleeiinnffoo(1), rrwwffiilltteerr(1),
rrwwnneettmmaasskk(1), ssiillkk(7), zzlliibb(3)
SiLK 3.11.0.1 2016-02-19 rwbag(1)