DragonFly On-Line Manual Pages
silk(7) SiLK Tool Suite silk(7)
NAME
SiLK - the System for Internet-Level Knowledge
DESCRIPTION
SiLK is a collection of traffic analysis tools developed by the CERT
Network Situational Awareness Team (CERT NetSA) to facilitate security
analysis of large networks. The SiLK tool suite supports the efficient
collection, storage, and analysis of network flow data, enabling
network security analysts to rapidly query large historical traffic
data sets. SiLK is ideally suited for analyzing traffic on the
backbone or border of a large, distributed enterprise or mid-sized ISP.
A SiLK installation consists of two categories of applications: the
analysis suite and the packing system.
Analysis suite
The SiLK analysis suite is a collection of command-line tools for
processing SiLK Flow records created by the SiLK packing system. These
tools read binary files containing SiLK Flow records and partition,
sort, and count these records. The most important analysis tool is
rrwwffiilltteerr(1), an application for querying the central data repository
for SiLK Flow records that satisfy a set of filtering options. The
tools are intended to be combined in various ways to perform an
analysis task. A typical analysis uses UNIX pipes and intermediate
data files to share data between invocations of tools.
The tools, configuration files, and plug-in modules that make up the
analysis tools are listed below, roughly grouped by functionality.
Filtering, sorting, and display
rrwwffiilltteerr(1) partitions SiLK Flow records into one or more 'pass' and/or
'fail' output streams. rwfilter is the primary tool for pulling flows
from the data store.
ssiillkk..ccoonnff(5) is the configuration file naming the Classes, Types, and
Sensors available at your installation.
rrwwssoorrtt(1) sorts SiLK Flow records using a user-specified key comprised
of record attributes, and writes the records to the named output path
or to the standard output. Users can define new key fields using plug-
ins written in C or PySiLK.
rrwwccuutt(1) prints the attributes of SiLK Flow records in a delimited,
columnar, human-readable format. Users can define new printable
attributes using plug-ins written in C or PySiLK.
SiLK Python Extension
ppyyssiillkk(3). PySiLK, the SiLK Python extension, allows one to read,
manipulate, and write SiLK Flow records, IPsets, and Bags from within
Python. PySiLK may be used in a stand-alone Python program or to write
plug-ins for several SiLK applications. This document describes the
objects, methods, and functions that PySiLK provides. The next entry
describes using PySiLK from within a plug-in.
ssiillkkppyytthhoonn(3). The SiLK Python plug-in provides a way to use PySiLK to
define new partitioning rules for rrwwffiilltteerr(1), new key fields for
rrwwccuutt(1), rrwwggrroouupp(1), and rrwwssoorrtt(1), and new key or value fields for
rrwwssttaattss(1) and rrwwuunniiqq(1).
Counting and statistics
rrwwuunniiqq(1) bins (groups) SiLK Flow records by a user-specified key
comprised of record attributes and prints the total byte, packet,
and/or flow counts for each bin. rwuniq can also print distinct source
IP and destination IP counts. Users can define new key fields and
value fields using plug-ins written in C or PySiLK.
rrwwccoouunntt(1) summarizes SiLK Flow records across time, producing textual
output with counts of bytes, packets, and flow records for each time
bin.
rrwwssttaattss(1) summarizes SiLK Flow records by a user-specified key
comprised of record attributes, computes values from the flow records
that match each key, sorts the results by the value to generate a Top-N
or Bottom-N list, and prints the results. Users can define new key
fields and value fields using plug-ins written in C or PySiLK.
rrwwttoottaall(1) summarizes SiLK Flow records by a specified key and prints
the sum of the byte, packet, and flow counts for flows matching the
key.
rrwwaaddddrrccoouunntt(1) summarizes SiLK flow records by the source or
destination IP and prints the byte, packet, and flow counts for each
IP.
IPset, Bag, and Prefix Map manipulation
rrwwsseett(1) reads SiLK Flow records and generates binary IPset file(s)
containing the source IP addresses or destination IP addresses seen on
the flow records.
rrwwsseettbbuuiilldd(1) reads (textual) IP addresses in dotted-quad or CIDR
notation from an input file or from the standard input and writes a
binary IPset file.
rrwwsseettccaatt(1) prints the contents of a binary IPset file as text.
Additional information about the IPset file can be printed.
rrwwsseettttooooll(1) performs union, intersection, difference, and sampling
functions on the input IPset files, generating a new IPset file.
rrwwsseettmmeemmbbeerr(1) determines whether the IP address specified on the
command line is contained in an IPset.
rrwwbbaagg(1) reads SiLK Flow records and builds binary Bag(s) containing
key-count pairs. An example is a Bag containing the sum of the byte
counts for each source port seen on the flow records.
rrwwbbaaggbbuuiilldd(1) creates a binary Bag file from a binary IPset file or
from a textual input file.
rrwwbbaaggccaatt(1) prints binary Bag files as text.
rrwwbbaaggttooooll(1) performs operations (e.g., addition, subtraction) on
binary Bag files and produces a new Bag file.
rrwwppmmaappbbuuiilldd(1) reads textual input and creates a binary prefix map file
for use with the Address Type (aaddddrrttyyppee(3)) and Prefix Map
(ppmmaappffiilltteerr(3)) utilities.
rrwwppmmaappccaatt(1) prints information about a prefix map file as text. By
default, prints each IP range in the prefix map and its label.
rrwwppmmaappllooookk