DragonFly On-Line Manual Pages
L2T_PROCESS(1) User Contributed Perl Documentation L2T_PROCESS(1)
NAME
l2t_process - A small script to process the CSV output from
log2timeline, sorts and extracts sorten dates
SYNOPSIS
l2t_process [OPTIONS] -b CSV_FILE [DATE_RANGE]
Where DATE_RANGE is MM-DD-YYYY or MM-DD-YYYY..MM-DD-YYYY
OPTIONS
-b|-body CSVFILE
The name of the file that contains the CSV output produced by
log2timeline.
-t|-tab The default input to the tool is a file that was created using
the CSV output module. However, the TAB module can also be
used, however you will need to tell the tool that the file is
TAB delimited instead of comma separated, using this option.
-i|-include
The tool detects possible timestomping activity against changes
made to MFT records (millisecond is of zero value). This option
makes the tool add lines that contain suspicious entries even
though they fall outside the supplied date filter.
-e|-exclude
The tool detects possible timestomping activity against changes
made to MFT records (millisecond is of zero value). If this
option is supplied the tool will not ask the user to add the
lines that are suspicous yet are outside the supplied date
range.
-v|-verbose
Making the script produce mode debug information (be more
verbose)
-y The default format for the date variable is mm-dd-yyyy, however
this default behavior can be changed with this option so the
format read is yyyy-mm-dd.
-V|-Version
Print the tools version number and exit.
-k|-keyword FILE
Include a keyword file that contains one keyword per line. The
tool will read the keyword file line-by-line, and then compare
each line in the CSV file against each of those keywords. The
tool will only print out those lines that match the keywords.
The words inside the keyword list are case insensitive.
-w|-whitelist FILE
Include a keyword file that contains one keyword per line. The
file has the same format as the keyword file, and does the same
thing, except that this file lists up keywords of words that
should not be contained in the timeline. That is to say, this
file defines the "known good" or whitelisted lines that should
be kept out of the timeline.
The tool starts by comparing the known keywords before
processing the whitelist, meaning that keywords are first
filtered out before the whitelist is processed. So the
whitelist can be used in conjunction to the blacklist to narrow
down the scope even more.
It can also be used to remove known "good entries" or entries
that are not relevant to the current investigation out of the
timeline.
-s|-scatter FILE
This only makes sense when the timeline contains records from
the MFT parser (NTFS filesystem). Then the tool will take the
creation time of each file that resides in the WINDOWS/System32
directory and scatter plot it against the MFT number of that
file. The tool will both plot the $FN and $SI creation time of
the file.
This can be useful during malware investigations, to quickly
find files that might have been added to the system32 folder.
When the operating system in installed, and during patching
there are usually several files written to the system32 folder
at once and since MFT's are associated sequentially there
should be clear association between MFT numbers and creation
time. However a typical malware does not create several files
in the system32 directory, a typical malware tries to hide and
does so by creating as few files as possible. That makes it
possible to view a scatter plot, showing the relationship
between creation time and MFT numbers to quickly spot those
outliers or anomalies. This technique can therefore be used for
data reduction.
This option creates a simple gnuplot data file and a gnuplot
script that can be used to create a simple scatter plot to see
those outliers. It will also make an attempt at identifying
those outliers with a simple algorithm. By default the tool
treats the entire dataset as a single slice and tries to find
the obvious outliers, however that behaviour can be changed
using the -m or --multi option to tell the tool to try to split
the dataset into slices.
The FILE portion should be the name of the output file the tool
writes to, it should only contain ASCII letters: a-z, A-Z,
underscore (_) and numbers 0-9, no dot. The files created will
be: FILE.dat and FILE.cmd
Then the tool gnuplot has to be run, like:
gnuplot FILE.cmd
Which will produce a file called FILE.png, containing the
scatter plot.
If the tool detects any outliers in the dataset then the file
FILE_outliers.txt will be created. That file will contain a
list of all those files that the tool detected as outliers.
-m|--multi
This option is only available when used with the -s FILE, to
create scatter plot of the creation time vs. $MFT entry
numbers. By default the tool treats the entire dataset as a
single slice and tries to detect outliers in it. Since the
relationship between $MFT entry numbers and creation time isn't
a simple line, in reality it consists of several straight
lines, there will be many false negatives when treating the
dataset as a single slice. Therefore the option of trying to
split the dataset into multiple smaller slices, and calculating
the outliers for each one of those has been provided.
This is a simple approach to this problem, and by no means
solves the issue at hand. This method does produce lots of
false positives (and it could also miss some, or produce false
negatives). However it will catch many of the items that get
missed by the first attempt.
Perhaps the best approach is to start with the default
behaviour of the tool, examine the graph manually. And if there
are some outliers in the dataset that are perhaps aligned with
another line, yet are obvious outliers, then to re-run the tool
using this option to try to see if it gets detected.
-h|-help
Print this help message
[DATE_RANGE]
The date range is formulated as one of the following:
MM-DD-YYYY All dates from the date supplied date and
forward from them. That is to say, the date
defines the starting date and all dates after
that date will be part of the selection.
MM-DD-YYYY..MM-DD-YYYY
This is a range, so all events that fall within
the boundaries set by these two dates will be
part of the selection.
DESCRIPTION
l2t_process takes as an input the CSV output produced from the CSV
output module of log2timeline and sorts the file. It also has the
capability to only let the file contain entries from a certain date
range, or a similar behavior of the tool mactime from the SleuthKit
(which works on mactime body files).
The tool also removes any duplicate entries that might appear in the
timeline. This can occur when recursive scans are made, since the same
timestamp can be present in more than one file, such as registry
entries both in NTUSER.DAT and inside various restore points. The tool
will remove the duplicate entries and change the filename so it
includes all the files that the timestamp is found in.
There is also "timestomp" detection, in the sense that if you used the
MFT module of log2timeline to parse the NTFS $MFT file, and there are
entries that have zero nanoseconds (second precision), which might be
an indication of timestomping (since those tools only work on 32-bits
of the timestamp, that is up to the second). So if you run the tool
with limited date range, and there are entries that fall outside the
date entry that have zero nanoseonds (second precision), the tool will
ask if you would like to include them in the timeline.
EXAMPLES
Process the file combined.txt and only include entries that occured
from January the 1st, 2004, until March the 31st the same year.
l2t_process -b combined.txt -y 2004-01-01..2004-03-31 > examine.txt
Go over the file combined.txt and only include lines that fit the
keyword list provided in the file dirty.txt
l2t_process -k dirty.txt -b combined.txt > dirty.txt
AUTHOR
Kristinn Gudjonsson <kristinn (a t) log2timeline ( d o t ) net> is the
original author of the program.
The tool is released under GPL so anyone can contribute to the tool.
COPYRIGHT AND LICENSE
Copyright 2009-2011 by Kristinn Gudjonsson (kristinn ( a t )
log2timeline ( d o t ) net )
log2timeline is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
Free Software Foundation, either version 3 of the License, or (at your
option) any later version.
log2timeline is distributed in the hope that it will be useful, but
WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
General Public License for more details.
You should have received a copy of the GNU General Public License along
with log2timeline. If not, see <http://www.gnu.org/licenses/>.
perl v5.20.2 2012-05-22 L2T_PROCESS(1)