CRAWDAD metadata: tools/process/pads/snmp_parser (v. 2006-09-21)

snmp_parser provides a C library for processing snmp traces, and several tool implementations. Using this library, users can develop their own snmp tools. The PADS system (http://www.padsproj.org) needs to be installed to build and use the library and tools.
[xml metadata]

Note: This metadata was prepared by the CRAWDAD team and verified by the data set (or tool) authors. We have made every effort to ensure its accuracy, but urge all users to consider the metadata and data carefully and be sure that their use in research is consistent with the nature and limitations of the data. We welcome any corrections. This metadata was prepared based on the following reference(s):


CRAWDAD metadata structure[what is CRAWDAD metadata]


[Tool] tools/process/pads/snmp_parser (v. 2006-09-21)

top

version v. 2006-09-21
changes
the initial version
bibtex
@MISC{tools-process-pads-snmp_parser-2006-09-21,
  author = {Jihwang Yeo},
  title = {{CRAWDAD} tool tools/process/pads/snmp_parser (v. 2006-09-21)}, 
  howpublished = {Downloaded from http://crawdad.cs.dartmouth.edu/tools/process/pads/snmp_parser},
  month = sep,  
  year = 2006
}
					
metadata last modified2006-11-14
summary
snmp_parser provides a C library for processing snmp traces, and several
tool implementations. Using this library, users can develop their own
snmp tools. The PADS system (http://www.padsproj.org) needs to be installed
to build and use the library and tools.
release date2006-09-21
web site http://www.crawdad.org/tools/process/pads/snmp_parser
wiki go to the wiki page for this tool
keywordSNMP, 802.11
authorsJihwang Yeo
license
/*
 * snmp.p: pads description of snmp trace
 *
 *      Author: Jihwang Yeo
 *      version: v. 2006-09-21
 *      Copyright (c) 2006 Dartmouth College
 *
 * This program is free software; you can redistribute it and/or modify it
 * under the terms of the GNU General Public License Version 2 as published by
 * the Free Software Foundation.
 *
 * This program is distributed in the hope that it will be useful, but WITHOUT
 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
 * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
 * more details.
 *
 * You should have received a copy of the GNU General Public License along with
 * this program; if not, write to the Free Software Foundation, Inc., 51
 * Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
 */
support
You can find more information about PADS at the wiki (see the link above) or 
PADS web page (http://www.padsproj.org).
Please let us know any bugfixes and suggestions at crawdad@cs.dartmouth.edu
build
Step 1. Download and install PADS from http://www.padsproj.org/
Step 2. Uncompress and extract snmp-parser.tar.gz
Step 3. `cd snmp-parser` then type `make`
    - GNUmakefile will generate pads library source files in gen directory.
    - GNUmakefile will create a system-dependent $AST_ARCH directory
    (e.g., darwin.ppc on MAC) and generate the executables in the directory
output
See "usage" for details about the output of each tool.
parameters
See "usage" for details about the parameters needed for each tool.
usage
1. snmp-accum

>snmp-accum [snmp trace file (data/snmp by default)]

(See "note" for the supported snmp formats)

The accumulator reports basic statistics (e.g., min, max, avg) and value distribution
of the input file with respect to each field defined in snmp.p.
We implemented snmp-accum by slightly modifying 'wsl-accum.c' in pads/demo directory.
usage
2. snmp-fmt
snmp-fmt [snmp trace file (data/snmp by default)]

(See "note" for the supported snmp formats)

The formatter converts a given trace format into a delimited format.
We implemented snmp-fmt by slightly modifying 'wsl-fmt.c' in pads/demo directory.
example
1 snmp-accum

>$AST_ARCH/snmp-accum data/sample.snmp.v31.IOS

(...)
= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
top.meas.clients.some_clients_0_t.clsv31ios.cls.allArrayElts.cDot11ClientSignalStrength : int32
= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
good vals:       1009    bad vals:          0    pcnt-bad:    0.000
  Characterizing int32 values:  min -102 max 0 avg -91.801
    => distribution of top 10 values out of 51 distinct values:
        val:       -100 count:        130  pcnt-of-good-vals:   12.884
        val:        -99 count:        110  pcnt-of-good-vals:   10.902
        val:        -94 count:         85  pcnt-of-good-vals:    8.424
        val:        -97 count:         82  pcnt-of-good-vals:    8.127
        val:        -95 count:         72  pcnt-of-good-vals:    7.136
        val:        -98 count:         68  pcnt-of-good-vals:    6.739
        val:        -96 count:         57  pcnt-of-good-vals:    5.649
        val:        -93 count:         53  pcnt-of-good-vals:    5.253
        val:       -101 count:         49  pcnt-of-good-vals:    4.856
        val:        -92 count:         43  pcnt-of-good-vals:    4.262
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
        SUMMING         count:        749  pcnt-of-good-vals:   74.232
(...)

The above example shows the accumulator result of cDot11ClientSignalStrength
(client signal strength) field. From the output we can see that the trace contains
1009 values as signal strength and there is no bad values in terms of data type
(int32 in this case). The output then shows basic statistics (min, max, and avg)
over the values, followed by the report of the top ten most frequent values.

2 snmp-fmt

>$AST_ARCH/snmp-fmt data/sample.snmp.v31.IOS

(...)
1078030565|ResBldg47AP1|392597232,IOS - 350 - 12.2(13)JA1
|1078030565|ResBldg47AP1|1|Dot11Radio0|71|11000000|127774391|13913074|0|80|124657587|9219659|401805|331275|51672|51672|43521|3991|51665|51665|6|0|0
|1078030565|ResBldg47AP1|003065a1fafb|-1|1|236|3|129.170.161.38|2296|-91|278452|105860|653|1128|2|1762|0
|1078030565|ResBldg47AP1|00904b5088af|-1|1|168|3|129.170.161.222|4148|-94|41730|4923|341|22|0|1935|0
|1078030565|ResBldg47AP1|00904b267708|-1|1|5|3|129.170.160.239|1084|-98|17490|5660|90|25|0|146|0
|1078030565|ResBldg47AP1|00904b86c8ba|-1|1|171|3|129.170.161.208|4076|-95|29785|8545|173|40|0|90|0
gorge:~/pads/snmp-parser jyeo$ vi data/sample.snmp.v31.IOS
(...)

The above example shows the formatter result on sample.snmp.v31.IOS trace.
The formatter converts the original trace format (see below) into tab-delimited format.


sys,1078030565,ResBldg47AP1,392597232,IOS - 350 - 12.2(13)JA1
if,1078030565,ResBldg47AP1,1,Dot11Radio0,71,11000000,127774391,13913074,0,80,124657587,9219659,401805,331275,51672,51672,43521,3991,51665,51665,6,0,0
cl,1078030565,ResBldg47AP1,003065a1fafb,-1,1,236,3,129.170.161.38,2296,-91,,278452,105860,653,1128,2,1762,0
cl,1078030565,ResBldg47AP1,00904b5088af,-1,1,168,3,129.170.161.222,4148,-94,,41730,4923,341,22,0,1935,0
cl,1078030565,ResBldg47AP1,00904b267708,-1,1,5,3,129.170.160.239,1084,-98,,17490,5660,90,25,0,146,0
cl,1078030565,ResBldg47AP1,00904b86c8ba,-1,1,171,3,129.170.161.208,4076,-95,,29785,8545,173,40,0,90,0
algorithm
snmp-parser consists of PADS description of SNMP traces (snmp.p) and source code of
accumulator and formatter application (snmp-accum.c and snmp-fmt.c). 
To build the applications, the users are required to install PADS system beforehand and
compile the snmp-parser source files (*.p and *.c) using PADS compiler.
PADS compiler generates a C library for manipulating snmp traces, which is used for
building snmp-parser applications (snmp-accum and snmp-fmt).
For more detail about the PADS, go to the wiki (see the link above) or pads web page
(http://www.padsproj.org).
note
SNMP data format
--------------------
1. V2.1 snmp format (data/sample.snmp.v21)

This trace format is used for collecting dartmouth/campus/snmp/fall01 traceset.

The first five lines are comments.  The first gives basic information:
#V2.1: file format version 2.1, timestamp of file creation, AP name, and date code YYMMDD
All timestamps are standard Unix timestamps (seconds since 1970). The other four comment
lines describe the format of lines that occur later in the file.  Other than the timestamp
and AP name, the rest of these fields are MIB variable names.
After the five comment lines comes a series of polls.  Each poll consists of one ''sys'' line,
one ''if'' line describing stats of the the wireless interface, and zero or more pairs of
''c1'' and ''c2'' lines, each pair describing a currently connected client.  The c1 and c2
lines are a collection of MIB variables from the AWC MIB (Aironet Wireless Communications is
the name of the company that developed our access points; Aironet was bought by Cisco
who then branded and sold the APs under their name).

#V2.1,1018929767,AdmBldg27AP2,020416
#sys,timestamp,AP,sysUpTime
#if,timestamp,AP,ifIndex,ifType,ifSpeed,ifInOctets,ifInUcastPkts,ifInErrors,ifInDiscards,ifOutOctets,ifOutUcastPkts,ifOutErrors,ifOutDiscards
#c1,timestamp,AP,awcDot11TpFdbAddress,awcDot11TpFdbClientState,awcDot11TpFdbLatestRxSignalStrength,awcDot11TpFdbLatestRxSignalQuality
#c2,timestamp,AP,awcTpFdbAddress,awcTpFdbClassID,awcTpFdbSrcOctetsImmed,awcTpFdbDestOctetsImmed,awcTpFdbIPv4Addr
        
2. V3.0 trace format (data/sample.snmp.v30)

This trace format is used for collecing a subset of dartmouth/campus/snmp/fall0304 traceset.
V3.0 has the similar format to V2.1 as follws:

#V3.0,1067662868,AcadBldg10AP10,031101
#sys,timestamp,AP,sysUpTime
#if,timestamp,AP,ifIndex,ifDescr,ifType,ifSpeed,ifInOctets,ifInUcastPkts,ifInErrors,ifInDiscards,ifOutOctets,ifOutUcastPkts,ifOutErrors,ifOutDiscards,awcDot11AssociatedStationCount,awcDot11ReassociatedStationCount,awcDot11RoamedStationCount,awcDot11DeauthenticateCount,awcDot11DisassociateCount,awcFtClientSTASelf,awcFtBridgeSelf,awcFtRepeaterSelf
#c1,timestamp,AP,awcDot11TpFdbAddress,awcDot11TpFdbAID,awcDot11TpFdbClientState,awcDot11TpFdbLatestRxSignalStrength,awcDot11TpFdbLatestRxSignalQuality,awcDot11TpFdbCurrentBSS,awcDot11TpFdbSSID
#c2,timestamp,AP,awcTpFdbAddress,awcTpFdbClassID,awcTpFdbSrcOctetsImmed,awcTpFdbDestOctetsImmed,awcTpFdbIPv4Addr,awcTpFdbSrcPktsImmed,awcTpFdbDestPktsImmed,awcTpFdbSrcErrorPktsImmed,awcTpFdbDestErrorPktsImmed

3. V3.1 trace format (data/sample.snmp.v31.IOS and data/sample.snmp.v31.VxWorks)

This trace format is used for collecting a subset of dartmouth/campus/snmp/fall0304 traceset.
At the time of this data collection, Dartmouth mainly used Cisco 340 and 350 APs. 
These used to run the VxWorks operating system. During December 2003 to May 2004,
our 350 APs migrated from running VxWorks  to the Cisco IOS (the APs didn't originally run IOS
as they were made by Aironet, a company that was later bought by Cisco). IOS uses completely
different SNMP MIBs to VxWorks, and so the variable names and their order are slightly different.
When the upgrades started taking place, we incremented the log version number to "V3.1"
(the first line of each log) to indicate the new variables being queried. We also folded both
the "c1" and "c2" client-specific lines into one "cl" line (this made the parser code easier to maintain).

To identify whether a given SNMP log is IOS or VxWorks, look at the ''sys'' line in a V3.1 log. The fifth field of this line is a formatted ''sysDescr'' indicating the OS version of the AP.

An example VxWorks file
#V3.1,1073710881,ResBldg48AP1,040110
#sys,timestamp,AP,sysUpTime,sysDescr
#if,timestamp,AP,ifIndex,ifDescr,ifType,ifSpeed,ifInOctets,ifInUcastPkts,ifInErrors,ifInDiscards,ifOutOctets,ifOutUcastPkts,ifOutErrors,ifOutDiscards,awcDot11AssociatedStationCount,awcDot11AuthenticatedStationCount,awcDot11ReassociatedStationCount,awcDot11RoamedStationCount,awcDot11DeauthenticateCount,awcDot11DisassociateCount,awcFtClientSTASelf,awcFtBridgeSelf,awcFtRepeaterSelf
#cl,timestamp,AP,awcDot11TpFdbAddress,awcDot11TpFdbAID,awcDot11TpFdbClientState,awcDot11TpFdbLatestRxSignalStrength,awcDot11TpFdbLatestRxSignalQuality,awcTpFdbClassID,awcTpFdbSrcOctetsImmed,awcTpFdbDestOctetsImmed,awcTpFdbIPv4Addr,awcTpFdbSrcPktsImmed,awcTpFdbDestPktsImmed,awcTpFdbSrcErrorPktsImmed,awcTpFdbDestErrorPktsImmed

An example IOS file:
#V3.1,1075784465,ResBldg47AP1,040203
#sys,timestamp,AP,sysUpTime,sysDescr
#if,timestamp,AP,ifIndex,ifDescr,ifType,ifSpeed,ifInOctets,ifInUcastPkts,ifInErrors,ifInDiscards,ifOutOctets,ifOutUcastPkts,ifOutErrors,ifOutDiscards,cDot11AssStatsAssociated,cDot11AssStatsAuthenticated,cDot11AssStatsRoamedIn,cDot11AssStatsRoamedAway,cDot11AssStatsDeauthenticated,cDot11AssStatsDisassociated,cDot11ActiveWirelessClients,cDot11ActiveBridges,cDot11ActiveRepeaters
#cl,timestamp,AP,cDot11ClientAddress,cDot11ClientRoleClassType,cDot11ClientPowerSaveMode,cDot11ClientAid,cDot11ClientAssociationState,cDot11ClientIpAddress,cDot11ClientUpTime,cDot11ClientSignalStrength,cDot11ClientSigQuality,cDot11ClientBytesSent,cDot11ClientBytesReceived,cDot11ClientPacketsSent,cDot11ClientPacketsReceived,cDot11ClientDuplicates,cDot11ClientMsduRetries,cDot11ClientMsduFails
download urlDownload (96 KB tar.gz) from US UK
download urlDownload (0.3 KB README) from US UK

[Author] Jihwang Yeo

top

emailjyeo@cs.dartmouth.edu
institutionDartmouth College
departmentComputer Science
positionProgrammer
address6211 Sudikoff Laboratory, Hanover, NH 03755-3510 USA
phone603-646-8746
fax603-646-1672
related data/toolsdartmouth/campus (v. 2009-09-09)
tools/process/pads/snmp_parser (v. 2006-09-21)