[Traceset] dartmouth/campus/syslog (v. 2009-09-09)

top

AP locations in meter as of the end of 2008 have been added.

The changed components are as follows:

@MISC{dartmouth-campus-syslog-2009-09-09,
  author = {Tristan Henderson and David Kotz and Jihwang Yeo},
  title = {{CRAWDAD} trace set dartmouth/campus/syslog (v. 2009-09-09)}, 
  howpublished = {Downloaded from http://crawdad.cs.dartmouth.edu/dartmouth/campus/syslog},
  month = sep,  
  year = 2009
}
					

The traceset of nearly continuous recording of the syslog records produced by the access points, 
from 2001-04-11 to 2004-06-30, and from 2005-09-01 to 2006-10-04.  UNIX timestamps 
have been added to each log record, and MAC addresses and AP names sanitized.

We configured the access points to transmit a syslog message 
every time a client card associated or disassociated with the access point
(We configured the Cisco APs and the Aruba APs differently. Please see the traces 
dartmouth/campus/syslog/01_04 and dartmouth/campus/syslog/05_06 for more details).

Dartmouth currently has no authentication to associate with the network, so we do not know 
the identity of users, and the IP address given to a user varies from time to time and 
building to building.

We have not released syslog trace collected from 2004-07-01 to 2005-08-31.

[Trace] dartmouth/campus/syslog/01_04 (v. 2004-12-18)

top

Syslog trace is newly created.

@MISC{dartmouth-campus-syslog-01_04-2004-12-18,
  author = {David Kotz and Tristan Henderson and Ilya Abyzov and Jihwang Yeo},
  title = {{CRAWDAD} trace dartmouth/campus/syslog/01_04 (v. 2004-12-18)}, 
  howpublished = {Downloaded from http://crawdad.cs.dartmouth.edu/dartmouth/campus/syslog/01_04},
  month = dec,  
  year = 2004
}
					

The trace of nearly continuous recording of the syslog records produced 
by the access points, from 2001-04-11 to 2004-06-30. UNIX timestamps have been added to 
each log record, and MAC addresses and AP names sanitized.

timestamp, AP name, the MAC address of the card, and type of message

Every MAC address has been sanitized, and the IP address or host name of client 
machines has been removed. To sanitize the MAC address, we randomized the bottom six 
hex digits. We collected every MAC address from all of our syslog, SNMP, an tcpdump traces, 
and built a huge table mapping real MACs to randomized MACs, ensuring that all mappings 
are unique.  Each access point name has been blinded in the form: AcadBldg10AP3  where this 
indicates the third AP in the tenth building of type 'Academic.' The building types are 
Adm (Admin), Ath (Athletic),  Lib (Library), Oth (Other - mainly sysadmin test APs), 
Res (Residential) and Soc (Social). Refer to note for details.

We only have a list of these holes in fall 2001. We had ``spatial holes'' because many APs 
did not send syslogs.  [Configuration mistake.] And temporal holes, because our syslog 
recording server(s) failed. You may refer to note for details. It also appears that 
the engineering school's APs, building name ''cummings'' did not send any messages 
after they installed a firewall in early 2002 until I noticed the problem and asked 
them to open a hole in the firewall in late 2002. 
We do not release the syslog trace collected from 2004-07-01 to 2005-08-31.

Since syslog messages are sent from the APs to a relaying server (ns1), and from ns1 to 
our syslog recording servers, as UDP messages, it is possible for them to be lost or 
reordered along the way. The timestamps are applied by the syslog daemon on our host, 
so the timestamps are monotonically increasing. But, the events may have been recorded 
out of order, and some may be missing.  We believe this effect is small enough to be negligible. 
We have two syslog recording servers, and we do not see the same event with different timestamps 
in the two servers. From 10/19/2003 this no longer applies.

[Trace] dartmouth/campus/syslog/05_06 (v. 2007-02-08)

top

This 2005-2006 syslog trace is newly created.

@MISC{dartmouth-campus-syslog-05_06-2007-02-08,
  author = {Tristan Henderson and David Kotz},
  title = {{CRAWDAD} trace dartmouth/campus/syslog/05_06 (v. 2007-02-08)}, 
  howpublished = {Downloaded from http://crawdad.cs.dartmouth.edu/dartmouth/campus/syslog/05_06},
  month = feb,  
  year = 2007
}
					

The trace of nearly continuous recording of the syslog records produced
by the access points, from 2005-09-01 to 2006-10-04. UNIX timestamps have been added to
each log record, and MAC addresses and AP names sanitized.

[Cisco APs]

We configured the Cisco access points to transmit a syslog message every time a client card 
authenticated, associated, reassociated, disassociated, or deauthenticated with 
the access point.  Each message contains the AP name, the MAC address of the card, 
and the type of message.  

[Aruba APs]

On our campus, we deployed Aruba wireless networks with an Aruba 5000 switch
as a master switch, which controls the wireless network in centralized manner.
The configuration has a three-level hierarchy (master-switches-APs) such that
a number of switches are attached to the master switch, and likewise a number
of APs are attached to each switch. We have three models of Aruba APs: 52, 61, and 72.

The wireless network is virtually divided into several zones (subnets),
each of which has a controller that controls a set of APs. Aruba syslog messages
come from either the master switch (the central controller) or each zone controller.
Since different zone covers different set of APs, separate syslog messages come from
each zone controller.

The Aruba system is able to configure multiple ESSIDs on each AP. At the time of
collecting these syslog data, we used four ESSIDs - "Kiewit Wireless", "Kiewit Video",
"Kiewit Voice", and "Hanover Inn". Among those ESSIDs, Kiewit Video/Voice 
are NATed and use private (RFC1918) addresses.

We do not have L2 assoc/disassoc/auth/deauth messages in the Aruba syslog trace 
because the Aruba switch does not give us those. What we do have are "station up" 
and "station down" messages which indicate when the switch sees a client connect 
with a BSSID. Though we do not know what these station up|down messages equate to 
in the 802.11 FSM, for most analyses it should be possible to more or less correlate 
these messages with the assoc|disassoc messages in the Cisco syslog.

Every MAC address has been sanitized by randomizing
the bottom six hex digits, and we mapped all IP addresses using a prefix-preserving
sanitizer.

[Cisco syslog]
Each access point name has been blinded in the form: AcadBldg10AP3 where this indicates 
the third AP in the tenth building of type 'Academic.' The building types are Adm (Admin), 
Ath (Athletic),  Lib (Library), Oth (Other - mainly sysadmin test APs), Res (Residential) 
and Soc (Social).

[Aruba syslog]
Each access point name is represented as location id ([building_id.floor_id.ap_id] like 15.1.1). 
We also provide a file called 'aruba_locid_table.csv' containing a mapping between 
location id and sanitized AP name (e.g., a pair of 15.1.1 and AcadBldg10AP3).

1. Directory and files

This trace consists of three tarballs (directories): 
'syslog-2005-2006-cisco', 'syslog-2005-2006-aruba', and
'syslog-2005-2006-merged'.
'syslog-2005-2006-cisco' and 'syslog-2005-2006-aruba' directories 
contain syslog records from cisco APs and Aruba APs, respectively. 
'syslog-2005-2006-merged' directory contains syslog records
merged from 'cisco' and 'aruba' syslog records, sorted by timestamps.
Each directory contains syslog trace files for each day during the measurement
period. File name follows the format YYMMDD.HHMMSS.{syslog or aruba or merged}.
The time used for file name is in UTC.

For Aruba syslog trace, we represent each access point name as location id
(with the format [building_id.floor_id.ap_id] like 15.1.1). Under the trace
root directory, we provide a file called 'aruba_locid_table.csv' containing
a mapping between location id and sanitized AP name
(e.g., a pair of 15.1.1 and AcadBldg10AP3).

2. Format of Cisco syslog records is as follows:

unix_timestamp timestamp1 AP_name1 timestamp2 counter AP_name2 timestamp2 syslog_message

- unix_timestamp : UNIX timestamp in UTC
- timestamp1 : the time that the syslog message was received
- AP_name1 : the (sanitized) hostname of the host that sent the syslog
- counter : internal counter
- AP_name2 : the (sanitized) hostname that the AP thinks it has. Sometimes AP_name1 and AP_name2 don't match.
- timestamp2 : the AP's clock. Sometimes timestamp1 and timestamp2 don't match.
- syslog_message : syslog message content

The following is a sample line in Cisco IOS syslog trace.

1157014202 Aug 31 04:50:02 AcadBldg33AP1 14375: AcadBldg33AP1 Aug 31 08:50:01: %DOT11-6-ASSOC: Interface Dot11Radio0, Station 0016cbf7ca65 Associated KEY_MGMT[NONE]

3. Format of Aruba syslog records is as follows:

unix_timestamp timestamp ip_address1 year [ip_address2] syslog_message

- unix_timestamp : UNIX timestamp in UTC
- timestamp, year : the time that the syslog message was received
- ip_address1 : the (sanitized) IP address of the controller
- ip_address2 : the (sanitized) IP address of another controller that sent this message.
Sometimes the messages don't come directly from the controller, they come from a controller
further down the tree.
- syslog_message : syslog message content

The following is a sample line in Aruba syslog trace.

1159954944 Oct 4 05:42:24 50.32.208.194 2006 [50.32.208.195] authmgr[510]: <INFO> station down <00:15:f9:9e:71:25> bssid 00:0b:86:d3:ab:80, essid Kiewit Voice, vlan 2242, ingress 0x1168 (tunnel 264), u_encr 1, m_encr 1, loc 15.1.1 slotport 0xfc7

This message means that a user connected to an AP is removed from the user table.
Previously it was connected to VLAN 2242 at location 15.1.1.  The reason for the
disconnection might be one of the following:
- User moved out of the network
- The user is logged out of the network.

[Trace] dartmouth/campus/syslog/aplocations_2008 (v. 2009-09-09)

top

AP locations as of the end of 2008, have been added.

@MISC{dartmouth-campus-syslog-aplocations_2008-2009-09-09,
  author = {Jihwang Yeo},
  title = {{CRAWDAD} trace dartmouth/campus/syslog/aplocations_2008 (v. 2009-09-09)}, 
  howpublished = {Downloaded from http://crawdad.cs.dartmouth.edu/dartmouth/campus/syslog/aplocations_2008},
  month = sep,  
  year = 2009
}
					

A comma-separated list of most of the APs on campus and their locations
as of the end of 2008, as defined in (x,y) coordinates in meter.

AP name (location id), x coordinate (meter), y coordinate (meter) where
AP name is represented as location id 
(with the format [building_id.floor_id.ap_id] like 15.1.1), 
in the same way as the dartmouth/campus/syslog/05_06 trace 
represents the AP names.

aplocations_meter.csv:
This file contains the locations of Aruba APs on campus 
as of the end of 2008. To protect the real locations 
of the APs, we converted the locations into meter coodinates
relative to an arbitrary location on campus.

AP names are represented as location id (with the format 
[building_id.floor_id.ap_id] like 15.1.1), in the same
way as the dartmouth/campus/syslog/05_06 trace represents
the AP names. Therefore, we think that users who use 
the dartmouth/campus/syslog/05_06 trace can easily 
associate the APs in the trace with their locations. 

However, please note that since the AP locations are based 
on the information collected as of the end of 2008, some AP 
locations may have changed (or may have been removed or added) 
from the time the dartmouth/campus/syslog/05_06 
trace was collected.

[Tool] tools/process/syslog/syslog_parser (v. 2006-11-01)

top

the initial version

@MISC{tools-process-syslog-syslog_parser-2006-11-01,
  author = {Tristan Henderson},
  title = {{CRAWDAD} tool tools/process/syslog/syslog_parser (v. 2006-11-01)}, 
  howpublished = {Downloaded from http://crawdad.cs.dartmouth.edu/tools/process/syslog/syslog_parser},
  month = nov,  
  year = 2006
}
					

syslog_parser is a script to parse
                the syslog traces from Cisco VxWorks, Cisco IOS and
                Aruba access points. This script was designed to parse
                the syslog traces in the dartmouth/campus/syslog
                tracesets, but should be useful for other traces as
                well.

# cisco_aruba_syslog_parser.pl: a script to parse syslogs 
#
#      Author: Tristan Henderson
#      version: v. 2006-11-01
#      Copyright (c) 2006 Dartmouth College 
#
# This program is free software; you can redistribute it and/or modify it
# under the terms of the GNU General Public License Version 2 as published by
# the Free Software Foundation.
#
# This program is distributed in the hope that it will be useful, but WITHOUT 
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
# FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
# more details.
#
# You should have received a copy of the GNU General Public License along with
# this program; if not, write to the Free Software Foundation, Inc., 51
# Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.

Please send your suggestions, bug reports and fixes to crawdad@cs.dartmouth.edu

cisco_aruba_syslog_parser.pl uses the Time::Local and
                Getopt::Std perl modules.
                If your perl does not include these modules, please
                install a newer version of perl before   
running the cisco_aruba_syslog_parser.pl script.

cisco_aruba_syslog_parser.pl parses syslog traces (see "usage" for the supported syslogs) 
and extracts the following information:

timestamp, client MAC address, message, AP MAC address

See "usage" for details about the parameters needed for each tool.

This is a script to parse the following syslog traces:

- Cisco VxWorks
- Cisco IOS 
- Aruba: note that we don't really know what the Aruba messages mean, but 
		I assume that "station up" means associate and "station down" 
		means disassociate. Since Aruba messages are received from a
        mobility controller, not an AP, they may not correspond
        directly to 802.11 associate/disassociate.

Note that we don't parse all messages, just ones that were interesting to us.

$./cisco_aruba_syslog_parser.pl -h
usage: ./cisco_aruba_syslog_parser.pl [OPTION] [SYSLOG]
-y <year>       define a year for syslogs 
# syslog messages don't contain the year. 
# you can pass the year using -y <year>.
# otherwise we assume the current year
-t              don't reformat time as a Unix timestamp
-r              show the reason for an event (where available)
-b <file>       file containing APs to ignore
-d              output debug info to STDERR
-a <file>       file containing Aruba APs names 
# for internal use
-h              show this help

An example VxWorks syslog record:
Jun 21 05:00:16 AdmBldg25AP1 AdmBldg25AP1 (Info): Station 0006257c081a Associated

An example IOS syslog record:
Jun 21 05:00:09 AcadBldg34AP2 2698: AcadBldg34AP2: Jun 21 09:00:09: %DOT11-6-ASSOC: Interface Dot11Radio0, Station   000d93737dab Reassociated KEY_MGMT[NONE]

An example aruba syslog record:
1125561901 Sep 1 04:05:01 50.110.24.0 2005 [50.110.24.131] authmgr[643]: <INFO> station down <00:02:2d:46:1f:62> bssid 00:0b:86:5c:e5:f9, essid Kiewit Wireless, vlan 2834, ingress 0x10c3 (tunnel 99), u_encr 1, m_encr 1, loc 167.3.2 slotport 0xfc3

$ ./cisco_aruba_syslog_parser.pl 20010411.vxworks.cisco | head
986990216 0040961e58be authenticated AdmBldg19AP3
986990247 0040961e58be authenticated AdmBldg19AP3
986990247 0040961e58be associated AdmBldg19AP3
986990293 0040961e58be authenticated AdmBldg19AP3
986990364 0040961e58be authenticated AdmBldg19AP3
986990484 0040961e58be authenticated AdmBldg19AP3
986991490 0040961e58be authenticated AdmBldg19AP3
986991491 00601db0635a authenticated AdmBldg16AP1
986991491 00601db0635a associated AdmBldg16AP1
986991532 0040961e58be authenticated AdmBldg19AP3

$ ./cisco_aruba_syslog_parser.pl 20040630.IOS.cisco | head
1088568001 0009b7f3ff1f reassociated AcadBldg4AP3
1088568003 00022d12c361 reassociated ResBldg69AP6
1088568003 00022d12c361 roamed ResBldg69AP4
1088568003 00022d12c361 disassociated ResBldg69AP4
1088568006 00022d12c361 authenticated ResBldg69AP4
1088568006 00022d12c361 associated ResBldg69AP4
1088568006 00022d12c361 roamed ResBldg69AP6
1088568008 00904b86f12a disassociated ResBldg44AP4
1088568013 00022dd9b5b2 disassociated SocBldg3AP2
1088568016 0009b7f3ff1f reassociated ResBldg97AP6

$ ./cisco_aruba_syslog_parser.pl 060831.072842.aruba | head
1157009322 001124567039 associated 98.1.2
1157009335 000d93e3e675 associated 167.3.3
1157009342 0016cff28931 associated 68.3.1
1157009344 00131ab19f7c disassociated 188.4.2
1157009344 00131ab19f7c associated 188.3.1
1157009349 001302f5e3e3 disassociated 119.1.1
1157009363 000d28120f0a disassociated 23.3.11
1157009363 000d28120f0a associated 23.3.1
1157020082 0013024da937 associated 119.4.1
1157020093 00131ab19f7c disassociated 188.3.1

[Author] David Kotz

top

[Author] Tristan Henderson

top

[Author] Ilya Abyzov

top

[Author] Jihwang Yeo

top