Windows IT Pro is the authoritative and independent resource for windows nt, windows 2000, windows 2003, windows xp. Features a collection of resources and magazines for windows IT professionals.
  
  
  Advanced Search 


February 21, 2006

Combining LogParser and Sed

Use two tools to build an effective event-monitoring solution
RSS
View this exclusive article with VIP access -- click here to join |
See More Security Articles Here | Reprints | Or sign up for our VIP Monthly Pass!

The Windows event logs store a huge amount of data. Scrolling down lists of events for specific information can be burdensome, and most administrators probably review the logs only when something bad happens or when something is broken.

In this month's Toolbox, I'll show you a method for extracting interesting data from event logs—such as security events, application misconfigurations, or application usage—and parsing the data by using tools I've talked about in previous Toolbox articles. By combining LogParser and Sed into a single solution, we can extract this valuable information and tighten it into an extremely usable format.

POP3/IMAP Usage Audit
For the purpose of this article, I've chosen to audit POP3/IMAP usage in Microsoft Exchange Server. If you're trying to eliminate POP3/IMAP usage for security reasons, or if you want to lock down POP3/IMAP with Secure Password Authentication or Secure Sockets Layer (SSL), you'll find this article's script useful.

Before you can use LogParser to extract any event log data, however, you need to know what you're looking for. For example, IMAP/POP3 connections create an EventID 1010 in the Application event log, so this is the data that we'll configure LogParser to extract. Looking for EventID 1010 will tell us who is using either protocol, and then we can take subsequent action. Knowing who uses your applications is an important first step toward securing them.

Use LogParser to Extract Data
We'll use LogParser to export the target data from the Windows event log into a text file. (For information about this tool, check out "LogParser," InstantDoc ID 42174.) Using SQL syntax, we'll extract only the fields we're interested in from the POP3 and IMAP4 events, then use I/O streams to parse the data. See "Toolbox: Grep," InstantDoc ID 46869, for a description of how you can pipe (|) data from one program to another. Let's look at the full command, then break it down.

logparser -i evt "SELECT 
  TimeGenerated,SourceName,
  EventID,Message FROM
  Application WHERE EventID
  =1010 AND TimeGenerated > SUB
 (SYSTEM_TIMESTAMP(), 
   TIMESTAMP( '08', 'dd' ) )" -
  q | grep 1010 | sed -r
  "{s/\Client //; s/\suc.{0,}
  box // ; s/\. For.{0,}$//"}|
  cscript mail.js admin@mydomain
  .com -s "Pop3/ IMAP Users" -
  smtp smtp.mydomain.local -p 

The LogParser command comes first. The -i evt switch specifies the input as the event log. This switch is optional, but it ensures that LogParser doesn't get confused. Next is a SQL statement that uses a SELECT statement to define the fields to return with the query. Running

logparser -h -i:evt 

reminds us that the event log includes the fields EventLog, RecordNumber, TimeGenerated, TimeWritten, EventID, EventType, EventTypeName, EventCategory, SourceName, Strings, ComputerName, SID, and Message. In our example, we're interested only in returning the TimeGenerated, SourceName, EventID, and Message fields, so we've tailored the SELECT statement to match.

Security events for your applications might fall in the Security, Application, or System event logs. Every application stores events differently. It's important to identify all possible sources of pertinent information. In our case, because IMAP/POP3 connections create events with ID 1010 in the Application event log, we've configured our Select statement's FROM and WHERE clauses to find these specific events.

Grep Lends Help
Before piping the LogParser command into other tools, you should first test it to ensure that it returns the data you seek by displaying the output to the console, as Figure 1 shows. You'll notice that a lot of data is returned, so you'll want to clean it up. In this example, the message body is long. Ideally, we'd like to extract just the username (e.g., joe@blackstatic.com) and possibly the client address. The command-line tool Grep can help with this task.

Grep lets us simply remove the lines of data that don't contain the actual events. To do this, we pipe the LogParser output into the Grep command and look for Event ID 1010. The following command begins the data distillation and strips the headers and footers:

logparser -i evt "select 
  TimeGenerated,SourceName,Event
  ID,Message FROM application
  where EventID=1010 OR EventID=
  101 7" -q | grep 1010 
   Previous  [1]  2  Next 


Reader Comments

You must log on before posting a comment.

If you don't have a username & password, please register now.




Top Viewed ArticlesView all articles
Friday at PASS Europe 2006

Kevin talks about the closing day of the event and shares a funny Microsoft film. ...

PsExec

This freeware utility lets you execute processes on a remote system and redirect output to the local system. ...

Escape From Yesterworld

Kevin points you to the funniest SQL Server website ever! ...


Security Whitepapers The Impact of Messaging and Web Threats

Why SaaS is the Right Solution for Log Management

Protecting (You and) Your Data with Exchange Server 2007

Related Events How IE7 & The New Extended Validation SSL Certificates Impact Your Site

Top 10 Email Security Challenges and Solutions

Introduction to Identity Lifecycle Manager "2"

Check out our list of Free Email Newsletters!

Security eBooks Spam Fighting and Email Security for the 21st Century

Understanding and Leveraging Code Signing Technologies

A Guide to Windows Certification and Public Keys

Related Security Resources Become a VIP member of the Windows IT Pro community!
Get it all with the VIP CD and VIP access. A $500+ value for only $279!

Subscribe to Windows IT Pro!
Solve your toughest technical problems with our experts and access 10,000 + articles online. 30% off

Monthly Online Pass - Only $5.95!
Get instant access to 10,000+ articles from Windows IT Pro Magazine!

TechNet Virtual Labs
Evaluate and test Microsoft's newest products.


Windows IT Pro Home Register FAQ for Windows WinInfo News
Europe Edition About Us Contact Us/Customer Service Media Kit Affiliates / Licensing  
SQL Server Magazine Office & SharePoint Pro Windows Dev Pro IT Job Hound ITTV
IT Library Technology Resource Directory Connected Home Windows Excavator Windows SuperSite 
 
 Windows IT Pro is a Division of Penton Media Inc.
 Copyright © 2008 Penton Media, Inc., All rights reserved. Terms and Use | Privacy Statement | Reprints and Licensing