Version 5

To read more about the upcoming version 5 see this.

Some of the new changes are:

- Totally new Collector service (now called Reaper) with host of new features.

  Less resource usage - no more lots of idle threads

  Multi-instance - multiple copies of the Reaper service can be run on the same machine

- Admin tool now includes a number of ways to manage the Reapers directly.

  Users can be managed from the tool

  Reaper services can be managed (stop/start/pause/continue) from the tool

  Setting up new Reaper service instance from Admin tool (not fully automated yet)

A Beta release is available under downloads.

Overview

Event Scavenger is a system for gathering the contents of specified Event Logs and store it in a database for referencing or reporting. It can be use by any IT professional and developer to help pinpointing problems or trends. A powerful reporting tool is provided to ease looking up and extracting data. 

Data is collected by one of two types of collectors. The first and original is the Scavenging service and the second is the new Event collector service.

Event Scavenger consists of the following parts:

• SQL Server Database - The database has a fairly simple structure that hold all the necessary data.
• Scavenging  service - This is a Windows service that polls specified event logs and store the data in the database. This service is also responsible for maintenance of the system - like archiving and deleting old data.  It can also imports log entries stored in other Event Scavenger databases and copy those its own database. The system allows for multiple collectors to populate a single database. Each collector must run on a separate machine though.
  At least one Scavenging service must run in an Event Scavenger environment to perform the maintenance duties.
• Event Collector service - This is also a Windows service but it is much simpler as the scavenging service. It only capture current events as they are raised.
• Admin tool - This is a simple Windows application to managed the database and settings.
• Viewer - This is a Windows application that is the main front end of the system. It has some powerful filtering and searching capabilities. It also allows you to analyze data using a special grouping control to give an aggregated view of data. Additionally it can export data for further analysis in an application like Excel for creating graphs etc.
• Command line query tool - Simple command line tool to query database and export data into csv format

Benefits

There are multiple benefits of using Event Scavenger over just viewing plain old event logs.

• Provides a consolidated view of events over multiple machines
• Filtering data quickly and easily - on the fly.
• Allows for advanced reporting on the data
• Store the data longer based on requirements
• Eliminate the need to give multiple people (developers/users) access to event logs directly. (yes, it could be a con as well - plan for it properly by using the sql server roles designed for this)

Viewer

The viewer gives you a view of all selected 'machine-logs' at a specified time. It provides several ways to filter and/or highlight data. You can also set up predefined views that specify a set of filters for reuse. There is also a top x filter that can be changed that limit the number of records returned from the database.

Filters

The application allows you to filter data by event type, machine-logs, event sources, event id's, from and to date/time and also text inside an event log message. Additionally you can set a filter based on currently selected entries (like 'only show similar' or 'exclude entries like these'). You can also save the current set of filters to a new view for reuse.
The machine-log, event sources and event id filters are 'multi-aware' meaning multiples of them can be selected at the same time to filter the data. The main window allows for the names of machine-logs, event sources and event id's to be typed in manually to quickly filter entries.

Highlighting

Entries can also be highlighted based on machine-log, event-sources and/or event id's. This makes it easier to spot similar entries.

Grouping

As from version 3.3.0 the viewer supports grouping the results shown. This is useful for finding related entries more quickly. Collapsing and expanding the groups are supported as well under Vista and Windows 7 (and onwards) - the reason is that the ListView control from earlier versions of Windows does not support this functionality.
Grouping can be done on the following fields:

• Machine name
• Log name
• Type
• Source
• Event ID
• Month
• Day
• Hour
• Summary

Exports

The application allows for exporting selected or all data currently displayed to csv format. This can then analyzed further in Excel to create graphs or summary data for reports.

Analyze data

A customized list control allows for viewing data in a summarized view. It allows for dragging and dropping of columns and group by fields.

Credit: I use an old customized version of http://superlist.codeplex.com for this.
From version 3.4 of the Viewer you can get a simple pie chart view of the displayed data.

Auto refresh

The application can be set up to automatically refresh if needed. This is useful if you need to wait for some events to happen.

Command line query tool

The command line query tool takes the following parameters:

Collectors (Windows services)

There are two type of collectors. The first (and original) is the Event Scavenging service.  The second is the new Event Collector service. Both services now support 'self-registration' with the Windows Service Manager with the command line parameters "-install" or "-uninstall".

Event scavenging service

It gathers event logs by polling. Each event log is processed on a separate thread so that any problems gather data from one log will not affect others.The gathering of logs is optimized to only collect new entries since the last time it was read (and successfully saved).  It also create separate threads for maintenance, archiving and importing foreign logs (if configured to do so).

This service polls the logs and read the entries one by one. It does not use any event triggered mechanism since that would mean it could 'miss' events and not access existing events in the log. This means you can import existing entries without loosing 'history' but it may be a bit slower in some cases. The advantage of this is that you don't need to install additional software on any remote machines.

Event collector service

This service subscribes to events as they are raised by the system. It is very light-weight and can handle higher volumes since it does not have to poll and loop through existing entries. Due to the way that event log events work it can only access event logs of the 'local' machine. This means each machine where you need this type of collector must have its own local installation.

At present this type of service does not make use of the centralized administration as the original service does - i.e. It does not pay attention to machine-log properties like being disabled, polling filters, polling frequency (obviously) etc.

Collectors and machine-logs

The following only applies to the original scavenging service: The system allows for multiple collectors (service instances) to gather data into a single database. Each 'machine-log' is 'assigned' to a collector that is responsible for gathering its content. This is useful for when one collector cannot access a certain event log due to any reason like security or location. A 'machine-log' entry can also be disabled stopping any data collection from that event log. The polling frequency for each machine-log can also be configured separately.

Stability

This is one very strong point that has a proven track record. Instances of the (original) collector service has been running for up to 5 years - sometimes months continuously and the only reason it had to be restarted was because something else on the server required a restart. It is actually in use in a production environment and used by real people.

Alerting

Event Scavenger on its own does not do any processing on the captured event log data it gathers - i.e. it does not have the ability to raise alerts or notify someone if it reads an 'Error' or 'Warning' event log entry. The main aim for this tool is gathering, storing and then reporting of the data through the UI tools.

However, I have another tool/project called QuickMon that does general monitoring/alerting. It has a SQL Query Collector that can raise alerts/notifications (i.e. email) if the sql query returns a specified value. Using a combination of Event Scavenger plus QuickMon then allows you to raise alerts/notifications based on data captured from event logs.