Event Log Management

Every event that occurs across a network can be recorded in an event log file. The list of events that are recorded by default can be modified to reflect the needs of the organizations system. Information stored in event log files is extremely useful to organizations as it provides real-time indications of network incidents as well as an audit trail of user activity. However extracting useful information can be challenging as it is very difficult to manage and filter the vast amount of data generated.

An organizations’ event log management is only as effective as the amount of data they are including from their networks activity. To be able to provide an accurate report on any particular part of the system, data needs to be generated for that part. For example, you cannot compile a report on who accessed a confidential file if you do not set up the file to raise an event (and have the event logged) when the file is accessed.

As the required level of monitoring depends on the organization and there are many event categories in security auditing, the first step is determining which event categories need to be audited. The following are a list of available categories:

• Account Logon Events
  Track users logon and logoff events.
• Account Management
  Tracks attempts to create users or groups, rename users or groups, enable user accounts, disable user accounts or change account passwords.
• Directory Service Access
  Used with auditing tasks on domain controllers.
• Logon Events
  Records creation and destruction of logon sessions (including remote sessions)
• Object Access
  Used to record user access of objects such as files.
• Policy Change
  Records changes to user rights assignment policies such as Windows Firewall Policy.
• Privilege Use
  Records when users exercise a user privilege.
• Process Tracking
  Tracks process information such as program activation/exit.
• System Events
  Records system events such as shutting down a computer.

Each of these categories contains many subcategories and events which can be used to create a complete audit trail of system activity. It is recommended that only essential events are setup for auditing as generating a large number of events can severely affect system performance.

To enable audit log and specify the files/folders to audit in your operating system please refer to http://support.microsoft.com/

Vantage and Event Logs

After file auditing settings have been implemented on the system, it is a simple process to start managing event logs and extracting information. Although the MS provided interface for event logging and tracing has improved dramatically from the original, Vantage simply does a much better job at it. Hey, don’t take my word for it. Try out both and see for yourself.

WebSpy Vantage’s ability to translate event log data into manageable information will, among other things, enable organizations to:

• Monitor failed authentication attempts
  Identify users trying to access files and folders they are not authorized to access, or the system failing to provide legitimate user access.
• Prevent data loss and leakage
  Identify the access, modification or printing of confidential files to prevent information leakage or identify the person behind accidental or deliberate data loss.
• Ensure employees adhere to specified work schedules
  Monitor event logs that record when an employee’s computer has been powered on or shut down.

Importing Event Logs into Vantage

The first step is to import Windows Event Logs into a storage in Vantage. This process can be added to run automatically at appropriate intervals using Tasks.

After creating a storage for Windows Event Logs, reports can be generated and analysis run. This will allow useful information to be extracted from Event Log data.

Vantage uses aliases for the creation of more meaningful information, for example, event ID’s are translated to an event category to enhance readability of generated reports and analysis. A list of event ID’s and their categories has been included at the bottom of this post for reference purposes.

Importing event logs into a storage:

1. Open Vantage and click the Storages tab
2. In the left pane, click Import Logs This will start the import dialog wizard
3. Enter a name for the storage in the Create a new storage dialog box, then click Next
4. Select the Windows Event Log radio button, then click Next
5. Select the Microsoft format (description: Windows Event Log), then click Next
6. Click Add, enter the name the computer in the Server dialog box, click OK and then click Next
7. Continue through the wizard and select any filter, field or partitioning options to include, then click OK The event log data will now be imported into the storage

Generating a Report

1. Click the Reports tab
2. Select the type of Report to generate Note: Vantage includes many default templates for Windows Event Logs such as Failed Events, Application Errors and Failure Audit Trends.
3. In the left pane, click Generate Report This will launch the Generate Report wizard
4. Select the storage to report on Note: This should be the storage created previously for Windows Event Logs
5. Select the document format(s) for the report
6. Enter the report name in the Document Name dialog box
7. Continue through the wizard and select any splitting, filtering or email options, then click OK The report will now be generated

Running an Analysis

1. Click the Summaries tab
2. In the left pane, click New Analysis This will launch the Create Analysis wizard
3. Enter a name for the analysis in the Name dialog box, select the storage, and check that the schema is set to All Windows Event Schemas, then click Next
4. Select the type of Analysis to run, then click Next
5. Continue through the wizard and select any filtering or summaries options, then click OK The summary will now be generated

The summary allows interactive drilldowns to any level for data mining and information exploration.

Also see previous blog ‘File Access Reporting – How to report on who accessed a file or a folder‘.

If you have any questions about reporting on event logs don’t hesitate to get in touch with our support team.

Event ID’s and Categories

To some it might be obvious, but plenty of search queries used by visitors coming to our site contains phrases such as; “Why monitor internet usage important” and “Why analyze log files”.

Majority of benefits directly relate to the network device being monitored so I will structured the business benefits based on this.

Web Proxy Servers

Web proxy servers maintain log files listing every request, from outgoing traffic, made to the proxy server. By monitoring and reporting on log files from web proxy servers you will be able to identify aspects such as: who is accessing external sites, what sites are being accessed, when the sites were accessed, how much time was spent on the sites, how the user navigates through the sites, what page or search phrase referred the user to the sites, and the type and size of data downloaded from the sites. Use this information to:

• Maximize Employee Productivity
  Identify employees who excessively use corporate Internet resources for recreational purposes. Effectively publishing and communicating Internet usage policies and making employees aware of monitoring activities, and corresponding breach consequences, will assist in reducing personal Internet use.
• Ensure Policy Compliance
  Identify misuse and ensure compliance with acceptable Internet usage policies by monitoring which sites are being viewed, for how long, what is being downloaded and by whom.
• Ensure Legal Compliance
  Mitigate risk of costly liability and litigation issues by ensuring compliance with acts and regulations relating to Internet usage.
• Reduce & Verify Bandwidth costs
  Assess bandwidth usage and identify excessive downloading from particular websites, of specific files, and by which employee. Verify accuracy of Internet Service Provider’s charges.
• Understand and Reward Acceptable usage
  Please read my previous blog covering this area.

Web Servers

Web servers maintain log files listing every request from incoming traffic made to the server. Reporting on these log files can tell you: who is accessing the internal site, what pages are being accessed, when the pages were accessed, how much time was spent on each page, how visitors navigated through the pages, what site or search phrase referred the visitor to the site, and the type and size of data downloaded from the site. Use this information to:

• Verify Effectiveness of Online Campaigns
  View the most common sites referring traffic to your own website to validate the effectiveness of online marketing initiatives. Display search terms commonly used in search engines referring to your company’s website to optimize the website’s search ranking and maximize bids on the correct search terms for online pay-per-click campaigns. Or why not use the search phrases to inspire a new blog post .
• Optimize Website Performance
  Prioritize web page sequences, improve navigation, improve browser support and reduce link breaks by monitoring incoming website traffic, commonly accessed pages, user agents (browsers) accessing your website, client and server errors.

Email and messaging

Every time an email or messaging server sends or receives information they store log files containing data about the sender, the receiver, timing of delivery or receipt, subject line, size of attachment and, depending on the server, name of attachment and content of message. Use this information to:

• Reduce Bandwidth costs
  Identify emails and messages with large attachments, who sent them, and if they were work related.
• Protect Confidential Information
  Monitor email and instant messaging activity to protect the transmission of confidential organizational information.
• Mitigate Litigation Risks
  Mitigate risk of costly liability and litigation issues by ensuring compliance with acts and regulations in relation to sexual harassments, bullying and discrimination that can arise from improper email and messaging usage.
• Maximize Email Virus Protection
  Analyze log files from email virus scanning software, or devices, to identify source of viruses. Identify who sent the virus, who received it, attachment name and how your virus scanner dealt with it.

Network and security devices

Network devices, such as switches, routers and proxies, and security devices, such as firewalls, anti-virus, spyware and spam applications, store log files containing data about network activity and the external and internal traffic that has been blocked or filtered. Use this information to:

• Improve Network Management
  Investigate traffic between computers, ports or applications to diagnose network problems. Gather information to help decide which protocols to prioritize over others. Better manage network resources and troubleshoot certain events.
• Strengthen Security Controls
  Verify the configuration of a network’s firewall and its control of network traffic. Identify and investigate security breaches, determine the source of email viruses and manage their organizational impact.
• Maximize Effectiveness of Existing Blocking & Filtering Solution
  Review websites that employees have been denied and granted access to in order to validate the effectiveness of existing Internet filtering service.

Event logs

Designed to provide an audit trail of system use, event logging records the actions that occur within the system, such as users logging in, failure of a component to start, or an attempt to print a document.

Every event that occurs across a network can be recorded in an event log file. The list of events that are recorded by default can be modified to reflect the needs of the organization’s system. Use this information to:

• Monitor failed authentication attempts
  Identify users trying to access files and folders they are not authorized to access, or the system failing to provide legitimate user access.
• Prevent data loss and leakage
  Identify the access, modification or printing of confidential files to prevent information leakage or identify the person behind accidental or deliberate data loss.
• Ensure employees adhere to specified work schedules
  Monitor event logs that record when an employee’s computer has been powered on or shut down.

Hopefully this will give readers a better understanding of the benefits involved. Perhaps it can be helpful when explaining to employees / employer why and how your Internet and network resources need to be monitored and reported on.

The concern for Internet misuse and costly drawbacks in the business arena were gradually rising and proactive Jack could already visualize an increasing need to monitor organizational web resources. Back then the software focused solely on analyzing and reporting on log files produced by proxy servers. Names such as Proxy Analyzer, Log File Reporter, Proxy Log Hog and Web Analyzer entered Jack’s brain but were quickly dismissed. He needed something better, something with more impact, a name that would draw attention, a descriptive name without the dullness, one name that could encompass the software’s functionality and create a conceptual understanding of what the company was all about….one name to rule them all…

WebSpy at trade shows

To no one’s surprise the name of choice was WebSpy. The name was almost perfect. It got a tick for all aforementioned qualities, it received the attention sought for and proved to be very controversial from a marketing point of view. I can personally vouch for the attention and curiosity we generate at events such as trade shows.

It is now more than a decade since the birth of WebSpy. The softwares’ features and capabilities have changed dramatically. Now allowing organizations to monitor and report on, not only web usage, but also; email usage, instant messaging, event logs, routers, website visitor traffic, firewalls, anti virus and anti-spam applications, the name WebSpy remains the same. Why you might ask. Why keep a name that actually understates the software’s capabilities? Why keep a name that may be perceived as associated with unethical spying activities?

Yes, WebSpy has experienced a few instances when organizations have been suspicious. Where initial cooperation has been resisted based on a misconception about us stemming from the name. However, the name comes with great weight and legacy. WebSpy was one of the industry pioneers in the 90’s and has many years of brand and reputation building. This is something that can’t be tossed out the window and replaced with a generic or uncontroversial name, just to prevent potential misunderstandings. Needless to say these misunderstandings are generally easy to clear and invalidate.

Instead we make an extra effort to prevent misconstructions. We use our website, marketing communication and workforce to always clearly state our mission and conviction. So for the record, just in case someone has missed it:

We do not by any mean promote or condone spying on employees. We are pro-internet access and provide businesses, government departments and educational organizations an alternative to blocking and filtering software. We emphasizes that organizational internet usage should be managed using an honest and open monitoring approach where acceptable internet usage policies are clearly communicated to employees and students.

The End

So here it is (you might want to grab a coffee first!).

Unless you have a sophisticated end point security or file auditing solution in place, you’re pretty much limited to the quality of data found in your Windows Security Event log. By default, accesses to your confidential files are not going to trigger any entries to be written to the Event log. You first need to setup file or folder auditing.

WebSpy have written a nice article to help you out with this: Managing Event Logs

Personally, I’m running Windows Vista SP1.  So I first turned on Object Access auditing by going to Control Panel | Administrative Tools | Local Security Policy | Local Policy | Audit Policy and set Audit Object Access for Success and Failure.

Windows Vista Local Security Policy

In Windows Explorer, navigate to the folder or files to audit, then Right-click | Properties | Security | Advanced | Auditing and click Continue when Vista’s User Access Control gets in the way.  Here you get the option to add Users or Groups to the audit policy. So if you only want to know when Joe Bloggs access the file/folder, then only add Joe Bloggs. If you want to know when anyone accesses the file/folder then add your entire company.

Scroll….

Click OK and apply the changes. If applying this to a folder, take note of the setting to ‘apply the auditing entries to containers within this container’ at the bottom and use as required.

Congratulations. That’s the auditing setup. Once people start accessing these files(s), the auditing information will get recorded to the Security Event Log on the machine that hosts the file(s) in question.

The next step is to import the Windows Security log into your flavour of WebSpy Vantage. I’m using Vantage Ultimate, but the steps are the same for Premium and Giga.

1. Run Vantage (as Administrator if on Vista)
2. Go to the Storages tab and click Import Logs
3. Run through the Import Wizard with these settings:

• Storage: New storage
  

  Input Dialog: Storages Page

• Input Type: Windows Event Log
  

  Input Dialog: Input Type Page

• Loader Selection: Microsoft


Input Dialog: Loader Selection

• Input Selection: Add
  Select either local computer, or multiple computers, enter authentication details and Click ‘Filter Event Logs’. Check the ‘Security’ Log and click OK.
  

  Input Dialog: Input Selection Page - Adding Event Logs

• Click OK to start the import.

If there are any issues with the import process, consult these three WebSpy Knowledgebase articles to do with issues importing event logs:

• Event Log Troubleshooting (Known Issues and Fixes)
• Importing Event Logs from machines on a different domain
• Required Services for Event Log Importing

The first article came in handy for me as I’m running on Vista and in order to import from the Local Security log, you need to run Vantage as Administrator. To do this, go to C:\Program Files\WebSpy\Vantage Ultimate 2.1\ right-click the WebSpy.Vantage.exe and select ‘Run as Administrator’.

Once data has been imported into your storage, check it out on the Summaries screen.

To to the Summaries Tab, Run an Analysis on your new storage (ad-hoc analysis will do) , and go to the Category Summary. There should be some ‘File System’ items there assuming the file has been accessed since setting up file auditing. You can then drilldown to Event Type to see ‘Audit Success’ or ‘Audit Failure’. To see who has Successfully accessed a certain file, drilldown into the ‘Audit Success’ item.

Unfortunately the good stuff is buried in the ‘Message’ field, which you can only access in the Individual Records view. This is because the Message field in Event logs is free form and could vary wildly resulting in millions of unique items. A Message Summary has therefore been excluded from a default ad-hoc analysis for very good performance reasons.

Event logs can also be quite verbose, and if you drilldown to Individual Records at this stage, you’ll see lots of messages like ‘A handle to an object was requested’ which probably isn’t of any great value from a reporting perspective. One way to filter out this noise is by Event ID.

I’ve discovered that the events that correspond to ‘An attempt was made to access an object’ have the ID 4663. (One day I’ll create an alias to map Event IDs to their meaningful description. If you come across a good  resource I can use for this, let me know!).  So go to the Event ID summary and drilldown into 4463 to the Individual Records view.

Once you’re at Individual Records, you can hover over the message field to get details. You can also use the find edit box to search for a particular user or file:

Drilldown into Successful File System Accesses (Event ID 4663)

You can export this view To Word Document, HTML, Text or CSV by right-clicking the Individual Records summary and clicking Export.

You can also create a report template to access this same information, but as there is no ‘Message’ summary to choose from, you need to use the Custom expression options, both when adding a column to a node in a Template, and when specifying your filter.

To add a column to a report that displays an Event Message:

1. Go to the Reports Tab and click New Template
2. Create an Analysis template based on the ‘All Windows Event Schemas’ schema
3. Click New Node and click the Advanced button to launch the Advanced editor.
4. On the General page, delete any existing Key columns and select Add | Key. In the Custom Expression section enter [Message] (include the square brackets) and click OK.

To filter the report:

1. Go to the Filters page of the New Node dialog (alternatively you can specify this filter in for all nodes using the Template Properties dialog)
2. Click Add | Field Value Filter. Select Category from the Summary drop down, and click Add. Enter ‘File System’ (without the quotes) and click OK. Click OK to add the filter.
3. Click Add | Field Value Filter. Select Event ID from the Summary drop down and click Add. Enter ‘4463′ (without the quotes) and click OK.
4. To filter on the Message field, Select Add | Manual Filter Expression.
5. Enter the expression:
6. [Message] LIKE “text to filter for”
   Change ‘text to filter for’ to the user or file that you want to search for. If you want to search for multiple strings, repeat the above expression separated by an AND or an OR, and place brackets wherever it makes sense. For example:
   • [Message] LIKE “scottg” AND [Message] LIKE “.avi”
     Will filter for all .avi files that scottg has accessed.
   • [Message] LIKE “scottg” OR [Message] LIKE “.avi”
     Will filter for any file that scottg has accessed and any avi that anyone has accessed.
   • ([Message] LIKE “scottg” AND [Message] LIKE “.avi”) OR [Message] LIKE “andrew”
     Will filter for any all avi files that scottg has accessed and any file that Andrew has accessed.
7. You can add the individual filters using Add | Manual Filter Expression multiple times, and then using the Manual Filter Expression editor at the bottom to change ANDs to Ors and place brackets appropriately, like so:
   

   Filtering for File Access Events by particular users

8. Right-click the Manual Filter Expression edit box and select Validate to make sure everything is good with the expression.
9. Modify chart settings, sorting, etc as appropriate.

Here’s the resulting report template for you, but please note that it includes the filter above (events for the user’s  ‘Asa’ and ‘Scottw’), so you will need to modify the filter and enter the users or files you want to filter on. Just use the user’s windows login name, and/or the name of the file.  Alternatively, remove the filter altogether if you want to see all File Audit events.

That’s it! Now run your report, automate it using the Tasks screen, and your set!