Of note, this release includes full support for Microsoft Exchange 2010 Tracking logs (previously supported with the Exchange 2007 loader, but missing a few fields), as well as JunOS (Juniper) Traffic Logs, IronPort Traffic Monitor Logs and Squid Syslog.

We’ve also included an experimental feature to allow multiple instances of WebSpy Vantage to run on the same operating system. The goal here is to run multiple reports at the same time using multiple instances of the application. To do this, we have also included a second experimental feature to disable storage locking. This allows multiple instances of Vantage to read from the same storage at once. These features can only be enabled by including a config file next to the Vantage’s executable. More on this feature here.

Here’s the full list of changes:

Application Changes

• New: Added suffix option to Import Windows Users wizard in Aliases.
• New: Date modifiers now supports h for hour and n for minute, e.g. %[-2h,yyyyMM - HH].
• New: Added tracing to storage publish task.
• Experimental: Multiple instances of Vantage can now be run simultaneously, by adding the multipleInstance key to the application config file.
• Experimental: Storage locking can be turned off to allow multiple instances of Vantage to run reports on a single storage simultaneously. This is done by adding the storageLocking key to the application config file.
• Fix: Import Organization merge options now appends attributes if keep existing user details is selected, and replaces attributes if update user details from the directory is selected.
• Fix: Import Organization merge no longer replaces user’s passwords.
• Fix: Fixed issue where no results were returned when filtering on time less than one day – such as past n hours.
• Fix: Storages are no longer duplicated in the Import new hits task dialog.
• Fix: Fixed issues where the Site Domain summary included sub-domains for European domains (.fr, .be etc).
• Fix: SQL server inputs now commit correctly if the user edits the input and only changes the port number.

Loader Changes

• New: IronPort Traffic Monitor Logs.
• New: Juniper JunOS Traffic Logs (SRX).
• New: Microsoft Exchange 2010.
• New: Squid Syslog.
• Improved: Astaro Security Gateway: Added support for an additional different syslog header.
• Improved: SonicWall: Split syslog format into Web and Firewall schemas, added support for User field, string-type Category field and split Protocol field.
• Fix: Microsoft FTMG: Changed type of Object Source field in from Int32 to String. Users will need to clear/field select/reload their storages before this change will apply.
• Fix: Astaro Mail Gateway: Improved format detection, fixed negative size issue, and Index out of bounds errors.
• Fix: IronPort WSA: Improved format detection.

How to update

To update your software, simply click Tools | Check for updates. Vantage Ultimate users will also need to update the Web Module in order to use the new loader formats that have been added. To update the Vantage Web Module, right-click the WebSpy system tray icon and select ‘Check for updates’. If you have issues with the Web Module update process, please see: http://www.webspy.com.au/forums/viewtopic.php?f=4&t=29

Let me know if you have any questions or issues!

Every organization is different with regards to the volume of logs it creates, but I’ve averaged three data sets submitted to us by customers to produce the following estimates.

You will create roughly 0.9 MB of IronPort WSA access logs per user per day.

Once imported into a WebSpy Storage, the Storage will be about 90% of your original log file size. If you apply NTFS compression to the storage folder, the actual size on disk of the WebSpy Storage will be about 30% of your original log data.

So an organization with 1000 users will produce about 900 MBs of access logs per day. The default WebSpy Storage  will be 810 MB, but with NTFS compression, the size on disk will around 270 MB.

As I said, this is a rough guide based on the average of three sets of sample logs we have in house, so please run your own tests and if you can, let us know your values in the comments below.

What’s New?

Clearswift SECURE Web Gateway W3C

Clearswift have just released the latest version of their SECURE Web Gateway, which includes a transaction log export function. This enables you to send transaction logs in W3C format to an off-box FTP server for analysis. If you are updating to the latest Clearswift SECURE Web Gateway, make sure you update your Vantage software to 2.2.0.55 in order to import your W3C Transaction logs. More information on using WebSpy Vantage with Clearswift SECURE Web Gateway.

Cisco Firewall Bandwidth loader

We have also introduced a new Loader for Cisco ASA, PIX and IOS Firewall devices. This new loader imports TCP, UDP, ICMP and GRE ’session close’ events into one schema, allowing you to aggregate size values across these events. This loader is called Cisco Firewall (Bandwidth) and is now available on the Loader Selection page of the Import Wizard. Previously, these events were imported into separate schemas so there was no great way to determine total bandwidth from your Cisco syslog files (without using Netflow and WebSpy FlowMonitor).

Palo Alto Networks and WatchGuard XTM

We’re also very happy to welcome Palo Alto Networks to the WebSpy supported log file list. Vantage now supports both the CSV and syslog file formats from your PA Firewall.

Another new addition is support for the latest WatchGuard XTM devices running firmware version 11.

Full List of Changes

Here’s the full list of changes included in this update:

• New: Clearswift SECURE Web Gateway W3C.
• New: Palo Alto Networks Firewall (CSV/Syslog)
• New: Cisco Firewall (Bandwidth): This new Cisco loader imports TCP, UDP, ICMP and GRE events from ASA, PIX and IOS syslogs into one schema to aggregate size values across these events.
• New: Added WatchGuard XTM: Currently http-proxy, https-proxy, smtp-proxy and firewall lines are supported.
• Fixed: ISA Server: Fixed format detection issues, and issues importing hits with very large size values.
• Fixed: IronPort WSA: Fixed format detection issues, as well as the import issue “Invalid value for DVS Scan Code”
• Fixed: Sophos WSA: Fixed format detection issues and invalid line issues.

How to update

To update your software, simply click Tools | Check for updates. To update the Vantage Web Module, right-click the WebSpy system tray icon and select ‘Check for updates’. If you have issues with the Web Module update process, please see: http://www.webspy.com.au/forums/viewtopic.php?f=4&t=29

Let me know if you have any questions or issues!

We’ve just completed our ‘WebSpy Reporting for Cisco IronPort’ video. Although this video is aimed at the Cisco channel we’d thought we share it with the rest of the WebSpy community as it provides a great high level overview of Vantage Ultimate’s ability to, not only import and spit out reports, but to learn about an organization’s structure in order to securely deliver the right report to the right person at the right time.

Vantage Ultimate is structured in two parts. One windows application, that handles administration of importing log files, learning about your organizational structure, creating and running reports, and one web application (aka the web module) where users can securely log in to view the report that has been produced for them.

Have a look at the short video below to find out what WebSpy’s founder, Jack Andrys, and product operations manager, Scott Glew, have to say about WebSpy’s and IronPort’s compatibility, as well as a demo of Vantage Ultimate in action.

What’s the point with secure and easy report distribution you might ask? Well, among other things:

Distribute Information. Distribute Responsibility.

IT Managers are too often left with the responsibility of managing and enforcing acceptable Internet usage for the entire organization.

Using Vantage Ultimate administrators can distribute reports to managers or department heads that show the activity of their direct subordinates, placing the responsibility of enforcing acceptable usage with the managers themselves.

Empower Users. Full Disclosure.

You can use Vantage Ultimate to distribute reports to each member in your organization detailing their own personal Internet usage (sites visited, search terms used etc). This empowers users to modify their own behavior once they understand how much of their activity is productive or unproductive and what it costs the organization. It also serves as a gentle reminder to employees that their activities are being recorded and that they should keep their online behavior within the bounds of the organization’s acceptable use policy.

From a privacy point of view, Vantage Ultimate is an easy way to provide employees with full disclosure regarding the information recorded about them.

Self Serve, On-Demand Investigation.

You can also use Vantage Ultimate to distribute original data storages to any of your organization’s members, enabling them to conduct their own ad-hoc drilldowns on any information they require. For example, distribute storages to HR managers enabling them to investigate the activity of employees that have handed in their resignation notice, but still have access to confidential company resources.

Protect Employee Privacy.

By assigning permissions to each of your Vantage Ultimate users, you can ensure they can only view the reports they are allowed to view. However, some employees need to view the traffic of all users, but do not necessarily need to identify users. For example, a network administrator may need to view the amount of traffic downloaded from a particular site per user, but does not need to know which users where involved.

This is easily achieved using the ‘Identify users’ permission, which masks all usernames with a hash code.

For more information about WebSpy’s compatibility with Cisco IronPort and free 30 day trials, visit www.webspy.com/ironport

Here’s the full list of changes:

• New: Juniper SA Series. Vantage can import and report on web traffic and VPN connections.
• New: Microsoft Forefront Protection for Exchange 2010 format.
• New: Avencis SSOx.
• Improved: IronPort WSA: Department and Message fields were sometimes returned as null. Fixed.
• Improved: Microsoft FTMG: Removed usage of deprecated “FilterInfo” field from W3C Web format.
• Improved: Microsoft IAS Radius: Added support for Source/Destination IP and port (field code 5000).

To update your Vantage application simply select Tools | Check for updates.

To update the Vantage Web Module, right-click the WebSpy icon in the Web Module server’s system tray, and select Check for updates. If you have any issues with the Web Module update process, please see my previous blog regarding Web Module Errors and Workarounds.

Majority of security vendors will give you a high level overview of the categories, such as Sports, Shopping, Online Community, Streaming Media, Employment and Gambling, but rarely provides intuitive ways to further investigate the traffic going to the sites within these categories. The nifty thing about WebSpy’s solutions is that, as long as categories are logged, you can use WebSpy to analyze web browsing in relation to these categories and get a much clearer overview of your organization’s web usage.

Classify Productive & Unproductive Categories

Assessing productivity in relation to predefined categories is what I would like to focus on today. I have imported and run an analysis on TMG logs using WebSpy Vantage. As previously mentioned, you can import logs from any security device we support – if the information is in the log file WebSpy can report on it.

TMG logs contain information whether traffic has been ‘Allowed’, ‘Denied’ or ‘Failed’. Using WebSpy Vantage you can easily drill down further into this information. For example, let’s say I’m interested in having a look what categories have been allowed, i.e. not blocked, I simple expand the ‘Allowed’ node and click ‘URL category’.

Allowed Categories - Click to Enlarge

This information is great but it doesn’t tell us anything about productivity. WebSpy Vantage not only provides this assessment for your entire organization, specific department and individual users, but also gives you the ability to customize the categories that are deemed productive as this can vary wildly depending on the industry and organization.

How?

You use WebSpy’s Aliases feature to sort categories in relation to your organization’s view of their productiveness. Our software comes with a default list of aliases so you can either edit these or set up new aliases. I’ll take you through the process of setting up an Alias from scratch.

1. Creating a New Alias

• Click on the Alias tab and select ‘New Alias’ in the top left corner
• Name your Alias something appropriate and provide a short description. I’ll name mine ‘Productivity’.
• Make sure ‘Apply alias to selected summaries’ option is checked
• Click ‘Schema’ to specify the log file type and scroll down to the bottom of the list to locate and select ‘URL Category’.
• Tick the ‘Group unresolved into a single name’ box and name it something appropriate. Let’s go with ‘Uncertain’.

 

2. Add Alias Groups

Once an alias has been added, you need to add alias groups. You can have as many alias groups as you want but for this purpose it makes sense to have only two, ‘Productive’ and ‘Unproductive’. There might be certain categories, such as ‘Education/Reference’ or ‘Blogs/Wiki’, that might be difficult to correctly deem as productive or unproductive and you’d rather not specify. If this is the case you don’t need to add an alias group as it will automatically be created for any category that hasn’t been grouped under the other alias groups. Remember how we ticket ‘Group unresolved into a single name’ and called it ‘Uncertain’ before.

• Click the Add Group button in the Groups task pad.
• Enter the desired alias group name (Productive) in the ‘Key’ edit box and click OK. Repeat steps for the ‘Unproductive’ group.
• At this stage you could also add items (categories) to your group but I’m going to show you another way of adding categories.

   

  

3. Add Categories to your ‘Productive’ and ‘Unproductive’ Alias Groups

This is where customization really works its charm. What is deemed as unproductive at one company might be completely legit and considered productive at another. For example, in a recruitment company one could assume it would perfectly normal for employees to visit other employment sites but this could be considered personal and unproductive at a hospital or real estate agent.

There’s a few different ways of adding items to an Alias group. While still in the Alias screen you can click ‘Refresh Unassigned’ in the top right part of your screen. Because you haven’t assigned anything yet all categories will be displayed. From here you can simply highlight the category group, for example ‘Unproductive’ and Ctrl + click all categories you want to place in that group. Once you’ve selected your categories right click and select ‘Add to selected group’. Repeat the process to add categories to your ‘Productive’ group.

Alternatively, you can go back to the ‘URL Category’ listings in the ‘Summaries’ tab and Ctrl + click selected categories, right click and select ‘Add to alias’, select your ‘Productivity’ alias from the drop down menu and select the ‘Productive’ or ‘Unproductive’ group.

4. Assess Productivity

With aliases, groups and items set up you’re ready to assess productive and unproductive browsing. In the ‘Summaries’ screen, left hand side under ‘Aliases’, simple select your ‘Productivity’ alias and the URL categories will be sorted in accordance with your view of their productiveness.

Productive vs Unproductive Browsing - Click to Enlarge

You can also investigate further by, for example, drilling down to determine what unproductive categories are most popular, what are the most popular unproductive websites within those categories, what hours during the day majority of unproductive sites are accessed (you might have a policy that allows personal web browsing during lunch hours), and of course who spends the most time on unproductive websites within your organization.

Top Unproductive Websites - Click to Enlarge

• How to import your log files and explore the information recorded by IronPort using the Summaries screen
• How to open the customized IronPort Report Templates and Aliases
• How to generate reports
• How to import your organizational structure and report on departments
• How to setup the Web Module and publish reports

PART 1: Importing log files & exploring your IronPort summaries

Once you have exported your IronPort access logs (see http://www.webspy.com.au/vendors/ironport/howto.aspx#ftp), this video takes you through importing your logs into WebSpy Vantage and analyzing data on the Summaries screen.

PART 2: Opening the customized IronPort Templates & Aliases, and running reports

This video takes you through opening the IronPort-specific report templates and aliases and generating a report that provides an overview of your organization’s Internet usage.

PART 3: Importing your Organization structure & generating department reports

This video shows you how to import your organizational structure into WebSpy Vantage from a directory server (such as Active Directory) using LDAP, and then generating a report that contains information on your newly imported departments.

PART 4: Using the Web Module.

This video takes you through configuring and using the WebSpy Vantage Web Module. Specifically, it takes you through the following tasks:

• Configuring the Web Module for Windows Authentication
• Adding a Web Module to Vantage
• Publishing reports to the Web Module
• Adding permissions for a user
• Synchronizing the Web Module
• Using the Dynamic Reports tab

PART 5: A quick word about tasks & conclusion

This video summarizes the actions taken in the previous four videos and also briefly discusses how to automate the reporting processing using scheduled tasks.

I hope this helps! Let me know if you have any questions by leaving a comment below.

This is a great update for Vantage Ultimate users as we’ve introduced a new feature/tab into the Web Module called ‘Dynamic Reports’.

If you’re publishing the same report to the Web Module each day, you can use the Dynamic Reports tab to select a date range and a department (or whatever organizational groups you have defined) and the Web Module will collate all the daily reports that match that filter into one report. This allows you to report on entire week, month or year by simply ‘reporting on reports’, rather than reporting months of raw storage data.

Here’s the full list of changes since the last auto update (2.2.0.32 on the 14th April 2010).

Application Changes

• Added Dynamic Reports feature to the Web Module.
• Rewrote the Web Module transfer protocol. New protocol adds version checking, connection checking, and integrity checking for high latency environments.
• Purge data from storage task no longer prevents importing new hits when all data is removed from an input within a storage.
• IPv6 addresses now show IPv4-mapped addresses as plain IPv4 addresses in summaries.
• IPv6 and IPv4 addresses are now freely interchangable in filter expressions.
• Fixed IPv6 drilldowns on the Summaries screen
• SQL inputs can now be resumed from the previous position. Previously any input that was partially imported would be skipped when importing new hits.
• Template-based analysis has been fixed, no longer results in blank/non-existent analysis.
• Added new string manipulation functions to expression language; Contains, StartsWith, EndsWith, IndexOf.

Loader Changes

• Astaro: Now checks that the ID field is present in a line before attempting to read it.
• Barracuda Web Filter: Added this format to replace Spy Filter.
• BlueCoat Proxy SG W3C: Added support for gmttime, timestamp, x-bluecoat-surfcontrol-is-denied and x-bluecoat-transaction-id.
• ClearSwift: Added a new loader group for ClearSwift that includes the MimeSweeper loaders
• ClearSwift SECURE Web Gatway: Now supported with the Web Appliance loader
• Clearswift Web Appliance: User summary displays Source IP if Username is blank.
• IronPort WSA: Fixed memory usage issues.
• Microsoft FTMG: Added category name lookup to SQL loader.
• Microsoft FTMG: No longer fails to import lines where the rule field contains square brackets.
• Microsoft FTMG: URL Category field is now a string instead of an integer. Added URL Categorization Reason field.
• Microsoft FTMG: Fixed memory usage issues.
• Microsoft IIS W3C: No longer hangs or crashes when loading a file that isn’t IIS W3C.
• NetAsq: Added support for srcname field. The Username summary is populated with user first, and then srcname if user is blank. The User summary is also now populated with Source IPs if the Username summary is blank.

To update WebSpy Vantage, simple select Tools | Check for updates.

To update the Web Module, login to the Web Module server, right-click the WebSpy system tray icon, and select Check for updates.

As always, please contact us if you have any issues or questions.

WebSpy’s Industry Fit

WebSpy is a global leader in reporting and analysis on Internet activity when used in partnership with security vendors such as Microsoft ISA Server, Microsoft Forefront TMG, Cisco IronPort, Blue Coat, Sophos, Astaro, Barracuda, Squid Proxy, and many more.

Below image neatly summarizes how WebSpy report on log files from vendor devices in the Unified Threat Management (UTM) and Secure Web Gateway (SWG) sectors, and specifically focus on reporting and analysis on Internet activity aspects within the SIEM sector.

THE REASONS

1. WebSpy Adds Value to Existing Product Portfolio

There’s a multitude of reputable, solid and reliable security vendors that frequently form a part of our resellers’ product portfolio. Their network and security devices do a great job providing network structure and actively protecting against security issues.

However, analysis and reporting is not their forte, not their core, and more often than not reporting is only a feature within their complete network and security solution.

Gartner’s latest Magic Quadrants on SWG and UTM vendors (or “SMB Multifunctional Firewalls” as labelled by Gartner) clearly highlights the security vendor’s weakness in reporting.

Straight from the horse’s mouth, here’s some vendor reporting issues as highlighted by Gartner:

• “Lacking enterprise-class administration and reporting capabilities.”
• “Advanced ad-hoc reporting features are lacking and custom reports are limited to filter settings on existing reports.”
• “Reports are very basic, and there are only a limited number of pre-developed reports.”
• “Per-user reports and forensic investigations are weak.”
• “On-box reporting is very basic and requires Windows and SQL database licenses for the reporting server.”
• “The number of canned reports is low and some reports do not have obvious features, such as pie graph options. Some customers complained about the scalability of the reporting interface.”
• “Users describe the vendor’s reporting and alerting as difficult to use.”
• “Although management is strong, users cite quality of reporting as a deficiency.”

With this information at hand it comes as no surprise that resellers want WebSpy’s reporting solutions to complement and add value to existing Internet security devices and provide their clients with valuable, advanced, customized and scalable reports on the exact use of web servers, web proxy, servers, email server, firewalls, switches, routers, and spam and virus application.

2. WebSpy Helps Generate and Facilitate SWG and UTM sales opportunities

You’ll be surprised by the amount of clients who base the decision of which Internet security device to purchase on reporting abilities.

WebSpy has a proven track record of assisting both Internet security vendors, such as IronPort, Microsoft ISA Server, Sophos, and their resellers in securing sales of their Internet security appliances. On numerous occasions our resellers have been able to secure deals, which could have been lost to a competing vendor/reseller, simply because they were able to throw advanced reporting into the mix.

3. WebSpy Substantially Increase Sales Revenue through Add-On Sales

Our resellers have found that recommending WebSpy reporting with every Internet security and network installation gives them the ability substantially increase add-on sales revenue with limited efforts involved.

The fact we offer competitive upgrade rebates if a reseller’s client have already invested time and money in a competing third-party reporting solution, or on-appliance reporting, naturally makes the transition to WebSpy even smoother.

Not convinced? Have a look at these:

• 10 Reasons to report on ISA Server using WebSpy
• 10 Reasons to report on IronPort using WebSpy
• 10 Reasons to report on Sophos using WebSpy
• 8 Reasons NOT to Use Microsoft Forefront TMG’s Reporting

When imported into WebSpy Vantage, the result is shown in a new summary called ‘Group’ which you can add to your reports.

We also added support for the custom fields Bytes Sent and Bytes Received. Due to the absence of a header in the IronPort access log, Bytes Received and Bytes Sent fields must both be present to be detected, and the Received field must precede the Sent field.

We also added support for the custom fields Request Body Size and Response Body Size. These fields can be included in your access log by adding %q (Request body size) and %b (Response body size) in the ‘Custom Fields’ edit box. Due to the absence of a header in the IronPort access log, Request Body Size and Response Body Size fields must both be present to be detected, and the Request field must precede the Response field.

We’ve also noticed that the values in the Bytes Sent and Bytes Received fields do not necessarily add up to the value logged for ‘Size’. We’re discussing this issue with our friends at IronPort and we will hopefully post a solution or explanation soon..
The information we first received about these fields indicated they represented Bytes Sent and Bytes Received. This is the way they are represented in the builds below (2.2.0.29). We will release a new build soon, with the field names changed to Request body size and Response body size. Body size is different to bytes sent/received as it does not include bytes from packet headers etc.

We’re yet to issue an automatic update for the Vantage applications, so in the mean time you can download the latest builds here:

Vantage Ultimate:
ftp://ftp.webspy.com/webspy/Builds/VantageUltimate2.2.0.29.zip

Vantage Web Module:
ftp://ftp.webspy.com/webspy/Builds/VantageWebModule2.2.0.8.exe

Vantage Giga:
ftp://ftp.webspy.com/webspy/Builds/VantageGiga2.2.0.29.zip

Vantage Premium:
ftp://ftp.webspy.com/webspy/Builds/VantagePremium2.2.0.29.zip

To apply the Vantage update, close Vantage and extract the downloaded file into Vantage’s installation folder (Usually c:\Program Files\WebSpy\Vantage <flavour> 2.2). Overwrite the existing files.

To apply the Web Module update, uninstall the Vantage Web Module from Add/Remove Programs (Programs and Features in Windows 7/Server 2008), then run the downloaded exe file, making sure you specify the same server, virtual directory and data location that your Web Module was previously using.

We will be releasing this as a public auto-update soon. Let us know if you have any issues.

We take great pride in this versatility and the fact that our software is virtually vendor neutral, or universal. At the time of writing our software support 128 different vendors, and more than 250 log file formats. But what does all this actually mean? What’s all this log file format gibberish and why is better to use a universal log file analyzer than a reporting solution that can only analyze a limited set of log files?

Let’s break it down…

What is a Log File?

A log file is a set of data that is automatically created and maintained by a security or network device of activity performed by it.

Web proxy servers maintain log files listing details on every request, from outgoing traffic, made to the proxy server – who is accessing external sites, what sites are being accessed, when the sites were accessed, what page or search phrase referred the user to the sites, and the type and size of data downloaded from the sites. Email servers store log files containing data about the sender, the receiver, timing of delivery or receipt, subject line, and size of attachment. Firewalls, and other security devices, normally contain data about network activity and the external and internal traffic that has been blocked or filtered.

Log files contains bundles of information and are usually not very structured or easy to decipher. Here’s what a  log file can look like:

Can you see the need for a log file analyzer?

Who are the Network and Security Device Vendors?

There’s a bunch of them to say the least. A recent WebSpy customer survey showed that Microsoft, Novell, Squid, IronPort, Blue Coat and CheckPoint were the top vendors whose products our clients wanted to analyze.

Checkout the whole list (128) of vendors we support at http://www.webspy.com.au/resources/logformats.aspx

What is a Log File Format?

When we state we can analyze more than 250 different log file formats we take into account the different log files formats produced depending on the vendors’ product, product version and log type

For example: Microsoft develops products such as Exchange, IIS, Proxy Server and ISA Server. ISA Server comes in different versions (2000, 2004 and 2006). Each version can log and store different types of log files (file, MSDE Database and SQL Database). So, ISA Server MSDE Database 2004 is one log file format we support, ISA Server file 2000 is another.

Benefits of a Universal Log Analyzer

The 250 something log file formats WebSpy analyze and report on are simply the most common ones. It is very rare that we come across a client who need to analyze a log file that is not already on our list of supported log files. On the odd occasion this does occur, the client can simply request support for their specific log file format, our developers work their magic and wham – we support one more log file format.

Most competing log file analyzers are hard-coded to analyze a particular log file type. When this is the case their clients will need a different log analyzer to achieve each individual reporting requirement, increasing the time and costs involved to produce all the required reports.

WebSpy’s clients, on the other hand, reap the rewards of using one application to achieve all their reporting requirements, spending less on software maintenance, hardware and administration.

You should be prompted to update your software on startup, but if you’ve turned off that feature, simply go to Tools | Check for Updates.

New Features

This new build sports the following new features:

• Support for Microsoft Forefront Threat Management Gateway (Beta)
  Microsoft Forefront Threat Management Gateway (FTMG) is still currently in Beta, and is due to be released around November 2009. For those that do not know, FTMG is the next version of Microsoft’s popular ISA Server. Information and downloads for FTMG can be found here http://www.microsoft.com/forefront/edgesecurity/isaserver/en/us/tmg-beta.aspx. We have added support for FTMG beta 2 and 3 for both the W3C text logs (recommended) and the internal SQL Server Express Database logs. If you are currently trialling FTMG, we are very interested to hear your feedback. Let us know how you go!
  

  Now Supported - Microsoft Forefront Threat Management Gateway

• Data purge
  You can now purge data from a storage, and schedule this purge to occur on a regular basis using Tasks. Purge options include data between a date range, data before a date, data after a date, data older than a date relative to now, and all data. This feature will let you easily maintain a single storage that only includes data for the last month or day.
  

  Options for Purging data from your storage

• Import Organization from CSV can now be scheduled using Tasks
  If you are importing your organizational structure from CSV, you can now schedule this action using Tasks. This enables you to update your organizational structure before any reports are run.
  

  Import Organization from CSV via Tasks

• Added Support for ExoServer Web
  If you’re running ExoServer Web, you can now analyze it’s logs using WebSpy Vantage.

Fixes

We also fixed some things that may have been bugging you:

• Improved the start time for the application by improving the logic to check for Storage damage.
• Fixed the IronPort loader (Fixed out of range issues on excessive size fields).
• “Having” filters no longer override the sort order of a Report Template node.
• Fixed an issue that may result in duplicated storages after migrating settings from earlier versions.
• Fixed the inability to remove invalid entities from web module permissions list (users that no longer exist).
• Fixed a timeout issue when publishing storages to the web module.

Why are you still reading? Go update now!

Have fun!

The survey only consisted of six questions and to prevent influencing answers and encouraging unique thoughts and opinions, the majority of questions were open ended. Those answers applicable for categorization have now been categorized, compared and correlated. The survey was conducted for internal purposes but I would like to share some of the findings and comments we received.

Vendor devices and log formats

The top ten vendor devices client used WebSpy’s software to analyze and report on:

1. Microsoft ISA Server (a whopping 48.6%)
2. Microsoft Exchange
3. Novell – BorderManager
4. Squid – Proxy Server
5. Microsoft Proxy Server
6. Microsoft – Internet Information Services (IIS)
7. Blue Coat – Proxy SG
8. Check Point – Next Generation
9. IronPort – Web Security Appliance
10. Microsoft – Windows Event Logs

Business Issues

The most common reason why clients started to monitor and report on their log files:

1. To achieve transparency, higher visibility, on overall internet and network resources. (No actual issue quoted)
2. To reduce issues related to lost productivity, inappropriate and illegal internet usage
3. To be able to perform detailed investigations on specific user and site levels
4. To reduce bandwidth cost and increase speed
5. To achieve monitoring requirements not satisfied by vendor device
6. To reduce viruses and other security issues
7. To produce reports required by management
8. To trend and forecast Internet and network usage
9. To investigate email usage
10. To immediately address critical issues using real-time monitoring and alerts

Most valued WebSpy software features

1. Dynamic Drilldowns
2. Ad-hoc analysis and summaries screen
3. Real-time monitoring and alerts
4. Comprehensive and detailed reports and analysis options
5. Predefined and customizable reporting and analysis templates
6. Aliases – Logical grouping of data to represent it more meaningfully
7. Profiles – Categorize web site URLs, email subject lines, instant message chats and any other logged data using customizable keyword profiling technology.
8. Extensive log file compatibility and support
9. Ease of use
10. Automated task scheduling

Suggested Improvements

The majority of answers related to suggested improvements and added software features were very specific to individual user experiences and quite difficult to categorize. However, here a list of suggested improvements that our development has taken to heart and decided to prioritize:

Increase and Improve Number of Default Report and Analysis Templates
Major changes and improvements to our reporting engine and interfaces are currently being planned. A short term goal is to provide the ability to create reports in PDF format. This should hopefully be available in the coming months.

Longer term, we will be incorporating a visual report designer (drag / drop charts and tables etc), as well as the ability to create reports that collate information from different sources. For example, using the information from Event logs, Web proxy logs and email server logs, you could produce a report that shows when a user logged on to their PC, what sites they browsed, what files they accessed, and the emails they sent – all sorted chronologically in the one table!

We currently have a large range of report templates available, but choosing the one you want can be a time consuming process, and you will most likely want to customize the report once you’ve found one that suits. We are planning to make the whole process of creating the report YOU want much, much easier.

Further Expand Profiles (Website Categorization)
When it comes to website categorization, WebSpy applications utilize keyword profiling in addition to importing category fields from log files (if available). There are upsides and downsides to keyword profiling. A simple keyword can instantly categorize thousands of sites which is much more efficient than maintaining a URL Category database. Being able to categorizing hits to your own organization’s website or intranet is also a very useful feature.

We are committed to distributing more frequent profile updates, without overwriting any customizations you have made. Longer term, we plan to provide an option for collaborative keyword customization, so that you benefit from the customization that other customers are making around the world. We also plan to integrate third party categorization services, as there are many great organizations that focus purely on URL categorization.

Improve Software Updates Correspondence
Our main mass communication channels include our newsletter, press releases blog and twitter. Here is where we inform existing and prospective clients about our software, updates, offers and much more. Starting September 2009 we will also send out emails, on a monthly basis, informing existing clients about the specific software updates implemented and are ready to download.

Further Improve Reporting Speed
Vantage 2.2 ranges (Premium, Giga and Ultimate) now use multithreading techniques to utilize the extra processing power on machines with multiple cores or CPUs to import log files and generate reports faster. In addition to the multi-processing improvements recently made in Vantage 2.2, the development team are planning to implement even more multi threading improvements in the next major version. These improvements will be available for users of both the Analyzer and Vantage range.

Thanks again for your participation!

The winner of the USB Camera Pen has been picked using a random draw and will be contacted personally via email.