This is a great update for Vantage Ultimate users as we’ve introduced a new feature/tab into the Web Module called ‘Dynamic Reports’.

If you’re publishing the same report to the Web Module each day, you can use the Dynamic Reports tab to select a date range and a department (or whatever organizational groups you have defined) and the Web Module will collate all the daily reports that match that filter into one report. This allows you to report on entire week, month or year by simply ‘reporting on reports’, rather than reporting months of raw storage data.

Here’s the full list of changes since the last auto update (2.2.0.32 on the 14th April 2010).

Application Changes

• Added Dynamic Reports feature to the Web Module.
• Rewrote the Web Module transfer protocol. New protocol adds version checking, connection checking, and integrity checking for high latency environments.
• Purge data from storage task no longer prevents importing new hits when all data is removed from an input within a storage.
• IPv6 addresses now show IPv4-mapped addresses as plain IPv4 addresses in summaries.
• IPv6 and IPv4 addresses are now freely interchangable in filter expressions.
• Fixed IPv6 drilldowns on the Summaries screen
• SQL inputs can now be resumed from the previous position. Previously any input that was partially imported would be skipped when importing new hits.
• Template-based analysis has been fixed, no longer results in blank/non-existent analysis.
• Added new string manipulation functions to expression language; Contains, StartsWith, EndsWith, IndexOf.

Loader Changes

• Astaro: Now checks that the ID field is present in a line before attempting to read it.
• Barracuda Web Filter: Added this format to replace Spy Filter.
• BlueCoat Proxy SG W3C: Added support for gmttime, timestamp, x-bluecoat-surfcontrol-is-denied and x-bluecoat-transaction-id.
• ClearSwift: Added a new loader group for ClearSwift that includes the MimeSweeper loaders
• ClearSwift SECURE Web Gatway: Now supported with the Web Appliance loader
• Clearswift Web Appliance: User summary displays Source IP if Username is blank.
• IronPort WSA: Fixed memory usage issues.
• Microsoft FTMG: Added category name lookup to SQL loader.
• Microsoft FTMG: No longer fails to import lines where the rule field contains square brackets.
• Microsoft FTMG: URL Category field is now a string instead of an integer. Added URL Categorization Reason field.
• Microsoft FTMG: Fixed memory usage issues.
• Microsoft IIS W3C: No longer hangs or crashes when loading a file that isn’t IIS W3C.
• NetAsq: Added support for srcname field. The Username summary is populated with user first, and then srcname if user is blank. The User summary is also now populated with Source IPs if the Username summary is blank.

To update WebSpy Vantage, simple select Tools | Check for updates.

To update the Web Module, login to the Web Module server, right-click the WebSpy system tray icon, and select Check for updates.

As always, please contact us if you have any issues or questions.

When imported into WebSpy Vantage, the result is shown in a new summary called ‘Group’ which you can add to your reports.

We also added support for the custom fields Bytes Sent and Bytes Received. Due to the absence of a header in the IronPort access log, Bytes Received and Bytes Sent fields must both be present to be detected, and the Received field must precede the Sent field.

We also added support for the custom fields Request Body Size and Response Body Size. These fields can be included in your access log by adding %q (Request body size) and %b (Response body size) in the ‘Custom Fields’ edit box. Due to the absence of a header in the IronPort access log, Request Body Size and Response Body Size fields must both be present to be detected, and the Request field must precede the Response field.

We’ve also noticed that the values in the Bytes Sent and Bytes Received fields do not necessarily add up to the value logged for ‘Size’. We’re discussing this issue with our friends at IronPort and we will hopefully post a solution or explanation soon..
The information we first received about these fields indicated they represented Bytes Sent and Bytes Received. This is the way they are represented in the builds below (2.2.0.29). We will release a new build soon, with the field names changed to Request body size and Response body size. Body size is different to bytes sent/received as it does not include bytes from packet headers etc.

We’re yet to issue an automatic update for the Vantage applications, so in the mean time you can download the latest builds here:

Vantage Ultimate:
ftp://ftp.webspy.com/webspy/Builds/VantageUltimate2.2.0.29.zip

Vantage Web Module:
ftp://ftp.webspy.com/webspy/Builds/VantageWebModule2.2.0.8.exe

Vantage Giga:
ftp://ftp.webspy.com/webspy/Builds/VantageGiga2.2.0.29.zip

Vantage Premium:
ftp://ftp.webspy.com/webspy/Builds/VantagePremium2.2.0.29.zip

To apply the Vantage update, close Vantage and extract the downloaded file into Vantage’s installation folder (Usually c:\Program Files\WebSpy\Vantage <flavour> 2.2). Overwrite the existing files.

To apply the Web Module update, uninstall the Vantage Web Module from Add/Remove Programs (Programs and Features in Windows 7/Server 2008), then run the downloaded exe file, making sure you specify the same server, virtual directory and data location that your Web Module was previously using.

We will be releasing this as a public auto-update soon. Let us know if you have any issues.

Check your version in Help | About. If you are running 2.2.0.27 or above, then you already have this update. If not, make sure you update to your software by selecting Tools | Check for updates.

TMG or UAG? What is the difference?

TMG is an outgoing proxy that protects your internal users from malware, viruses and the like. TMG generates some great web proxy log files to import into WebSpy Vantage allowing you to monitor where your users are going on the Internet, how much they’re downloading etc.  TMG, unlike ISA, now has deep packet inspection for HTTPS traffic, plus a bunch of other new features.

UAG is an incoming proxy that provides employees, partners and vendors secure remote access to corporate resources such as Outlook Web Access (OWA) and Sharepoint (MOSS). It utilizes the TMG engine, but this is mainly just to protect the UAG server (more on this topic here http://technet.microsoft.com/en-us/library/ee522953.aspx).

TMG can also publish your OWA and MOSS sites, but this is no longer recommended by Microsoft. They recommend using a dedicated UAG server to perform this function.

Upgrading to TMG

If you’re thinking about migrating your ISA server (2004 or 2006) to TMG, you may like to check out this migration guidance video with Mohit Saxena (Senior Technical Lead) and Jim Harrison (Program Manager). http://edge.technet.com/Media/ISA-to-TMG-Migration-Guidance/

Microsoft Forefront TMG Migration Video

Reporting on TMG

If you’re using TMG at the moment, we invite you to analyze your web proxy and/or firewall logs using WebSpy Vantage and tell us what you think!  Download your copy of WebSpy Vantage here, and import your logs using the Microsoft FTMG format:

Microsoft Forefront Threat Management Gateway

Supported on Windows 7

The following WebSpy products support Windows 7:

• WebSpy Analyzer (Standard, Premium and Giga)
  Analyzer customers will need to download the latest version of their product as support for Windows 7 was uploaded today (9th November 2009).
• WebSpy Vantage (Premium, Giga and Ultimate)
  All products within the Vantage range have officially supported Windows 7 since version 2.2 was released on  24th June 2009.
• WebSpy Insight for Microsoft SBS Premium
  Although Insight for Microsoft SBS Premium does not mention Windows 7 support in its documentation, it will install and run on Windows 7 without issue.

Not ‘Officially’ Supported on Windows 7

WebSpy Live, Sentinel and FlowMonitor are not yet officially supported on Windows 7 and will be updated soon. In the mean time, here are some instructions on how to get these products working on Windows 7 right now.

WebSpy Live

The existing version of Live will install and run on Windows 7 with one minor issue. When you run the application, there will be no Triggers, Aliases, or Profiles available.

To fix this:

1. Install and run WebSpy Live on Windows 7.
2. Shutdown WebSpy Live by right-clicking the Live icon in the Windows system tray and selecting ‘Shutdown’
   

   Shutting down WebSpy Live on Windows 7

3. Download the following file containing WebSpy Live’s default Triggers, Aliases and Profiles:
   WebSpy Live Default Files (zip – 19.7 KB).
4. Extract the zip file to C:\Users\<your user profile>\AppData\Roaming\WebSpy\Live 2.2 and overwrite the existing files.
5. Run WebSpy Live. You will now be able to see a list of Triggers, Profiles and Aliases on the respective Configuration screens in WebSpy Live.

WebSpy Sentinel

WebSpy Sentinel will not yet install on Windows 7 due to an issue with the included version of WinPCap (the packet driver used by Sentinel).

To fix this:

1. Download and install the latest version (4.1.1) of WinPCap from http://www.winpcap.org/install/default.htm
2. Install WebSpy Sentinel. The product should install and run without issue.

FlowMonitor

Unfortunately no work around is available for WebSpy FlowMonitor. We will update the product to work with Windows 7 as soon as possible.

Got a problem?

If you experience an issue running these products on Windows 7, or any other Windows operating system for that matter, please let us know!

You should be prompted to update your software on startup, but if you’ve turned off that feature, simply go to Tools | Check for Updates.

New Features

This new build sports the following new features:

• Support for Microsoft Forefront Threat Management Gateway (Beta)
  Microsoft Forefront Threat Management Gateway (FTMG) is still currently in Beta, and is due to be released around November 2009. For those that do not know, FTMG is the next version of Microsoft’s popular ISA Server. Information and downloads for FTMG can be found here http://www.microsoft.com/forefront/edgesecurity/isaserver/en/us/tmg-beta.aspx. We have added support for FTMG beta 2 and 3 for both the W3C text logs (recommended) and the internal SQL Server Express Database logs. If you are currently trialling FTMG, we are very interested to hear your feedback. Let us know how you go!
  

  Now Supported - Microsoft Forefront Threat Management Gateway

• Data purge
  You can now purge data from a storage, and schedule this purge to occur on a regular basis using Tasks. Purge options include data between a date range, data before a date, data after a date, data older than a date relative to now, and all data. This feature will let you easily maintain a single storage that only includes data for the last month or day.
  

  Options for Purging data from your storage

• Import Organization from CSV can now be scheduled using Tasks
  If you are importing your organizational structure from CSV, you can now schedule this action using Tasks. This enables you to update your organizational structure before any reports are run.
  

  Import Organization from CSV via Tasks

• Added Support for ExoServer Web
  If you’re running ExoServer Web, you can now analyze it’s logs using WebSpy Vantage.

Fixes

We also fixed some things that may have been bugging you:

• Improved the start time for the application by improving the logic to check for Storage damage.
• Fixed the IronPort loader (Fixed out of range issues on excessive size fields).
• “Having” filters no longer override the sort order of a Report Template node.
• Fixed an issue that may result in duplicated storages after migrating settings from earlier versions.
• Fixed the inability to remove invalid entities from web module permissions list (users that no longer exist).
• Fixed a timeout issue when publishing storages to the web module.

Why are you still reading? Go update now!

Have fun!

Well, it’s now been released and can be downloaded from the Microsoft Download Center:
http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=e05aecbc-d0eb-4e0f-a5db-8f236995bccd

One of the major improvements in Beta 3 is URL filtering which leverages Microsoft Reputation Services (MRS). With regards to this, Microsoft says:

“At the time of this release, the MRS database content is being populated and updated continuously as part of the initial beta service offering. As this process continues, URL filtering categorization accuracy and comprehensiveness will increase. A telemetry package designed for improving the quality of URL filtering database and collecting your feedback is planned to be released soon. Please check back for updates in August.”

Support for importing Microsoft TMG log files into your favorite WebSpy product is coming soon so stay tuned! Subscribe to the WebSpy blog RSS feed or follow us on twitter if you want to be notified as soon as it’s available.