This video covers registering Vantage and the Web Module.

This video will cover the Organization section of the software, showing you how to import your users and groups from Active Directory / LDAP.

This video will take you through the system requirements, and the installation of Vantage and the Web Module.

We’ve just completed our ‘WebSpy Reporting for Cisco IronPort’ video. Although this video is aimed at the Cisco channel we’d thought we share it with the rest of the WebSpy community as it provides a great high level overview of Vantage Ultimate’s ability to, not only import and spit out reports, but to learn about an organization’s structure in order to securely deliver the right report to the right person at the right time.

Vantage Ultimate is structured in two parts. One windows application, that handles administration of importing log files, learning about your organizational structure, creating and running reports, and one web application (aka the web module) where users can securely log in to view the report that has been produced for them.

Have a look at the short video below to find out what WebSpy’s founder, Jack Andrys, and product operations manager, Scott Glew, have to say about WebSpy’s and IronPort’s compatibility, as well as a demo of Vantage Ultimate in action.

What’s the point with secure and easy report distribution you might ask? Well, among other things:

Distribute Information. Distribute Responsibility.

IT Managers are too often left with the responsibility of managing and enforcing acceptable Internet usage for the entire organization.

Using Vantage Ultimate administrators can distribute reports to managers or department heads that show the activity of their direct subordinates, placing the responsibility of enforcing acceptable usage with the managers themselves.

Empower Users. Full Disclosure.

You can use Vantage Ultimate to distribute reports to each member in your organization detailing their own personal Internet usage (sites visited, search terms used etc). This empowers users to modify their own behavior once they understand how much of their activity is productive or unproductive and what it costs the organization. It also serves as a gentle reminder to employees that their activities are being recorded and that they should keep their online behavior within the bounds of the organization’s acceptable use policy.

From a privacy point of view, Vantage Ultimate is an easy way to provide employees with full disclosure regarding the information recorded about them.

Self Serve, On-Demand Investigation.

You can also use Vantage Ultimate to distribute original data storages to any of your organization’s members, enabling them to conduct their own ad-hoc drilldowns on any information they require. For example, distribute storages to HR managers enabling them to investigate the activity of employees that have handed in their resignation notice, but still have access to confidential company resources.

Protect Employee Privacy.

By assigning permissions to each of your Vantage Ultimate users, you can ensure they can only view the reports they are allowed to view. However, some employees need to view the traffic of all users, but do not necessarily need to identify users. For example, a network administrator may need to view the amount of traffic downloaded from a particular site per user, but does not need to know which users where involved.

This is easily achieved using the ‘Identify users’ permission, which masks all usernames with a hash code.

For more information about WebSpy’s compatibility with Cisco IronPort and free 30 day trials, visit www.webspy.com/ironport

This release will be welcomed with open arms by many customers for the following reasons:

• General usability improvements in the Web Module
  Multi-select / delete options, Ajax progress indicators to avoid page refreshes, export from Dynamics Report tab and more (see below)
• Fixes to the Microsoft Forefront TMG loader
  See my other post: Microsoft Forefront TMG logs size fields the wrong way around. Also fixed ‘value cannot be null’ error when importing SQL logs.
• Fixes to storage corruption issues
  This build should prevent ‘Normalization Index’ storage corruption issues from occurring. This often occurred after importing data, editing some log inputs and reimporting.
• New loaders and more fixes
  See below for the full list

To update your Vantage application, simply choose Tools | Check for updates. To update the Web Module, right-click the WebSpy icon in your system tray and select ‘Check for updates’. If you have any issues updating the Web Module, please see my previous post Web Module Update Errors and Workarounds.

Web Module Changes:

• New: Task progress is now updated without refreshing the page
• New: Added multi-select / delete functionality to Reports, Analyses and Storages tables.
• New: Added export functionality to Dynamic Reports view.
• New: Added Performance section on the Options tab to enabling multi-processing (improves Analysis speed)
• Fix: Dynamic Reports view now supports Trend reports.
• Fix: Organization selector on Dynamic Reports view now always reflects updated data under IE6/7/8.Fix: Fixed javascript errors in IE when expanding the organization filter.
• Fix: Report template names are no longer truncated on the Dynamic Reports view.
• Fix: Fixed errors that may occur when collating reports on the Dynamic Reports page.
• Fix: Authentication errors are now logged with stack trace.

Vantage Changes

• Fixed: ‘Normalization index’ storage corruption problems.
• Fix: Report collation: Added support for collation of Min/Max aggregates on DateTime columns (time of first hit etc). Also added support for arrayed fields (for example, category fields with a comma separated list of categories)
• Fix: Import windows wizard now remembers settings for Import all or selected users
• Fix: Organization: Filtered LDIFs may now be imported when references to some users are missing (for example, if a user’s manager does not exist in the LDIF)
• Fix: Improved connection and error handling between Vantage and the Web Module.

Loader Changes

• New: BlueReef Sonar Total Management Module
• New: Microsoft Sharepoint 2007
• New: SmoothWall Guardian 7.0 format
• New: Sun One Proxy (Supported under Sun One Webserver)
• Fixed: Astaro: Improved format detection
• Fixed: Cisco: Strings in the IP fields of 113019 lines are now imported
• Fixed: IronPort WSA: Improved log format detection
• Fixed: Microsoft Exchange 2007: No longer raises issues regarding total-bytes or internal-message-id fields
• Fixed: Micorosft FTMG (Web) SQL: No longer encounters value could not be null errors
• Fixed: Microsoft FTMG: Added option to reverse bytes received/sent fields. See Microsoft Forefront TMG logs size fields the wrong way around
• Fixed: Microsoft IIS W3C: Now imports cs-method and connection ID
• Fixed: Sophos Web Appliance: Switched the outgoing and ingoing sizes so that they are now the correct way around
• Fixed: Fixed import new hits issue associated with W3C formats. You must reload your logs before this change will take affect. Formats affected include: BlueCoat, Clearswift, Microsoft Exchange 2007, Microsoft FTMG, Microsoft Windows Media Services, WebSpy Live Tracking Log

Enjoy!

• How to import your log files and explore the information recorded by IronPort using the Summaries screen
• How to open the customized IronPort Report Templates and Aliases
• How to generate reports
• How to import your organizational structure and report on departments
• How to setup the Web Module and publish reports

PART 1: Importing log files & exploring your IronPort summaries

Once you have exported your IronPort access logs (see http://www.webspy.com.au/vendors/ironport/howto.aspx#ftp), this video takes you through importing your logs into WebSpy Vantage and analyzing data on the Summaries screen.

PART 2: Opening the customized IronPort Templates & Aliases, and running reports

This video takes you through opening the IronPort-specific report templates and aliases and generating a report that provides an overview of your organization’s Internet usage.

PART 3: Importing your Organization structure & generating department reports

This video shows you how to import your organizational structure into WebSpy Vantage from a directory server (such as Active Directory) using LDAP, and then generating a report that contains information on your newly imported departments.

PART 4: Using the Web Module.

This video takes you through configuring and using the WebSpy Vantage Web Module. Specifically, it takes you through the following tasks:

• Configuring the Web Module for Windows Authentication
• Adding a Web Module to Vantage
• Publishing reports to the Web Module
• Adding permissions for a user
• Synchronizing the Web Module
• Using the Dynamic Reports tab

PART 5: A quick word about tasks & conclusion

This video summarizes the actions taken in the previous four videos and also briefly discusses how to automate the reporting processing using scheduled tasks.

I hope this helps! Let me know if you have any questions by leaving a comment below.

Unfortunately there are two issues with the Web Module auto update process. Everyone on a 64 bit operating system will encounter issue #1 (Unable to locate installation location), and some of you may encounter issue #2 (System.IO.FileLoadException).

This article describes the errors and how to work around them to successfully update the Web Module.

Issue #1

The usual process to update your Web Module is to log into your Web Module server, right-click the WebSpy system tray icon and select ‘Check for updates’.

If you do this on a 64 bit operating system, you will receive the following error:

Web Module Updater error: Unable to locate installation location

You can fix this issue by clicking Yes, and specifying the Web Module’s installation location, which is usually somewhere under c:\inetpub\wwwroot (or just c:\inetpub\wwwroot if you didn’t specify a virtual directory when installing).

This will allow the updater to continue and you will be prompted to download and install the latest update.

Issue #2

Unfortunately, you may encounter another error during the update installation. The text of the error will be something along the lines of:

System.IO.FileLoadException: Could not load file or assembly ‘ICSharpCode.SharpZipLib, Version=0.84.0.0, Culture=neutral, PublicKeyToken=1b03e6acf1164f73′ or one of its dependencies. The located assembly’s manifest definition does not match the assembly reference. (Exception from HRESULT: 0×80131040)

Work Around

We are currently working on solutions to both of these issues. In the mean time, here is a work around to install the update. On the Web Module server:

1. Download this file:
   http://update.webspy.com/autoupdate/files/vantagewebmodule/vantagewebmodule2.2.0.18.zip
2. Stop IIS (See instructions below).
3. Right-click the WebSpy system tray icon and click Exit.
4. Backup your existing Web Module installation by copying everything in your Web Module’s installation folder (usually under c:\inetpub\wwwroot (or just c:\inetpub\wwwroot if you didn’t specify a virtual directory when installing) into a completely separate location (i.e. don’t keep it in a sub-folder).
5. Extract the downloaded zip file to your web module’s installation folder. Overwrite all the existing files.
6. Start IIS (see instructions below).

Your Web Module will now be updated to the latest version.

Stopping and Starting IIS

In IIS Manager (Start | Control Panel | Administrative Tools | Internet Information Services (IIS)) , right-click the site you want to start or stop, and click Start or Stop

We sincerely apologize for the inconvenience and will hopefully have a solution out soon. If you have any problems with the process above, please contact support at webspy dot com.

Here’s a quick video outlining some of the differences between TMGs Reporting, and what can be achieved using WebSpy Vantage. The video does not illustrate all the limitations outlined below, so please read on.

Whats is in the Forefront TMG report?

The default TMG report contains the following sections

• Summary
• Web Usage
• Application Usage
• Traffic and Utilization
• Security
• Malware Protection
• URL Filtering
• Network Inspection System

Each section contains overviews such as ‘Top users’ and ‘Top Sites’.

If your reporting requirements can be satisfied with these overviews – that’s great! Unfortunately, when you start thinking about what system administrators and other people in your organization actually need to make informed decisions, this report is quite limiting.

The 8 Limitations of Microsoft Forefront TMG’s Reporting

Here is what I consider to be the 8 main limitations of Microsoft Forefront TMG’s reporting functionality.

1. No Drilldowns

Want to see the sites that the top 5 users accessed? Want to see the users that downloaded the most traffic from youtube? These are fairly standard reporting requirements that simply cannot be achieved using the inbuilt TMG reporting.

WebSpy Vantage lets you either interactively drilldown into a user or site, or produce a regular report that includes further details about what your top users have actually been up to.

2. No Filtering

When you generate a report in TMG, you can only filter the report by a date range. There is no way to filter out anonymous (unauthenticated) traffic or exclude traffic coming from advertising servers (such as doubleclick and 2mdn.net) that tend to dominate most of the top 10 sites.

This can easily be achieved using WebSpy’s software. Check out my video on how to remove clutter from your web reports.

3. No Customization

Customization of each overview in the TMG report is limited to the number of items to show (e.g. top 5 or top 50 users), and the sort order (Incoming Bytes, Outgoing Bytes, Requests and Total Bytes).

What about the time a user spent browsing the web, or the number of users that visited a specific site? There is no way to add custom columns such as total browsing time, average session time, or number of users/sites/IPs to the report tables.

Or say you simply want to change your top users chart from a bar to pie to easily see the percentage used. Nope sorry!

If you do make one of the two available customizations in a TMG report, you then get the annoying Apply / Discard message to save changes to the configuration database.

All of these customizations can be achieved using WebSpy Vantage, and it doesn’t touch your TMG server to apply a change to a report.

4. Limited Report Distribution

When you generate a report, you get the option to email it to a specific email address. What if you would like to create a report for every department, and then email it to the managers of each department? Or better yet, host the report on a secure web server where department managers can log in and view their reports?

WebSpy Vantage Ultimate comes with a secure ‘Web Module’ specifically for this purpose and managers still receive a link to the report via email.

5. Cluttered ‘Top Sites’ List

The ‘Top sites’ list can become particularly cluttered due to the inclusion of sub-domains. I don’t want to mentally add up the size values from farm1.static.flickr.com, farm2.static.flickr.com, and farm3.static.flicr.com – I just want to know how much was downloaded from flickr.com.

This is compounded by the inability to exclude sites that are merely placing advertising banners on the actual sites users are visiting (as mentioned in the ‘No Filtering’ limitation above).

WebSpy Vantage breaks URLs down into separate components and lets you analyze each part separately. Look at the Site Domains summary to remove sub-domains and see only flickr.com. Or perhaps you want to see the keywords a user entered into search engines like Google? Or perhaps the top pages accessed within a website? No problem. Just include the Site Keywords or Site Resource summaries in your Vantage reports.

6. No Grouping or Aliasing

There is no way to group users into departments or locations, or IP addresses into subnets, or extensions such as .html, .pdf or .exe into file types. The ability to group and represent raw log data in more meaningful ways, as offered by WebSpy Vantage, can increase the value of a report tremendously.

7. No Productivity Assessment

One of the major features introduced in TMG since ISA Server 2006 is the included URL categorization technology.

Although the TMG report gives you an overview of the categories that have been visited, the report does not use this information to display a productivity assessment for your users.

WebSpy Vantage not only provides this assessment, but also the ability to customize the categories that are deemed productive as this can vary wildly depending on the industry and organization.

8. Not browser independent

This is a minor limitation that can be a major annoyance. The report that TMG produces is a HTML report that only displays correctly in Internet Explorer. As Forefront TMG is a Microsoft product, this is not exactly surprising, but still very annoying if IE is not your default browser.

How to get awesome reports from Forefront TMG

If you have had personal experience with any of the above limitations, you’ve probably been hunting for an alternative solution. I strongly recommend checking out the WebSpy Vantage range of products, and if you would like secure report distribution via the ‘Web Module’, Vantage Ultimate is what you are after.

If you agree or disagree with anything in this article, I encourage you to leave your thoughts in the comments.

Cheers!

Scott

In the mean time, you can do it by editing an XML file manually.

If you go to the Web Module’s data folder you will find a file called “Vantage Web Module.Reports”.  If you open this in Notepad, you’ll notice chunks of xml for each report:

<WebReport>
<Guid>89208266-42e5-44bc-baa2-157c404c9688</Guid>
<Title>Business Unit Report</Title>
<Date>633945813293343143</Date>
<Type>Analysis</Type>
<Access>
<Everybody>False</Everybody>
<Attributed>True</Attributed>
<Specific>True</Specific>
<SpecificEntities>
<item>person:CN=Luke,OU=Users,OU=Australia,OU=Webspy,DC=wsy,DC=com</item>
</SpecificEntities>
<Managers>True</Managers>
<ManagerLevelRestriction>3</ManagerLevelRestriction>
</Access>
<Attribution>person:CN=Scott,OU=Users,OU=Australia,OU=Webspy,DC=wsy,DC=com
</Attribution>
</WebReport>


Depending on how you published your reports, the unique ID of the person that currently has access to the reports will be mentioned in either the ‘SpecificEntities’ or ‘Attribution’ section.

You just need to find/replace this with the unique ID of the person you would like to transfer these reports to. You can find the unique ID of a person on the Organization screen in Vantage.  It’s called ‘Distinguished Name’:

Finding a user's unique ID in Vantage

A few more things:

• Make a backup of your original “Vantage Web Module.Reports” file before making any change.
• As you can see above, people need to be entered into this XML file using the syntax
  “person:<uniqueID>“
• You will also need to stop IIS before making any change as the web module caches this data in its memory while running.

As mentioned, creating a user interface to do this is a planned feature so follow us on Twitter, or subscribe to our RSS feed for updates!

In the mean time, I thought I’d share a way to re-brand the main elements in the Web Module by editing a few files and replacing a few images.

The only issue with this technique is that any future auto-updates for the Web Module will overwrite your edited files, so you just need to keep a copy of your customized files, so that you can restore them again after the auto-update.

Before you begin

In order to edit anything, you first need to know where your Web Module is located on your web server’s hard drive. This can be found by opening IIS Manager (Start | Control Panel | Administrative Tools | Internet Information Services (IIS) Manager) expanding the left hand server/site tree to find your Web Module.

• In IIS7, select the Web Module and click Basic Settings… in the right hand ‘Actions’ panel. The location is specified in ‘Physical Path’.
  

  Finding the Web Module's physical path in IIS7

• In IIS6, right-click the Web Module and select Properties… then go to the Home Directory tab. The location is specified in ‘Local Path’.

Windows may also prevent you from editing these files directly due to permissions issues. I’ve found a good technique is to copy the files you want to edit to your desktop, edit them, and then copy them back into the Web Module’s physical path. Windows will then prompt you to elevate to administrator and the copy/replace will succeed.

Ready To Go…

There are a few places where the WebSpy logo and WebSpy Text is presented.

• The login page
• The header bar
• The welcome Page
• Report cover pages

The Login page

The logo displayed on the login page can be found at /images/logo.png. Replace this image with your own logo. Then open Default.aspx in the Web Module’s root folder in a text editor such as notepad, and replace the following line

<img runat=”server” alt=”WebSpy” src=”~/Images/Get.ashx?image=Logo” />

with

<img runat=”server” alt=”WebSpy” src=”~/Images/logo.png” />

Before

Web Module's Login Page Before logo.png Change

After

Web Module Login Page After logo.png change

The header bar

The header bar utilizes the image located a /Images/bauble.png. Replace this image with your own custom image.

Then open Navigation.Master in the Web Module’s root folder in a text editor such as notepad, and replace the following line

Also look for the text:

Before

Web Module's Header Bar Before Bauble.png and Text Changes

After

Web Module's Header Bar After Bauble.png and Text Change

The Welcome Page

When you first login to the Web Module, you are presented with a Welcome Page. The first line on this page reads “Welcome to the WebSpy Vantage Web Module. You can change this by editing the first line in the Welcome.aspx file located in the Web Module’s root folder. Edit the section in bold below:
<%@ Page Language=”C#” MasterPageFile=”~/Navigation.Master” AutoEventWireup=”true” CodeBehind=”Welcome.aspx.cs” Inherits=”WebSpy.Vantage.WebModule.Welcome” Title=”Insert Custom Text Here” %>

Before

Web Module's Welcome Page Before Text Change

After

Web Module's Welcome Page After Text Change

The Report Cover Pages

The Image used on the cover page of reports is much easier to change.

1. Login to the Web Module as Administrator
2. Go to the Options Tab
3. Click ‘Report Logo’ under Web Module Options
4. Click Choose File, and select the image or logo you would like displayed on your report cover page
5. Click Upload

Before

Web Module's Report Cover Page Before Report Logo Change

After

Web Module's Report Cover Page After Report Logo Change

Summary

The changes above cover a majority of the areas your users will come into contact with in the Web Module. There may be a few more instances of the word “WebSpy” but for the most part, it should just be a matter of opening the relevant .aspx file and editing the html.

As I mentioned, if you auto-update the Web Module (via the system tray icon on the Web Module server), your edited files will be overwritten. I recommend keeping a copy of your edited files in a safe place outside the Web Module’s physical folder, so that you can copy them back in after the update. If the only changes you make are the ones above, then you’ll need to keep a copy of:

• /Navigation.Master
• /Default.aspx
• /Welcome.aspx
• /Images/logo.png
• /Images/bauble.png

We will also be adding the functionality to make these changes ‘properly’ in a future build, so follow us on Twitter, or subscribe to our RSS feed for updates!

Cheers!

To use Windows Authentication, there are just a few things you need to do.

1. Set the Web Module’s Authentication type to IIS Integrated, and add your administrators in the form of domain\username
2. Enable Windows Authentication and disable Anonymous authentication in IIS.
3. Ensure all users in your Organization screen have a login name in the form of domain\username. Use the ‘Prefix’ option to prefix “domain\” (without the quotes) to your usernames names when importing your Organization from LDAP or LDIF.
4. Connect Vantage and the Web Module using the new authentication details and synchronize your Organization.

1. Set the Web Module’s Authentication type to IIS Integrated, and add your Administrators.

When you first install the Web Module, the first screen you see is the ‘Initial Configuration Wizard’ that guides you through the process of selecting your authentication type and specifying your administrator(s). If you have already been through this Wizard and are currently using Vantage In-Built or Client Certificate authentication, you can easily reset this initial configuration wizard. Simply login to the Web Module with your current administrator details and go to Options | Maintenance | Reset Initial Configuration Wizard.

Note: You can also change your authentication and administrator options individually using the Authentication and Administrator options on the Options tab of the Web Module.  However, for ease of demonstration, I’ll use the Initial Configuration Wizard method.

Now that you’re at the Initial Configuration Wizard, proceed through the wizard, selecting IIS Integrated authentication and entering your administrators in the form of domain\username (replace domain with your organization’s AD domain, and username with the sAMAccountName of your administrator.

Initial Configuration Wizard - Welcome Page

Initial Configuration Wizard - Authentication Page

Initial Configuration Wizard - Delegate Administrators Page

Initial Configuration Wizard - Summary Page

Click Finish, and if the authentication was successfully changed, you should get a message saying ‘The specified credentials were not accepted”.

The 'Specified credentials were not accepted' message.

Don’t panic at this point. This message is an indication that the authentication was successfully changed and that the Web Module is now listening for IIS to pass through Windows Usernames. The reason you’re getting this message is because IIS is not yet passing through Windows Usernames to the Web Module. This is configured in the next step.

2. Enable Windows Authentication and disable Anonymous authentication in IIS.

Now that the Web Module is expecting IIS to authenticate your users, you need to set up  IIS  to do this.

1. Open IIS by navigating to Start | Control Panel | Administrative Tools and double-clicking on Internet Information Services (IIS) Manager.
2. Navigate to the Web Module site or virtual directory in the left hand ‘Connections’ Panel. It will be located under <Server Name>\<Sites>. For example, MyServer->Sites->Default Web Site->webmodule.

• If you’re running IIS7 ( Windows Server 2008, Vista or Windows 7)
  1. Select the Web Module site and ensure the ‘Features’ tab is selected at the bottom of the middle pane.
  2. Double-click the ‘Authentication’ feature.
  3. Right-click ‘Anonymous Authentication’ and select Disable
  4. Right-click and ‘Windows Authentication’ and select Enable
  5. Restart IIS by selecting your server in the right hand connections pane, and clicking Restart in the ‘Actions’ pane on the right.

• If you’re running IIS6 or 5.1 (Windows Server 2003, Windows XP)
  1. Right-click Web Module site and select Properties.
  2. Go to the Directory Security tab
  3. Under ‘Authentication and access control’ click the Edit button.
  4. Uncheck ‘Enable anonymous access’ and check ‘Integrated Windows authentication’
  5. Restart IIS by right-clicking the local server, select All Tasks, and then click Restart IIS.

If you added your own Windows login name as an administrator in step 1, you can now test the authentication is working. Go back to the Web Module in your browser and click Refresh. You will be presented with an ‘Authentication Required’ dialog where you can enter your username and password.

Authentication Required Dialog

Again, ensure your username is in the form of domain\username. Click OK, and you should log straight into the Web Module using Windows Authentication.

3. Ensure all users in your Organization screen have a login name in the form of domain\username

Now your administrator account can log into the Web Module using Windows Authentication, but all other users will not be able to log in unless they have their login name specified in the form of domain\username. This is done in Vantage Ultimate on the Organization screen.

Organization Screen showing correct login name for Windows Authentication

If you’re importing your users from LDAP or LDIF, make sure you use the ‘Prefix’ option on the User Details page to prefix domain\ before your imported usernames. For example:

4. Connect Vantage and the Web Module using the new authentication details, and synchronize your Organization.

In order to publish information to the Web Module, you need to add a connection between Vantage and the Web Module. This is done on the Web Module screen in Vantage Ultimate.

1. Click Add Web Module (or if you already had a web module before changing the authentication details, select it and click Properties)
2. Enter the server & virtual directory of the Web module, and enter the correct credentials ensuring domain is specified.
3. Click OK to connect.

Connect to Web Module dialog

Once connected, synchronize Vantage with the Web Module by clicking the Synchronize link in the Web Module task pad. You may also want to provide permissions for your users in the Permissions section on the Web Module screen.

That’s it. You can now test that everything is working by getting one of your users to access the Web Module’s URL. They should sail straight in with no username/password prompt.

I hope this helps! Please let me know your feedback by emailing me, or leaving a comment below.