The survey only consisted of six questions and to prevent influencing answers and encouraging unique thoughts and opinions, the majority of questions were open ended. Those answers applicable for categorization have now been categorized, compared and correlated. The survey was conducted for internal purposes but I would like to share some of the findings and comments we received.

Vendor devices and log formats

The top ten vendor devices client used WebSpy’s software to analyze and report on:

1. Microsoft ISA Server (a whopping 48.6%)
2. Microsoft Exchange
3. Novell – BorderManager
4. Squid – Proxy Server
5. Microsoft Proxy Server
6. Microsoft – Internet Information Services (IIS)
7. Blue Coat – Proxy SG
8. Check Point – Next Generation
9. IronPort – Web Security Appliance
10. Microsoft – Windows Event Logs

Business Issues

The most common reason why clients started to monitor and report on their log files:

1. To achieve transparency, higher visibility, on overall internet and network resources. (No actual issue quoted)
2. To reduce issues related to lost productivity, inappropriate and illegal internet usage
3. To be able to perform detailed investigations on specific user and site levels
4. To reduce bandwidth cost and increase speed
5. To achieve monitoring requirements not satisfied by vendor device
6. To reduce viruses and other security issues
7. To produce reports required by management
8. To trend and forecast Internet and network usage
9. To investigate email usage
10. To immediately address critical issues using real-time monitoring and alerts

Most valued WebSpy software features

1. Dynamic Drilldowns
2. Ad-hoc analysis and summaries screen
3. Real-time monitoring and alerts
4. Comprehensive and detailed reports and analysis options
5. Predefined and customizable reporting and analysis templates
6. Aliases – Logical grouping of data to represent it more meaningfully
7. Profiles – Categorize web site URLs, email subject lines, instant message chats and any other logged data using customizable keyword profiling technology.
8. Extensive log file compatibility and support
9. Ease of use
10. Automated task scheduling

Suggested Improvements

The majority of answers related to suggested improvements and added software features were very specific to individual user experiences and quite difficult to categorize. However, here a list of suggested improvements that our development has taken to heart and decided to prioritize:

Increase and Improve Number of Default Report and Analysis Templates
Major changes and improvements to our reporting engine and interfaces are currently being planned. A short term goal is to provide the ability to create reports in PDF format. This should hopefully be available in the coming months.

Longer term, we will be incorporating a visual report designer (drag / drop charts and tables etc), as well as the ability to create reports that collate information from different sources. For example, using the information from Event logs, Web proxy logs and email server logs, you could produce a report that shows when a user logged on to their PC, what sites they browsed, what files they accessed, and the emails they sent – all sorted chronologically in the one table!

We currently have a large range of report templates available, but choosing the one you want can be a time consuming process, and you will most likely want to customize the report once you’ve found one that suits. We are planning to make the whole process of creating the report YOU want much, much easier.

Further Expand Profiles (Website Categorization)
When it comes to website categorization, WebSpy applications utilize keyword profiling in addition to importing category fields from log files (if available). There are upsides and downsides to keyword profiling. A simple keyword can instantly categorize thousands of sites which is much more efficient than maintaining a URL Category database. Being able to categorizing hits to your own organization’s website or intranet is also a very useful feature.

We are committed to distributing more frequent profile updates, without overwriting any customizations you have made. Longer term, we plan to provide an option for collaborative keyword customization, so that you benefit from the customization that other customers are making around the world. We also plan to integrate third party categorization services, as there are many great organizations that focus purely on URL categorization.

Improve Software Updates Correspondence
Our main mass communication channels include our newsletter, press releases blog and twitter. Here is where we inform existing and prospective clients about our software, updates, offers and much more. Starting September 2009 we will also send out emails, on a monthly basis, informing existing clients about the specific software updates implemented and are ready to download.

Further Improve Reporting Speed
Vantage 2.2 ranges (Premium, Giga and Ultimate) now use multithreading techniques to utilize the extra processing power on machines with multiple cores or CPUs to import log files and generate reports faster. In addition to the multi-processing improvements recently made in Vantage 2.2, the development team are planning to implement even more multi threading improvements in the next major version. These improvements will be available for users of both the Analyzer and Vantage range.

Thanks again for your participation!

The winner of the USB Camera Pen has been picked using a random draw and will be contacted personally via email.

All three products in the Vantage range (Premium, Giga and Ultimate) have been updated with the following features:

• Multi-processing
  Vantage now utilizes the extra processing power on machines with multiple cores or CPUs to import log files and generate reports faster.
  
  

  The new Performance tab in Vantage

• Improved report styles
  HTML (including MHT) reports are now sporting an updated look and feel.
  
  

  New Report Style

• Updated Aliases and Profiles
  Added support for the latest search engines, social networking sites, operating systems and user agents.
  
  

  Profiles Aliases support the latest Internet tools and technologies.

• Storage Repair Utility
  Storages can become damaged if there is a system crash or if WebSpy.Vantage.exe process is ended when an import is in progress. Damaged storages are now detected automatically and you have the option of repairing them. Storages can also be manually checked and repaired on the Storage properties dialog.
  
  

  Storage Repair Utility

• Save / Open Tasks
  Scheduled Tasks can now be saved to an external file (.Tasks) for backup and migration purposes.
  
  

  Save and Open Tasks

Obviously, the multi-processing feature is the big one! Many of you (Vantage customers) have been running Vantage on multi-core or multi-cpu machines and just dying for Vantage to grab hold of the CPUs and make them work. You should be happy with this new build. Import a folder full of log files and watch 6 to 8 logs import simultaneously instead of watching them import one after the other. Run a full analysis or a comprehensive report with no filters and watch your CPU jump into gear to generate the report in 30% – 50% of the time. We’re expecting this to be a very popular update, especially among our larger customers.

Multi-processing is a great new feature, but be aware that there are some circumstances where Vantage will not be able to utilise more CPU power when generating reports. For example, if your report contains filters that significantly cuts down the amount of data in your storage that needs analysis, Vantage’s analysis engine and your CPUs will not be significantly pushed. Also see my earlier blog for more information on how the report structure can also affect the performance.

To get hold of this build, simply download the latest version of Vantage from our website. It will run side by side with your existing Vantage 2.1 installation, which minimizes any downtime, and makes it easy to transfer all your settings across (use the Migration Wizard in the Tools menu). Then you can uninstall Vantage 2.1 when you’re ready.

Let us know how you go by leaving comments below, or by tweeting us.

Cheers!
Scott

The multi-processing build features a new tab called ‘Performance’ in Tools | Options. Ensure ‘Use multi-processing’ is checked, and set the ‘Maximum Concurrent Threads’ value to twice the number of logical/physical CPUs you have. For example, set it to 4 on a dual core, 8 on a quad-core, or 16 on a machine with two quad-core CPUs. Feel free to also play around with this value, but we have found this formula to be optimal and we’ll automatically set this as default in future builds.

The new Performance tab in Vantage

You’ll notice that if you import a folder of logs, about 6-8 logs will import simultaneously. Simply importing one log file will not show any speed improvement.

Multi-processing will only benefit report generation if your report template contains side-by-side summaries as opposed to drilldown summaries. Here’s what I mean by a template with side-by-side summaries:

A report template that only has side-by-side summaries

And here’s what I mean by a template with drilldown summaries

A report template with only drilldown summaries

Basically, each side-by-side summary will get processed simultaneously, but any drilldowns will be processed sequentially.

Most report templates consist of both side-by-side AND drilldown summaries, such as:

A report template with both side-by-side and drilldown summaries

The amount that these templates benefit from multi-processing will depend on the number of side-by-side summaries.

If this sounds like something you’d like to try, please contact me for a copy of the build!