This release will be welcomed with open arms by many customers for the following reasons:

• General usability improvements in the Web Module
  Multi-select / delete options, Ajax progress indicators to avoid page refreshes, export from Dynamics Report tab and more (see below)
• Fixes to the Microsoft Forefront TMG loader
  See my other post: Microsoft Forefront TMG logs size fields the wrong way around. Also fixed ‘value cannot be null’ error when importing SQL logs.
• Fixes to storage corruption issues
  This build should prevent ‘Normalization Index’ storage corruption issues from occurring. This often occurred after importing data, editing some log inputs and reimporting.
• New loaders and more fixes
  See below for the full list

To update your Vantage application, simply choose Tools | Check for updates. To update the Web Module, right-click the WebSpy icon in your system tray and select ‘Check for updates’. If you have any issues updating the Web Module, please see my previous post Web Module Update Errors and Workarounds.

Web Module Changes:

• New: Task progress is now updated without refreshing the page
• New: Added multi-select / delete functionality to Reports, Analyses and Storages tables.
• New: Added export functionality to Dynamic Reports view.
• New: Added Performance section on the Options tab to enabling multi-processing (improves Analysis speed)
• Fix: Dynamic Reports view now supports Trend reports.
• Fix: Organization selector on Dynamic Reports view now always reflects updated data under IE6/7/8.Fix: Fixed javascript errors in IE when expanding the organization filter.
• Fix: Report template names are no longer truncated on the Dynamic Reports view.
• Fix: Fixed errors that may occur when collating reports on the Dynamic Reports page.
• Fix: Authentication errors are now logged with stack trace.

Vantage Changes

• Fixed: ‘Normalization index’ storage corruption problems.
• Fix: Report collation: Added support for collation of Min/Max aggregates on DateTime columns (time of first hit etc). Also added support for arrayed fields (for example, category fields with a comma separated list of categories)
• Fix: Import windows wizard now remembers settings for Import all or selected users
• Fix: Organization: Filtered LDIFs may now be imported when references to some users are missing (for example, if a user’s manager does not exist in the LDIF)
• Fix: Improved connection and error handling between Vantage and the Web Module.

Loader Changes

• New: BlueReef Sonar Total Management Module
• New: Microsoft Sharepoint 2007
• New: SmoothWall Guardian 7.0 format
• New: Sun One Proxy (Supported under Sun One Webserver)
• Fixed: Astaro: Improved format detection
• Fixed: Cisco: Strings in the IP fields of 113019 lines are now imported
• Fixed: IronPort WSA: Improved log format detection
• Fixed: Microsoft Exchange 2007: No longer raises issues regarding total-bytes or internal-message-id fields
• Fixed: Micorosft FTMG (Web) SQL: No longer encounters value could not be null errors
• Fixed: Microsoft FTMG: Added option to reverse bytes received/sent fields. See Microsoft Forefront TMG logs size fields the wrong way around
• Fixed: Microsoft IIS W3C: Now imports cs-method and connection ID
• Fixed: Sophos Web Appliance: Switched the outgoing and ingoing sizes so that they are now the correct way around
• Fixed: Fixed import new hits issue associated with W3C formats. You must reload your logs before this change will take affect. Formats affected include: BlueCoat, Clearswift, Microsoft Exchange 2007, Microsoft FTMG, Microsoft Windows Media Services, WebSpy Live Tracking Log

Enjoy!

Microsoft TMG Log showing Bytes Sent consistently larger than Bytes Received

This issue has been confirmed by the Microsoft Forefront TMG team, and unfortunately there is no ETA for a fix.

We obviously don’t want our reports showing incorrect usage figures, so we’ve fixed our TMG loader so that it imports the ‘bytesrecvd’ field into the Bytes Sent aggregate, and the ‘bytessent’ field into the Byte Received aggregate.

But what if Microsoft release a fix? What we’ve done is implemented a loader property to allow you to turn off this behavior. This will allow you to import your old logs with the fields reversed, and your new logs with the fields the right way around.

To access the loader property:

• On the import wizard, select the Microsoft FTMG format and click the Properties button on the toolbar
• Select Microsoft FTMG from the drop down list
• Notice the option to ‘Reverse Bytes Sent and Received to compensate for bug in TMG’s logging’. Leave this checked until Microsoft issue a fix.

Microsoft Forefront TMG Loader Option to Reverse Bytes Sent and Received

This fix is available in Vantage build 2.2.0.48 (and above) which has been released as an auto update. So simply select Tools | Check for updates to ensure you have this fix.

Richard has been working with Forefront Threat Management Gateway (TMG) 2010 and its predecessors for more than 12 years. He has designed and deployed network security solutions using TMG and ISA for SMB’s, military and defense organizations, and Fortune 500 companies around the world.

In addition to his isaserver.org blogs, Richard has his own ISA/TMG blog where he recently posted some useful tips on changing WebSpy Vantage’s scheduled task recurrence interval using the schtasks.exe command line tool. Adding more frequent import options (i.e. hourly) is on the product roadmap but until then, using the command line tool is a great alternative.

We do recommending visiting tmgblog.richardhicks.com – brimming with ISA Server and TMG information and tips, here’s just some of the latest blogs:

• Changing WebSpy Vantage Scheduled Task Recurrence Interval
• Websense Integration Support for Forefront Threat Management Gateway (TMG) 2010
• Load Balancing and Forefront TMG Firewall Clients
• How to Slipstream Service Pack 1 for TMG

So let’s use WebSpy Vantage to drill into that Anonymous user and find out what is going on.

One way to do this is to run an Ad-hoc analysis on the Summaries screen and drilldown into the Anonymous user to view all the information about that user. However, TMG and ISA tend to log a lot of information that may not be relevant to this particular investigation, so I’ve created some report templates (one for ISA and one for TMG) and a set of Aliases that pull out some relevant information.

Download our Anonymous Traffic Investigation Report

If you’re running WebSpy Vantage download the Anonymous Traffic Report Templates & Aliases

Then open the .Templates file on the Reports tab, and the .Aliases file on the Aliases tab. Once you have both files opened, go to the Reports tab and click either the ‘Anonymous Traffic Investigation (ISA)’ or the ‘Anonymous Traffic Investigation (TMG)’ report. Then click the ‘Generate report’ link and run the report template on your ISA or TMG storage.

The report gives you the ability to drill into the Allowed, Denied and Failed traffic to see a list of the unauthenticated IPs, Sites, Rules responsible for blocking or allowing the traffic, unauthenticated Applications and Result Codes.

Main causes of anonymous traffic

What you will probably find is that most of the Anonymous traffic is being denied by your TMG or ISA firewall. When a client first requests a web page, the proxy will challenge the client for authentication. These events are often logged with the result code 12209 meaning ‘authorization is required to fulfill the request’. These requests are therefore denied by the proxy until the client’s credentials are authenticated.

Have a look at the amount of traffic being denied and then checkout the Result Codes associated with the denied traffic. Chances are you’ll see ‘proxy authentication required’ appear predominantly.

If you also look at the Applications section you may also find that Windows Updates are sailing through your TMG or ISA firewall unauthenticated.

Filter out unauthenticated traffic from Reports

The most logical next step is to filter out the information you do not want in your reports. You’ll probably still want to include Windows Update traffic in your reports, but you’re probably not so interested in the ‘proxy authentication required’ information. So let’s filter that out.

To do this:

1. Go to the Reports tab and select the report you want to filter (such as your Organization report)
2. Click ‘Edit Template’, then click ‘Template Properties’.
3. In the filter section at the bottom of the dialog, click Add | Field value filter.
4. Select the ‘Result Code’ summary and select the Status Code Names (ISA-FTMG) alias.
5. On the toolbar, search for Authorization, and check the following two items:
   • The server requires authorization to fulfill the request. Access to the Web Proxy filter is denied.
   • The server requires authorization to fulfill the request. Access to the Web server is denied. Contact the server administrator.
6. Ensure the ‘Exclude’ radio button is selected and click OK.

If you decide that you don’t care about seeing ANY unauthenticated traffic in your reports, you can always simply filter out the Anonymous user from your reports.

To do this:

1. Go to the Reports tab and select the report you want to filter (such as your Organization report)
2. Click ‘Edit Template’, then click ‘Template Properties’.
3. In the filter section at the bottom of the dialog, click Add | Field value filter.
4. Select the ‘Username’ summary.
5. On the toolbar, click Add and type ‘anonymous’. Click OK.
6. Ensure the Exclude radio button is selected and click OK.

Hopefully this article improves your understanding of the ‘anonymous’ user, and gives you some actions to take for your specific reporting situation.

If you have any questions, please leave a comment below.

Trend Micro found that the use of social networking sites grew globally from 19% in 2008 to 24% in 2010.The fact that the use of social networks is growing, and will most likely continue to grow, probably doesn’t come as a surprise to anyone, but it is always interesting to attach real figures to this trend.

The report doesn’t imply whether this growth relates to the use of social networks in a productive manner to drive businesses, or employees wasting companies’ time and money. Regardless of the reasons behind the growth, Trend Micro warned that without proper oversight, the increased use merely makes organizations more viable as malware distribution points.

“Social networking is an extremely important tool both for personal and professional relationship building,” David Perry, global director of education at Trend Micro, said in a statement. “While most companies’ concerns around social networking in the office center around the loss of employee productivity, what they may not realize is that many social networking sites are built on interactive technologies that give cybercriminals endless opportunities to exploit end users, steal personal identities or business data and corrupt corporate networks with malware.”

Trend Micro are pretty clear (and we couldn’t agree more) that blocking social networking sites is not the solution. Not only is blocking counter-productive, cause employee resentment and can increase costly turnovers, the report also states that “Trying to just prevent users accessing social networks from work could potentially increase the risk to an organization as users look for ways around computer security, possibly increasing the chance of exposure to security threats”.

Damned if you Don’t, Damned if you Do

Many organizations are still fumbling in the dark when it comes to the best approach for handling, and effectively embracing, social networking. On the one hand, it is distracting for employees, productivity can definitely suffer from excessive usage and organizations become more vulnerable to cybercriminals, data leakage, malware etc. On the other hand, social networking is a great way to communicate with customers, generate leads, build brand reputation, increase SEO and can have beneficial impact on productivity if used in moderation.

Recommendations

Sadly there is no one-size-fits-all solution that can smoothly be implemented in every organization. According to a recent Symantec survey only 5% of organizations block social networking sites outright. One-third doesn’t block but do have policies stating that social networks can only be used for business purposes. Meanwhile, 42% of organizations have no policy or blocking whatsoever.

As with any other internet related issue, the high level social networking best practices should include:

1. Educating the whole organization about security threats related to social networking
2. Establishing (and communicating) social networking acceptable usage policy that is regularly updated to keep relevant
3. Monitor to ensure policy adherence
4. Re-educate

Please bear in mind that the monitoring aspect isn’t just to make sure employees spend no more than their acceptable time on social networking sites. The software also acts as a great tool to find out who in your organization may need additional assistance in identifying threats, such as phishing sites (read blog on how one of our clients used a phishing attack as an opportunity to educate the workforce), or simply to verify the configuration of the network’s firewall and other threat management systems.

Don’t hesitate to get in touch with any questions or comments.

Majority of security vendors will give you a high level overview of the categories, such as Sports, Shopping, Online Community, Streaming Media, Employment and Gambling, but rarely provides intuitive ways to further investigate the traffic going to the sites within these categories. The nifty thing about WebSpy’s solutions is that, as long as categories are logged, you can use WebSpy to analyze web browsing in relation to these categories and get a much clearer overview of your organization’s web usage.

Classify Productive & Unproductive Categories

Assessing productivity in relation to predefined categories is what I would like to focus on today. I have imported and run an analysis on TMG logs using WebSpy Vantage. As previously mentioned, you can import logs from any security device we support – if the information is in the log file WebSpy can report on it.

TMG logs contain information whether traffic has been ‘Allowed’, ‘Denied’ or ‘Failed’. Using WebSpy Vantage you can easily drill down further into this information. For example, let’s say I’m interested in having a look what categories have been allowed, i.e. not blocked, I simple expand the ‘Allowed’ node and click ‘URL category’.

Allowed Categories - Click to Enlarge

This information is great but it doesn’t tell us anything about productivity. WebSpy Vantage not only provides this assessment for your entire organization, specific department and individual users, but also gives you the ability to customize the categories that are deemed productive as this can vary wildly depending on the industry and organization.

How?

You use WebSpy’s Aliases feature to sort categories in relation to your organization’s view of their productiveness. Our software comes with a default list of aliases so you can either edit these or set up new aliases. I’ll take you through the process of setting up an Alias from scratch.

1. Creating a New Alias

• Click on the Alias tab and select ‘New Alias’ in the top left corner
• Name your Alias something appropriate and provide a short description. I’ll name mine ‘Productivity’.
• Make sure ‘Apply alias to selected summaries’ option is checked
• Click ‘Schema’ to specify the log file type and scroll down to the bottom of the list to locate and select ‘URL Category’.
• Tick the ‘Group unresolved into a single name’ box and name it something appropriate. Let’s go with ‘Uncertain’.

 

2. Add Alias Groups

Once an alias has been added, you need to add alias groups. You can have as many alias groups as you want but for this purpose it makes sense to have only two, ‘Productive’ and ‘Unproductive’. There might be certain categories, such as ‘Education/Reference’ or ‘Blogs/Wiki’, that might be difficult to correctly deem as productive or unproductive and you’d rather not specify. If this is the case you don’t need to add an alias group as it will automatically be created for any category that hasn’t been grouped under the other alias groups. Remember how we ticket ‘Group unresolved into a single name’ and called it ‘Uncertain’ before.

• Click the Add Group button in the Groups task pad.
• Enter the desired alias group name (Productive) in the ‘Key’ edit box and click OK. Repeat steps for the ‘Unproductive’ group.
• At this stage you could also add items (categories) to your group but I’m going to show you another way of adding categories.

   

  

3. Add Categories to your ‘Productive’ and ‘Unproductive’ Alias Groups

This is where customization really works its charm. What is deemed as unproductive at one company might be completely legit and considered productive at another. For example, in a recruitment company one could assume it would perfectly normal for employees to visit other employment sites but this could be considered personal and unproductive at a hospital or real estate agent.

There’s a few different ways of adding items to an Alias group. While still in the Alias screen you can click ‘Refresh Unassigned’ in the top right part of your screen. Because you haven’t assigned anything yet all categories will be displayed. From here you can simply highlight the category group, for example ‘Unproductive’ and Ctrl + click all categories you want to place in that group. Once you’ve selected your categories right click and select ‘Add to selected group’. Repeat the process to add categories to your ‘Productive’ group.

Alternatively, you can go back to the ‘URL Category’ listings in the ‘Summaries’ tab and Ctrl + click selected categories, right click and select ‘Add to alias’, select your ‘Productivity’ alias from the drop down menu and select the ‘Productive’ or ‘Unproductive’ group.

4. Assess Productivity

With aliases, groups and items set up you’re ready to assess productive and unproductive browsing. In the ‘Summaries’ screen, left hand side under ‘Aliases’, simple select your ‘Productivity’ alias and the URL categories will be sorted in accordance with your view of their productiveness.

Productive vs Unproductive Browsing - Click to Enlarge

You can also investigate further by, for example, drilling down to determine what unproductive categories are most popular, what are the most popular unproductive websites within those categories, what hours during the day majority of unproductive sites are accessed (you might have a policy that allows personal web browsing during lunch hours), and of course who spends the most time on unproductive websites within your organization.

Top Unproductive Websites - Click to Enlarge

The previous Soho Alpha version will stop working on the 30th of June 2010 – to continue testing you need to download the latest version.

If you haven’t downloaded WebSpy Soho Alpha before simply get the latest version and off you go. If you’re an Alpha tester you need to keep reading…

WINDOWS USERS:

Before installing the new build, Windows users must uninstall the previously installed version of Soho Alpha on every machine. Thanks to a bug in the previous version, you need to follow the two steps below to do this:

STEP 1 – UNINSTALL FROM START MENU

• Windows Vista/7 – Start | Control Panel | Programs and Features
• Windows XP – Start | Control Panel | Add/Remove Programs

STEP 2 – UNINSTALL MANUALLY

1. Open an administrative command line window
• Windows Vista/7: Go to Start | All Programs | Accessories, right-click on Command Prompt and select "Run as administrator".
• Windows XP: Go to Start | Run, type "cmd" and press OK
2. In the command line window that appears, type the following commands:
   • sc stop "WebSpy Soho Agent"
   • sc delete "WebSpy Soho Agent"
3. The service has now been removed from the system, and the new version may be installed without issue

MAC USERS

Simply launch the new install package and it will automatically overwrite the old one.

LATEST UPDATES:

• Now installs on Mac OS 10.5
• More stable – less crashing and freezing
• Plenty of work has been done based on your feedback. We’re hoping that the below issues are more or less fixed but would really appreciate if you could let us know if you’re still experiencing:
  • High CPU usage after sleep or hibernate
  • All computers disappearing from current activity chart except the local computer
  • Random traffic spikes

FEEDBACK

Thank you to everyone that has submitted feedback so far.

• Please let us know if your network card works or doesn’t work with Soho
• Have a feature suggestion or found a bug? Submit it and vote it up here
• Dedicated Soho Alpha Feedback thread in our forums

Thanks for being involved in our alpha testing phase. We hope you enjoy using the product and we look forward to your comments!

Here’s some news-worthy statistics you might have come across recently:

Employee Computer & Internet Abuse Statistics

• 30 to 40% of Internet use in the workplace is not related to business
• 64% of employees say they use the Internet for personal interest during
  working hours
• 70% of all Internet porn traffic occurs during the nine-to-five work day
• Internet abuse costs Irish SMBs €580 million per year
• The FIFA World cup will cost American companies about $121.7 million in lost productivity

Your Organizational Costs

So what does this mean to your organization? What percentage of the above costs does your company contribute? Quick, locate information on the number of companies in your country from the local bureau of statistics and divide it with the latest unproductive Internet usage costs …or… click link to nifty little ‘Annual Cost Calculator’ below.

Annual Cost Calculator for Unproductive Internet Usage

Annual Cost Calculator - Unproductive Internet Usage

How to use it

All you need to do is to:

• Click image link above to navigate to calculator
• Enter number of employees in your organization
• Enter average employee cost per hour
• Click ‘Calculate’

The calculator will work its magic and display an overview for different yearly cost scenarios depending on minutes of unproductive browsing per day and the percentage of employees engaging in unproductive Internet usage.

• Average Employee Cost per Hour: If you’re not sure about your average employee cost the US Department of Labor – Bureau of Labor Statistics estimated it to be approximately $29 per hour in March 2010 (including salary, overhead costs, benefits, payroll taxes, etc.).
• Percentage of Employees Engaging in Unproductive Internet Usage: Latest statistics estimates that approximately 64% of employees use the Internet for non-work related interest during work hours.
• Unproductive Usage – Minutes per day: Latest poll figures indicates at least 1-2 hours (60 – 120 minutes) per day.

Limitations

• You must navigate away from this page to be able to use it. Sorry, this a php blog platform, a language I am far from fluent in or would ever dream of trying to incorporate with JavaScript. I’m a marketing person for crying out loud – cut me some slack.
• It only takes unproductive costs into consideration. To get the true number you also need to consider bandwidth costs, legal liability costs (such as sexual harassment or hostile workplace law suits because you, as an employer, didn’t do anything to prevent inappropriate or illegal Internet usage), costs related to network disruptions or slowdowns because all your employees are streaming the World Cup, and so on.
• It doesn’t consider other human factors. It isn’t an artificially intelligent calculator that can simulate a real office environment and factor in variables such as motivation and how a certain amount of unproductive (or ‘Workplace Internet Leisure Browsing’ (WILB) as coined by Melbourne University study) can actually increase employee concentration levels and helps make a more productive workforce.

• How to import your log files and explore the information recorded by IronPort using the Summaries screen
• How to open the customized IronPort Report Templates and Aliases
• How to generate reports
• How to import your organizational structure and report on departments
• How to setup the Web Module and publish reports

PART 1: Importing log files & exploring your IronPort summaries

Once you have exported your IronPort access logs (see http://www.webspy.com.au/vendors/ironport/howto.aspx#ftp), this video takes you through importing your logs into WebSpy Vantage and analyzing data on the Summaries screen.

PART 2: Opening the customized IronPort Templates & Aliases, and running reports

This video takes you through opening the IronPort-specific report templates and aliases and generating a report that provides an overview of your organization’s Internet usage.

PART 3: Importing your Organization structure & generating department reports

This video shows you how to import your organizational structure into WebSpy Vantage from a directory server (such as Active Directory) using LDAP, and then generating a report that contains information on your newly imported departments.

PART 4: Using the Web Module.

This video takes you through configuring and using the WebSpy Vantage Web Module. Specifically, it takes you through the following tasks:

• Configuring the Web Module for Windows Authentication
• Adding a Web Module to Vantage
• Publishing reports to the Web Module
• Adding permissions for a user
• Synchronizing the Web Module
• Using the Dynamic Reports tab

PART 5: A quick word about tasks & conclusion

This video summarizes the actions taken in the previous four videos and also briefly discusses how to automate the reporting processing using scheduled tasks.

I hope this helps! Let me know if you have any questions by leaving a comment below.

The Australian Attorney-General’s Department recently confirmed ongoing discussions about implementing a data retention regime in Australia requiring ISPs to hold customers web browsing history and private emails and make both available on request from government agencies. Industry insiders said the regime being considered by the Australian Government could see data held for up to ten years, much longer than EU Directive time of 24 months.

While reading this, looming in the back of mind is the labour party’s proposed mandatory internet filtering policy. The proposed policy was originally aimed at keeping children safe online but in reality it involves blacklisting and blocking a plethora of online material deemed inappropriate by the government. Opponents of the policy don’t dispute the worth of providing tools to help parents protect their children, but take issue with the expense, side-effects and technical issues of this scheme. Find out more from Electronic Frontiers Australia’s filtering fact sheet.

Although using a slightly different approach, the ultimate goals behind the two regimes is making the internet a safer place, circumventing crimes and facilitating investigation of suspected criminals. But is snooping on individuals’ private browsing and controlling access to material really the way to go? Regardless of the questionable effectiveness and obvious drawbacks, such as costs, there are some major concerns regarding privacy and freedom. Do we really want the government to decide what we can view online and have the ability to access our own personal browsing history? I am aware the intentions are good but it does send chills down my spine.

In relation to the data retention scheme, Electronic Frontier Australia (EFA) chair Colin Jacobs said, “At some point data retention laws can be reasonable, but highly-personal information such as browsing history is a step too far”. Jacobs added, “You can’t treat everybody like a criminal. That would be like tapping people’s phones before they are suspected of doing any crime.”

Jacobs does raise valid concerns. Where do you draw the line? Is accepting government access to everyone’s browsing history a precursor to tapping people’s phones? Setting up surveillance cameras in everyone’s home? Thought police?

Extract from Orwell’s 1984:
“There was of course no way of knowing whether you were being watched at any given moment. How often, or on what system, the Thought Police plugged in on any individual wire was guesswork. It was even conceivable that they watched everybody all the time. But at any rate they could plug in your wire whenever they wanted to. You had to live—did live, from habit that became instinct—in the assumption that every sound you made was overheard, and, except in darkness, every movement scrutinized.”

Alright, I know we’re far from the dystopia described in Orwell’s 1984 but I can’t help drawing parallels between the personal privacy lost to the state. Orwell once explained that the scene of the book (1984) is laid in Britain in order to emphasize that the English-speaking races are not innately better than anyone else, and that totalitarianism, if not fought against, could triumph anywhere.

Side Note

WebSpy is pro-internet access and provide businesses, government departments and educational organizations an alternative to blocking and filtering software. We emphasizes that organizational internet usage should be managed using an honest and open monitoring approach where acceptable internet usage policies are clearly communicated to employees and students.

Employers need to ensure their internet resources are used in a productive, secure and legal manor. Private filtering and monitoring by a government is a completely different kettle of fish and we have deliberately refrained from public comments on Australia’s proposed filtering policy. However, with the new discussions on government access to private web browsing and email records we felt the need to at least raise a few concerns and if nothing else point out the difference between private and organizational internet reporting.

Whilst we do see benefits in families and individuals monitoring their own internet usage and bandwidth cost, as a company we do NOT support government legislation allowing national filtering or access to private browsing records.

What are your thoughts?