This video covers registering Vantage and the Web Module.

This video will cover the Organization section of the software, showing you how to import your users and groups from Active Directory / LDAP.

This video follows on from #3, and will cover the Profiles section of the software, and how to use site categorization.

In this video we’ll look at how to import some data into a storage, use the Summaries section, and start customizing Aliases.

This video will take you through the system requirements, and the installation of Vantage and the Web Module.

This video is a high level overview, giving you a general insight into the software and its different parts.

This time around I thought I share some tips focusing on how individuals can ensure they’re being smart and safe when engaging in social networking activities. Hopefully these tips can assist you, or the people working in your organization, in avoiding being exposed to the dark side of social networking where hackers, identity thieves, fraudsters and stalkers are lurking in the shadows.

I’ve picked five tips I thought were quite helpful and perhaps not as obvious as not sharing personal information such as birth date and address, keeping passwords safe, be selective whose friendship requests you accept, use appropriate virus/phishing software etc.

1. Limit work history details on LinkedIn

Although tempting, it is not advisable to use LinkedIn as an online resume displaying all information about time and place with previous employers and educational institutions.

Too much personal information publicly available simply makes it too easy for identity thieves to use the information to fill out loan applications or guess password security questions. A hacker recently used social reverse engineering to find information, such as post code, home town, high school graduated from etc, in order to hack into VP candidate Sarah Palin’s web mail account.

If you really feel the detailed information will help you when looking for a new job then expand the details during the job hunting process and cut back later after you have a position.

LinkedIn also offers some capabilities to restrict information. You can close off access by others to your network of contacts, something you don’t have to share if you don’t want. This is a common practice by sales professionals and recruiters not wanting to expose their valuable network to others who might poach customers or prospects from them.

2. Avoid sharing location information

Hopefully you’re already aware that sharing phone and address details online is a very bad idea. You might also think twice before posting “Away for the weekend…returning on Monday” on public sites like Twitter and LinkedIn, to avoid attracting attention from potential stalkers or burglars.

What you might not have considered is that, over time, seemingly innocent and non-specific information can be pieced together, giving lurkers a much more complete and rich picture of you, your family, your habits and other personal information.

Software like Twitter and Foursquare are often used at conferences, parties and other social scenes where alcohol is consumed, increasing chances of personal and location information slipping out. Travel plans and experiences while on holiday is often communicated via Twitter, giving clue to others that you’re not at home, leaving your family or possessions at risk for intruders.

Your Foursquare check-ins to local restaurants and shops can also make it fairly simple to determine what area you live in and your habits. Something as innocent as a Foursquare airport check-in is a tell-tale sign you’ll be away from home for at least one night.

3. Search yourself

Sometimes referred to as vanity search, it is a very good idea to search your name on Google and check out your profile as others see it on social networking sites. Understand where you show up and what information is available about you, and then adjust your profile, settings and habits appropriately.

If you unexpectedly see your name in locations you don’t frequent, it could give you a heads up someone else is using your identity online. Set up a Google alert with your name. Google Alerts will email you weekly, daily or immediate notifications when the Google robots comes across your name online.

4. Setup an OpenID account

OpenID is an open source standard for creating a single sign-on to multiple online services and applications. With OpenID, your password is only given to your identity provider, and that provider then confirms your identity to the websites you visit. Other than your provider, no website ever sees your password, so you don’t need to worry about an unscrupulous or insecure website compromising your identity

As a framework, OpenID accounts are available from multiple providers. Companies like AOL, Microsoft, Sun, and Novell are beginning to accept and provide OpenIDs. It is estimated that there are over 160-million OpenID enabled URIs with nearly ten-thousand sites supporting OpenID logins.

OpenID is making inroads into the SaaS application market to better manage user accounts. We’re also likely to see OpenID used in online social networking sites to help verify users identities and reduce impersonators and false identities. If the social networking sites you frequent don’t use OpenID or a similar technology, e-mail the site creator and lobby for adding it.

5. Don’t violate your company’s social networking policies

As blogging and social networking sites enter the workplace corporate acceptable use policies (AUP) are being updated to define boundaries for employees, contractors and the company.

Data leakage incidents (loss of corporate, confidential or customer information), making inappropriate public statements about the company, using corporate resources for personal uses and harassing or inappropriate behavior toward another employee can all be grounds for reprimand or dismissal. Social networking sites are another way those things can happen and they create an easy digital paper trail to investigate.

Data leakage (or loss) prevention is currently one of the hottest areas in security. Companies are looking for ways to prevent company confidential and proprietary information from slipping through the firewall. Most incidents probably occur via email or file transfers but IM chat tools, blog posts, Twitter messages and even online resume content could disclose proprietary company information.

Even using social networking sites on company time or using company resources could be a violation of the company’s acceptable use policy. Before you become the corporate poster child for some publically humiliating episode from using social networks at work, check your corporate AUP to make sure you aren’t violating the policy.

Other Resources
http://www.microsoft.com/protect/parents/social/socialnet.aspx
http://www.networkworld.com/community/tips-for-safe-social-networking

You can also read through these steps on this page: Analyzing SonicWALL log files with WebSpy.

Creating and Importing SonicWALL log files

Analyzing SonicWALL log files

We intend to make some SonicWALL specific report templates available on our SonicWALL how to page soon.

Until then, feel free to create your own templates, or modify our existing web reports to include the extra goodies contained in the SonicWALL logs.

TIP: To modify an existing web report, right-click the report and choose ‘Duplicate template’. Then choose the “SonicWall Web” schema. You’ll then have a report template that you can modify to include all the SonicWALL summaries, such as Categories, and Source and Destination Interface.

If you need some assistance getting the report(s) you need, feel free to contact me, or support@webspy.com.

Take a look at our dedicated Astaro pages to get an idea of what can be achieved when analyzing Astaro Web Gateway log files with WebSpy Vantage.

I’ve created some quick videos to show you how to enable the correct logging options on the Astaro Security Gateway appliance, how to import these log files into Vantage, and analyze the data on the Summaries screen.

Configure Logging

The best way to configure logging is to setup a 3rd party syslog server (such as Kiwi Syslog) on a machine in your network, then configure the Astaro Security Gateway to send syslog messages to that server. The syslog server then creates log files that can be imported into WebSpy Vantage. This video takes you through that process.

Importing and Analyzing Astaro logs

Once you have successfully configured syslogging on your Astaro Security Gateway, you can import the log files into WebSpy Vantage and analyze activity on the Summaries screen. This video takes you through that process.

We intend to make some Astaro specific report templates available on our Astaro How To page soon.

Until then, feel free to create your own templates, or modify our existing web reports to include the extra goodies contained in the Astaro logs.

TIP: To modify an existing web report, right-click the report and choose ‘Duplicate template’. Then choose the “Astaro Security Gateway – Filter with category” schema. You’ll then have a report template that you can modify to include all the Astaro summaries, such as Actions and Categories.

If you need some assistance getting the report(s) you need, feel free to contact me, or support@webspy.com.

From not wanting to touch social media with a ten foot pole, employers are these days increasingly aware of the benefits of social media and web 2.0 in the workplace. Clearswift used an independent market research firm and interviewed approximately 250 online office workers and 150 managers across the UK, US, Australia and Germany.

Social Media and Web 2.0 Benefits According to Managers:

• More than half (52%) of managers think web collaboration is critical for the future success of the company.
• 91% believe it can help increase brand awareness.
• 89% believe it can help in generating new business.
• 88% believe it can help improving employee productivity.
• 47% of managers believe staff are ‘happier and more motivated’ as a result of using these tools in the workplace.

Web 2.0 and security: New approach needed?

It is naturally refreshing to see that organizations are realizing the benefits of web collaboration and social media tools. Filters and blocks are gradually removed, and the great organism that is the Internet is allowed to breath more freely. However, there’s always bad seeds, internally or externally, that will take advantage of this openness and use it to commit crimes, spread viruses, leak information and ultimately make it harder for organizations to embrace the splendor of web 2.0 and social media in it’s entirety.

According to Clearswift’s report:

• Security is the biggest Web 2.0 concern, with 61% of companies having voiced concerns about security as a
  result of social media.
• More than half (51%) of managers think employees are oblivious to security concerns when it comes to IT.
• 47% of companies have had at least one security incident as a result of internet application usage
• Only 64% have specific tools in place to secure Web 2.0 exchanges

Current popular approaches to Web 2.0 security issues typically involve “big brother” style monitoring and locking down social networking sites. Such approaches may serve to erode employment relationships and diminish business value to be gained from web collaboration. It is a positive sign, therefore, that 64% of companies recognize that a new approach to security is needed in this era of web collaboration.

A New Approach to Security

Clearswift SECURE Web Gateway and Clearswift SECURE Email Gateway are trusted by organizations globally to deliver internet security for business. They maintain productivity by enabling information to flow safely into and out of the workplace. Adding WebSpy reporting to the equation enables managers to provide employees with the Internet resources they need, while resting assured the resources are used as intended.

5 Tips for Effective Reporting while taking Workforce into Consideration

1. Allow employees to view their own Internet usage
   More often than not, employees tend to underestimate the time they spend browsing non-work related sites. Allowing employees to view, for example, their productive and non-productive activity can help foster and drive responsible Internet usage behavior.
2. Help employees sticking to the rules
   If you have set a limit of, for example, no more than 10 hours of recreational surfing per month, then ensure you alert employees when they are approaching that limit.
3. Distribute reports – distribute responsibility
   Frequently IT managers and administrators are given the ultimate responsibility of managing, enforcing and communicating acceptable Internet usage for an entire organization. Take some of the pressure off the IT department and distribute organizational Internet activity reports to responsible managers or department heads. This will enable them to see how Internet usage affects the security and performance of their own department and distributes the responsibility of enforcing acceptable usage with the managers themselves.
4. Protect employee privacy
   If distributing Internet usage reports across your organization it is important to protect employees’ personal data. Make sure you use reporting software designed to protect privacy rights by only allowing authorized users to see the employee’s identity. For instance, Network Administrators may need to investigate all traffic going to a particular site but should not need to know the user names – in this case
   user names should be anonymous for them but available for HR.
5. Automation
   Use a reporting solution that easily lets you customize and automate these guidelines for you

DOWNLOAD REPORT & RECOMMENDATIONS

Every organization is different with regards to the volume of logs it creates, but I’ve averaged three data sets submitted to us by customers to produce the following estimates.

You will create roughly 0.9 MB of IronPort WSA access logs per user per day.

Once imported into a WebSpy Storage, the Storage will be about 90% of your original log file size. If you apply NTFS compression to the storage folder, the actual size on disk of the WebSpy Storage will be about 30% of your original log data.

So an organization with 1000 users will produce about 900 MBs of access logs per day. The default WebSpy Storage  will be 810 MB, but with NTFS compression, the size on disk will around 270 MB.

As I said, this is a rough guide based on the average of three sets of sample logs we have in house, so please run your own tests and if you can, let us know your values in the comments below.

But there is another advantage to switching to text. They take up considerably less disk space. Here are some figures:

Number of Records in 235 MBs of log data:

235 MB of TMG’s W3C text logs contains 326,824 records. An SQL Express database of the same size (mdf and ldf files) contains only 40,308 records. In other words, w3C text logs can store over 8 times as much data in the same amount of disk space.

A rule of thumb:

By switching to W3C text logs, the disk space taken by your log files will be roughly 12% of the SQL Express or MSDE log files. This can be reduced even further by compressing your text logs.

• MSDE/SQL logs: budget for 5 KB per record
• W3C Text logs: budget for 0.71 KB per record

How many records your ISA or TMG server creates per day will depend on the number of users in your organization and how much traffic they generate, but about 16,000 records per user is a reasonable estimate.

A real world example

If you are hitting 500 GB of SQL Express/ MSDE logs per month (about 86,128,205 records), simply switching to W3C text logs will reduce this down to 61 GB.

Once imported into a WebSpy Storage, the storage size would be roughly 53 GB (87% of the original W3C text logs).

With NTFS compression applied to the Storage folder, the WebSpy Storage would be roughly 13.4 GB (22% of the original W3C text logs).

Applying NTFS compression to your WebSpy Storages folder is certainly a good idea. This does not impact performance. If anything, it may improve performance slightly as there is less disk fragmentation within the storage.

Disadvantages and Alternatives

Please be aware that by changing your logging to text, the default reporting functionality within TMG will no longer work. However, the reporting supplied by WebSpy Vantage should more than adequately replace this feature.

If you are still concerned about changing the logging method, you can utilize a script published by Microsoft to convert your SQL Express logs to W3C text.  You can then keep the text logs and set some more stringent data retention policies on the SQL Express logs, such as clearing logs every week. You can download this script as part of the Microsoft Forefront Threat Management Gateway (TMG) 2010 Tools & Software Development Kit.

Additional Resources

• Here’s a great article by Marc Grote at isaserver.org on the pros and cons of the different logging options in ISA and TMG. It also takes you through how to exclude fields to reduce the amount of data being logged:
  http://www.isaserver.org/tutorials/Microsoft-Forefront-TMG-Logging-options-Forefront-TMG.html
• Also take a look at Richard Hicks’ blog regarding MSDE performance with ISA Server 2006:
  http://tmgblog.richardhicks.com/2009/10/31/msde-performance-with-microsoft-isa-server-2006/
• Here’s another article on isaserver.org by Richard Hicks on the logging enhancements in TMG 2010
  http://www.isaserver.org/articles/Logging-Enhancement-Microsoft-Forefront-Threat-Management-Gateway-TMG-2010.html

The figures above were produced using some sample logs received from customers with similar (but not exactly the same) logging settings. If you have changed to text logging, I’d be very interested to hear the sort of disk savings you are seeing, and I’m sure others would to. So please leave a comment below.

As the term ‘hit’ can sometimes be confusing, let’s start off by properly define hits.

What is a hit?

WebSpy classifies a hit as an individual file or item that has passed through your logging device and been recorded in a log file.

The actual content and size of a hit can vary widely. A hit may consist of a small picture or text file, a large zip file or executable, or any other individual file. A hit can be a file downloaded from an Internet site, an email that’s been sent or received, a file downloaded from an FTP site and so on.

When dealing with web log files, one web page can be made up of many hits – the main page, the pictures on the page, the files on the page and so on. In some situations, a user cannot control the content of the hits they access, such as in the case of advertising pop-up messages.

How many hits on one page?

Here’s just an example of number of hits when browsing to some of the more popular sites out there:

• facebook.com – 75 hits
• apple.com/iphone – 69 hits
• youtube.com – 29 hits
• google.com – 4 hits
• yahoo.com – 40 hits
• twitter.com – 67 hits

These hit counts are just from one user (me) visiting a few sites. Imagine the amount of hits/data you’ll get for an organization with 100, 1,000 or 10,000 users!

Also interesting is the fact that out of those hits, only a small proportion actually displays in your log files as a hit to facebook.com or youtube.com. I logged into my Facebook account and at the same time ran a Firefox add-on called Firebug. Firebug allows you to see all the resources on a specific page and how long they took to download. As you can see below only one of the hits are attributed to facebook.com. The rest are from fbcdn.net. The same test on a YouTube page also yielded one out of 29 hits attributed to youtube.com. The rest was from ytimg.com, doubleclick.net, googlesyndication.com and gstatic.com.

Click to Enlarge

Using Aliases to merge your hits

fbcdn.net and ytimg.com are Facebook’s and YouTube’s CDNs (Content Delivery Network). Basically a network of servers hosting images and resources, spread out across the globe, to enable local delivery and thus improve performance. Unfortunately it doesn’t improve the accuracy of your reports, but there’s an easy way to fix this using Vantage. Simply merge the hits from the main domain (facebook.com, youtube.com, or any other site) and their CDNs by creating an Alias in Vantage’s Summaries screen.

To do this simply:

• Click on the ‘Summaries” tab in Vantage
• Click ‘Site Domain’ to get a listing of sites
• Ctrl + click the sites you want to add to the Alias (in this case facebook.com and fbcdn.net), right click and click ‘Add to Alias”
  
  

  Click to Enlarge

   

• Pick the Alias called ‘Web sites’ and name it something intuitive…why not facebook.com

   

• Now enable the ‘Web site’ Alias on the left hand side

 

• You’re good to go, from now on all your Facebook traffic will be reported on more accurately by merging actual facebook.com hit with hits on Facebook’s CDN (fbcdn.net).

If you want to find out more about using Aliases you can read about it in our User Guide (starting at page 46).

Microsoft TMG Log showing Bytes Sent consistently larger than Bytes Received

This issue has been confirmed by the Microsoft Forefront TMG team, and unfortunately there is no ETA for a fix.

We obviously don’t want our reports showing incorrect usage figures, so we’ve fixed our TMG loader so that it imports the ‘bytesrecvd’ field into the Bytes Sent aggregate, and the ‘bytessent’ field into the Byte Received aggregate.

But what if Microsoft release a fix? What we’ve done is implemented a loader property to allow you to turn off this behavior. This will allow you to import your old logs with the fields reversed, and your new logs with the fields the right way around.

To access the loader property:

• On the import wizard, select the Microsoft FTMG format and click the Properties button on the toolbar
• Select Microsoft FTMG from the drop down list
• Notice the option to ‘Reverse Bytes Sent and Received to compensate for bug in TMG’s logging’. Leave this checked until Microsoft issue a fix.

Microsoft Forefront TMG Loader Option to Reverse Bytes Sent and Received

This fix is available in Vantage build 2.2.0.48 (and above) which has been released as an auto update. So simply select Tools | Check for updates to ensure you have this fix.

So let’s use WebSpy Vantage to drill into that Anonymous user and find out what is going on.

One way to do this is to run an Ad-hoc analysis on the Summaries screen and drilldown into the Anonymous user to view all the information about that user. However, TMG and ISA tend to log a lot of information that may not be relevant to this particular investigation, so I’ve created some report templates (one for ISA and one for TMG) and a set of Aliases that pull out some relevant information.

Download our Anonymous Traffic Investigation Report

If you’re running WebSpy Vantage download the Anonymous Traffic Report Templates & Aliases

Then open the .Templates file on the Reports tab, and the .Aliases file on the Aliases tab. Once you have both files opened, go to the Reports tab and click either the ‘Anonymous Traffic Investigation (ISA)’ or the ‘Anonymous Traffic Investigation (TMG)’ report. Then click the ‘Generate report’ link and run the report template on your ISA or TMG storage.

The report gives you the ability to drill into the Allowed, Denied and Failed traffic to see a list of the unauthenticated IPs, Sites, Rules responsible for blocking or allowing the traffic, unauthenticated Applications and Result Codes.

Main causes of anonymous traffic

What you will probably find is that most of the Anonymous traffic is being denied by your TMG or ISA firewall. When a client first requests a web page, the proxy will challenge the client for authentication. These events are often logged with the result code 12209 meaning ‘authorization is required to fulfill the request’. These requests are therefore denied by the proxy until the client’s credentials are authenticated.

Have a look at the amount of traffic being denied and then checkout the Result Codes associated with the denied traffic. Chances are you’ll see ‘proxy authentication required’ appear predominantly.

If you also look at the Applications section you may also find that Windows Updates are sailing through your TMG or ISA firewall unauthenticated.

Filter out unauthenticated traffic from Reports

The most logical next step is to filter out the information you do not want in your reports. You’ll probably still want to include Windows Update traffic in your reports, but you’re probably not so interested in the ‘proxy authentication required’ information. So let’s filter that out.

To do this:

1. Go to the Reports tab and select the report you want to filter (such as your Organization report)
2. Click ‘Edit Template’, then click ‘Template Properties’.
3. In the filter section at the bottom of the dialog, click Add | Field value filter.
4. Select the ‘Result Code’ summary and select the Status Code Names (ISA-FTMG) alias.
5. On the toolbar, search for Authorization, and check the following two items:
   • The server requires authorization to fulfill the request. Access to the Web Proxy filter is denied.
   • The server requires authorization to fulfill the request. Access to the Web server is denied. Contact the server administrator.
6. Ensure the ‘Exclude’ radio button is selected and click OK.

If you decide that you don’t care about seeing ANY unauthenticated traffic in your reports, you can always simply filter out the Anonymous user from your reports.

To do this:

1. Go to the Reports tab and select the report you want to filter (such as your Organization report)
2. Click ‘Edit Template’, then click ‘Template Properties’.
3. In the filter section at the bottom of the dialog, click Add | Field value filter.
4. Select the ‘Username’ summary.
5. On the toolbar, click Add and type ‘anonymous’. Click OK.
6. Ensure the Exclude radio button is selected and click OK.

Hopefully this article improves your understanding of the ‘anonymous’ user, and gives you some actions to take for your specific reporting situation.

If you have any questions, please leave a comment below.

• How to import your log files and explore the information recorded by IronPort using the Summaries screen
• How to open the customized IronPort Report Templates and Aliases
• How to generate reports
• How to import your organizational structure and report on departments
• How to setup the Web Module and publish reports

PART 1: Importing log files & exploring your IronPort summaries

Once you have exported your IronPort access logs (see http://www.webspy.com.au/vendors/ironport/howto.aspx#ftp), this video takes you through importing your logs into WebSpy Vantage and analyzing data on the Summaries screen.

PART 2: Opening the customized IronPort Templates & Aliases, and running reports

This video takes you through opening the IronPort-specific report templates and aliases and generating a report that provides an overview of your organization’s Internet usage.

PART 3: Importing your Organization structure & generating department reports

This video shows you how to import your organizational structure into WebSpy Vantage from a directory server (such as Active Directory) using LDAP, and then generating a report that contains information on your newly imported departments.

PART 4: Using the Web Module.

This video takes you through configuring and using the WebSpy Vantage Web Module. Specifically, it takes you through the following tasks:

• Configuring the Web Module for Windows Authentication
• Adding a Web Module to Vantage
• Publishing reports to the Web Module
• Adding permissions for a user
• Synchronizing the Web Module
• Using the Dynamic Reports tab

PART 5: A quick word about tasks & conclusion

This video summarizes the actions taken in the previous four videos and also briefly discusses how to automate the reporting processing using scheduled tasks.

I hope this helps! Let me know if you have any questions by leaving a comment below.

“I would like to know how much of my bandwidth is being eaten by each protocol. I will then use this information to determine if circuit may need to be increased due to increased traffic”.

This customer was collecting syslog messages from a Cisco Firewall, then using WebSpy Vantage to generate reports. In theory, this sounds like a fair plan. Unfortunately, the Cisco Firewall logs many different types of messages. Some to do with denied packets, some to do with authentication, some for vpn and so on. The information contained within each message changes. Some events include the size information that is required for any type of bandwidth assessment and some don’t. Correlating the required events to get any sort of accurate ‘bandwidth’ representation is a bit of a nightmare.

Fortunately, there’s a simpler method. If you search the Cisco website or the Internet for bandwidth utilization reporting, you’ll no doubt be pointed in the direction of NetFlow.

NetFlow is a network protocol developed by Cisco Systems to run on Cisco IOS-enabled equipment for collecting IP traffic information [Source Wikipedia http://en.wikipedia.org/wiki/Netflow]

There are a couple of commands to enter on your router to turn NetFlow on, and then you just need a NetFlow collector to receive the Netflow information and generate reports.

Fortunately WebSpy has developed a little tool called FlowMonitor that collects the Netflow information and writes a log file that can then be imported into WebSpy Vantage and reported on.

The FlowMonitor Management Console

Once your FlowMonitor logs are imported into WebSpy Vantage, you can run the default FlowMonitor report to see the size of traffic flowing between IP addresses, subnets, router interfaces or protocols. Alternatively you can create your own custom reports to see exactly what you want to see.

NetFlow doesn’t record usernames or URLs so it’s not great for reporting on the web sites your users are visiting, but it is great for network administration and trouble shooting. Identify chatty IP addresses, protocols that are chewing too much bandwidth, the times throughout the day when incoming or outgoing links become heavily utilized and so on.

For information on how to configure your router and deploy FlowMonitor, see the FlowMonitor Installation and User Guide. You can also download a free trial here.

FlowMonitor is a handy little tool. Ask your friendly WebSpy account manager about it today!

Vantage uses multiple threads to perform certain tasks. As general rule, the more threads being used simultaneously, the higher the CPU utilization. There are a few situations where Vantage uses multiple threads simultaneously:

CPU usage when importing log files

When importing more than one log file, each log will be imported with a separate thread. As CPU usage increases when more threads are used, importing a single log file won’t push your CPU, but importing a folder full of logs will.

CPU usage when generating reports

CPU performance can also be affected by the structure the report you are running. Report templates have what we call ‘Nodes’ in them. You can go into a report template, right-click | add node. Think of each node as an SQL query. When generating a report, each node gets processed in a separate thread, and if the nodes are ‘at the same level’ they get processed at the same time.  Here’s a screenshot showing what I mean by nodes at the same level.

A report template with two 'levels' of nodes

The three ‘red’ nodes will be processed at the same time, and then the three ‘green’ nodes will be processed at the same time. The green nodes won’t be processed until the red nodes have been processed. The more nodes being processed at the same time increases the number of simultaneous threads and the amount of CPU being used.

CPU usage when filtering reports

CPU usage is also affected by the number of records being processed from your storage. If you are running a report on your entire storage with no filters, then Vantage will be pushing all records in your storage through the reporting engine. If you run the same report but with a filter for a specific user, then Vantage will seek through the records in the storage until it finds a record for that user, then push that record through the reporting engine. This results in a ‘trickle’ of records being pushed through the reporting engine so it doesn’t get a chance to really push your CPUs.

A filter that excludes a lot of information that exists in your storage is the most common reason for low CPU utilization while running a report.

In Short

The number of logs, report template structure and filters can all have an effect on the way Vantage utilizes your CPUs.

We also have some exciting ideas on our roadmap to ensure Vantage utilizes as many CPUs as you can throw at it. Until then, I hope the above information helps you understand when and why your CPU usage will and won’t be pushed.

This is a great update for Vantage Ultimate users as we’ve introduced a new feature/tab into the Web Module called ‘Dynamic Reports’.

If you’re publishing the same report to the Web Module each day, you can use the Dynamic Reports tab to select a date range and a department (or whatever organizational groups you have defined) and the Web Module will collate all the daily reports that match that filter into one report. This allows you to report on entire week, month or year by simply ‘reporting on reports’, rather than reporting months of raw storage data.

Here’s the full list of changes since the last auto update (2.2.0.32 on the 14th April 2010).

Application Changes

• Added Dynamic Reports feature to the Web Module.
• Rewrote the Web Module transfer protocol. New protocol adds version checking, connection checking, and integrity checking for high latency environments.
• Purge data from storage task no longer prevents importing new hits when all data is removed from an input within a storage.
• IPv6 addresses now show IPv4-mapped addresses as plain IPv4 addresses in summaries.
• IPv6 and IPv4 addresses are now freely interchangable in filter expressions.
• Fixed IPv6 drilldowns on the Summaries screen
• SQL inputs can now be resumed from the previous position. Previously any input that was partially imported would be skipped when importing new hits.
• Template-based analysis has been fixed, no longer results in blank/non-existent analysis.
• Added new string manipulation functions to expression language; Contains, StartsWith, EndsWith, IndexOf.

Loader Changes

• Astaro: Now checks that the ID field is present in a line before attempting to read it.
• Barracuda Web Filter: Added this format to replace Spy Filter.
• BlueCoat Proxy SG W3C: Added support for gmttime, timestamp, x-bluecoat-surfcontrol-is-denied and x-bluecoat-transaction-id.
• ClearSwift: Added a new loader group for ClearSwift that includes the MimeSweeper loaders
• ClearSwift SECURE Web Gatway: Now supported with the Web Appliance loader
• Clearswift Web Appliance: User summary displays Source IP if Username is blank.
• IronPort WSA: Fixed memory usage issues.
• Microsoft FTMG: Added category name lookup to SQL loader.
• Microsoft FTMG: No longer fails to import lines where the rule field contains square brackets.
• Microsoft FTMG: URL Category field is now a string instead of an integer. Added URL Categorization Reason field.
• Microsoft FTMG: Fixed memory usage issues.
• Microsoft IIS W3C: No longer hangs or crashes when loading a file that isn’t IIS W3C.
• NetAsq: Added support for srcname field. The Username summary is populated with user first, and then srcname if user is blank. The User summary is also now populated with Source IPs if the Username summary is blank.

To update WebSpy Vantage, simple select Tools | Check for updates.

To update the Web Module, login to the Web Module server, right-click the WebSpy system tray icon, and select Check for updates.

As always, please contact us if you have any issues or questions.

One of Forefront TMG’s major strengths is obviously its URL categorization and filtering abilities. Since TMG now takes care of the threat management aspects, clients converting from other solutions, such as ISA Server, no longer need a third party filtering solution and will most likely save a considerable amount of money.

However, the reporting functionality included in Forefront TMG are not much different from ISA Server 2006, i.e. very little flexibility or customization for those with reporting requirements beyond general overviews cluttered with irrelevant information.

We’ve blogged a lot about TMG reporting in the past and have now uploaded new and dedicated WebSpy Vantage and Microsoft Forefront TMG pages outlining:

• 10 Reasons to Use WebSpy Vantage to Report on Forefront TMG
• How to:
• Set-up TMG Logging for WebSpy
• Import TMG Logs into WebSpy Vantage
• Forefront TMG Report Templates and Aliases (created to make your life a lot easier)
• Run Reports
• Analyze and Drilldown into Data

Have a look at WebSpy Vantage and Microsoft Forefront TMG.

Hopefully it can assist you in your quest for sophisticated Forefront TMG reporting.

A large chunk of our clients use WebSpy Vantage to report on a wide spectrum of devices (web proxy server, email server, firewalls, routers, switches etc) from more than 150 different vendors and not solely as a web analytics tool. Saying that, many WebSpy Vantage users do take advantage of its web server reporting abilities and have experienced first hand the benefits of log file analysis over GA’s JavaScript tagging. Clients generally like the way they can work with the data in GA, but not completely happy with the data GA provides and therefore use WebSpy Vantage to complement and provide detailed drilldowns into certain areas.

Log File Analysis vs. JavaScript tagging

In order to enable Google Analytics to report on incoming traffic you need to copy and paste a custom JavaScript to every page within your site or site template. Since Google Analytics is relying on JavaScript the following issues are very likely to affect the accuracy of your data:

• Not reporting on visitors who disable JavaScripts in their browsers
• Overestimating visitors who regularly clear their cookies
• Limited or no reporting for non-standard page extensions

Limited or No Reporting for Non-Standard Page Extensions

This is the area I want to expand on today. GA won’t give you any information on resources accessed within the site that is not an actual page containing the custom JavaScript. You won’t be able to see how many times visitors downloaded your documents, such as white paper, product catalog or PowerPoint presentation, in file types such as .doc, .pdf, .txt, .pps, .zip, .xls, etc.

Alright, I’m telling lies, there are roundabout ways of tracking these downloads but it includes tagging all document links with a _trackPageview() JavaScript. This piece of JavaScript assigns a pageview to any click on a tagged link. Not only does this involve a pretty cumbersome process of adding scripts to all your website document links, you also need to take into account the error in reported document downloads that will occur since you are not always in control, and can tag, all instances of your document links appearing on the inter-web. For example, if other websites are linking straight to your documents, or if people are arriving at those resources from other external sources (e.g., an email containing an url to the pdf).

Reporting on your web server log files is a much more reliable way of getting the accurate information you need. It’s pretty darn easy as well.

Using Vantage to Analyze Non-Standard Page Extensions

Below is just a brief example of how easy it is to use WebSpy Vantage to get the reliable information on different file type downloads.

After importing my web server logs, for a randomly selected period in April, into Vantage I can get an overview of all the different file types accessed by simply clicking ‘Site Extension’.

Let’s say I am curious to find out more about the .pdf documents accessed by visitors during this period. I click on the .pdf site extension and am immediately presented with a variety of options to further investigate .pdf downloads.

For example, I can drilldown into .pdf ‘Site Resource’ to get a list of all the pdf documents accessed. To locate the most popular pdf I simply sort the list by number of hits.

It seems like the most popular .pdf document on this particular day one was our ‘5 Reasons to recommend WebSpy Reporting with 44 hits. Again, I can now drilldown further into any site extension to find out relevant information on referring sites, search engines used, the keywords used search engines etc.

Referrer Keywords

For those that don’t know what Soho is all about, check out this video:

Soho is a dashboard application that displays download and upload traffic statistics for each computer in your network. If you haven’t yet tried Soho, please give it a go and download it here.

It is our intention to keep our Alpha testers up to date with our ongoing development. Right now, I’d like to inform you about some issues experienced by a handful of testers and how to go about resolving them.

Learn to restart the Soho Agent

First of all, one of the handiest things we can tell you right now is how to restart the Soho Agent. This single step is resolves 99% of all Soho issues, at least temporarily. If these steps seem too complicated, rebooting your PC also has the same effect.

To restart the Soho Agent on Windows:

1. Launch the Services Console by going to Control Panel | Administrative Tools | Services. Or if you like handy short cuts, try Start | Search (or Run), Type ‘services.msc’ (without the quotes) and press enter.
2. Right-click the “WebSpy Soho Agent” service and select Restart. If you get a ‘time out’ error message or warning, ignore it and right-click the service again and select Start.

To restart the Soho Agent on Mac OS:

1. Open the terminal from /Applications/Utilities/Terminal
2. Type sudo launchctl stop “WebSpy Soho Agent”
3. Enter your user password if requested.
4. Wait about 5 seconds.
5. Type sudo launchctl start “WebSpy Soho Agent”
6. Again, enter your user password if requested

OK, now you have the skills, here are the issues and work-arounds:

100% CPU usage after sleep or hibernate

Some users reported Soho’s impressive ability to consume 100% of their CPU when their computer wakes from sleep or hibernation. A few users experienced a slow and sluggish PC, and uninstalled Soho immediately.

If you’re looking for the Soho process in Windows Task Manager you will not see it until you click the Show processes from all users button (or checkbox in XP). This is because the Soho Agent runs under the System user account in order for it to run, no matter who is logged onto the PC.

From here you can end the WebSpy.Soho.Agent.exe process and everything should return to normal. You can then restart the “WebSpy Soho Agent” to get Soho working again (see steps above).

We believe we have fixed this and are in the middle of some final testing. All going well, we should have a new build ready for you very soon. In the mean time, you may like to disable sleep and hibernation in your PC’s power options.

Soho doesn’t install on Mac OS 10.5??

Our first Alpha release did not install on Mac OS 10.5 (Leopard). This was due to a silly checkbox in our packaging system not being checked. We’ve checked the checkbox and uploaded a new build to our web site. You can download it from here:
http://www.webspy.com.au/products/soho/download.aspx

Note: Soho will only install on Mac OS 10.5 (Leopard) and 10.6 (Snow Leopard). Versions 10.4 (Tiger) and below are not supported.

All computers disappear from the Current Activity chart except the local computer

Sometimes all computers will disappear from the Current Activity chart leaving your local computer all by itself. This happens when the communication between Soho Agents becomes jammed. You can usually resolve the issue by restarting the Soho Agent on your local computer (see steps below). If this doesn’t work, restart the Agents on all other computers running Soho. We are currently working on a fix for this issue.

The Soho User Interface is completely blank

If there is no information in the Total, Current Activity or History chart, this is because the Soho Agent has stopped running. Reasons for this may vary, so please let us know if this is regularly occurring. Restarting your agent usually resolves the issue (see steps above).

Feedback

Thank you to everyone that has submitted feedback so far.

Just a reminder to please let us know if your network card works or doesn’t work with Soho on this page:
http://www.webspy.com.au/products/soho/supportednics.aspx

You may also like to review the current list of features and bugs and vote them up or down at:
http://webspysoho.uservoice.com/

There is also a dedicated Soho Alpha Feedback thread in our forums at:
http://www.webspy.com.au/forums/viewtopic.php?f=8&t=12

We will let you know when fixes are available to the above issues.

Here’s a quick video outlining some of the differences between TMGs Reporting, and what can be achieved using WebSpy Vantage. The video does not illustrate all the limitations outlined below, so please read on.

Whats is in the Forefront TMG report?

The default TMG report contains the following sections

• Summary
• Web Usage
• Application Usage
• Traffic and Utilization
• Security
• Malware Protection
• URL Filtering
• Network Inspection System

Each section contains overviews such as ‘Top users’ and ‘Top Sites’.

If your reporting requirements can be satisfied with these overviews – that’s great! Unfortunately, when you start thinking about what system administrators and other people in your organization actually need to make informed decisions, this report is quite limiting.

The 8 Limitations of Microsoft Forefront TMG’s Reporting

Here is what I consider to be the 8 main limitations of Microsoft Forefront TMG’s reporting functionality.

1. No Drilldowns

Want to see the sites that the top 5 users accessed? Want to see the users that downloaded the most traffic from youtube? These are fairly standard reporting requirements that simply cannot be achieved using the inbuilt TMG reporting.

WebSpy Vantage lets you either interactively drilldown into a user or site, or produce a regular report that includes further details about what your top users have actually been up to.

2. No Filtering

When you generate a report in TMG, you can only filter the report by a date range. There is no way to filter out anonymous (unauthenticated) traffic or exclude traffic coming from advertising servers (such as doubleclick and 2mdn.net) that tend to dominate most of the top 10 sites.

This can easily be achieved using WebSpy’s software. Check out my video on how to remove clutter from your web reports.

3. No Customization

Customization of each overview in the TMG report is limited to the number of items to show (e.g. top 5 or top 50 users), and the sort order (Incoming Bytes, Outgoing Bytes, Requests and Total Bytes).

What about the time a user spent browsing the web, or the number of users that visited a specific site? There is no way to add custom columns such as total browsing time, average session time, or number of users/sites/IPs to the report tables.

Or say you simply want to change your top users chart from a bar to pie to easily see the percentage used. Nope sorry!

If you do make one of the two available customizations in a TMG report, you then get the annoying Apply / Discard message to save changes to the configuration database.

All of these customizations can be achieved using WebSpy Vantage, and it doesn’t touch your TMG server to apply a change to a report.

4. Limited Report Distribution

When you generate a report, you get the option to email it to a specific email address. What if you would like to create a report for every department, and then email it to the managers of each department? Or better yet, host the report on a secure web server where department managers can log in and view their reports?

WebSpy Vantage Ultimate comes with a secure ‘Web Module’ specifically for this purpose and managers still receive a link to the report via email.

5. Cluttered ‘Top Sites’ List

The ‘Top sites’ list can become particularly cluttered due to the inclusion of sub-domains. I don’t want to mentally add up the size values from farm1.static.flickr.com, farm2.static.flickr.com, and farm3.static.flicr.com – I just want to know how much was downloaded from flickr.com.

This is compounded by the inability to exclude sites that are merely placing advertising banners on the actual sites users are visiting (as mentioned in the ‘No Filtering’ limitation above).

WebSpy Vantage breaks URLs down into separate components and lets you analyze each part separately. Look at the Site Domains summary to remove sub-domains and see only flickr.com. Or perhaps you want to see the keywords a user entered into search engines like Google? Or perhaps the top pages accessed within a website? No problem. Just include the Site Keywords or Site Resource summaries in your Vantage reports.

6. No Grouping or Aliasing

There is no way to group users into departments or locations, or IP addresses into subnets, or extensions such as .html, .pdf or .exe into file types. The ability to group and represent raw log data in more meaningful ways, as offered by WebSpy Vantage, can increase the value of a report tremendously.

7. No Productivity Assessment

One of the major features introduced in TMG since ISA Server 2006 is the included URL categorization technology.

Although the TMG report gives you an overview of the categories that have been visited, the report does not use this information to display a productivity assessment for your users.

WebSpy Vantage not only provides this assessment, but also the ability to customize the categories that are deemed productive as this can vary wildly depending on the industry and organization.

8. Not browser independent

This is a minor limitation that can be a major annoyance. The report that TMG produces is a HTML report that only displays correctly in Internet Explorer. As Forefront TMG is a Microsoft product, this is not exactly surprising, but still very annoying if IE is not your default browser.

How to get awesome reports from Forefront TMG

If you have had personal experience with any of the above limitations, you’ve probably been hunting for an alternative solution. I strongly recommend checking out the WebSpy Vantage range of products, and if you would like secure report distribution via the ‘Web Module’, Vantage Ultimate is what you are after.

If you agree or disagree with anything in this article, I encourage you to leave your thoughts in the comments.

Cheers!

Scott

But seriously…

If you always treat corporate communication as though the evil boss was listening in on the other end, at all times, you cannot go wrong. Personal correspondence with loved ones should be kept to decent content; employees have been fired, and reputations have been destroyed, over ignoring Acceptable Usage Guidelines. It is also doubtful whether the tone of the conversation would be so explicit if conducted, for instance, over the phone, where the entire open-plan office can eavesdrop, so bear this in mind when sending personal emails from work.

Spending time browsing for porn at work is also a bad idea, as not only does it put you at risk (sexual harassment suits have been started over nothing less), but you are also exposing your employer to further liabilities. In addition, not all pornographic web sites are “healthy”, and will often attempt to upload malicious software to your machine. Think of it as a new, 21st Century vector for the sexually transmitted disease…

The same is also true for most software and music download sites. More often than not, these are simply hotbeds of malware that will be disastrous for you, should it end up on your local machine. Having to petition your sysadmin to perform a complete reinstallation of your desktop because you were browsing unproductive sites at work is one of the fastest ways of irritating someone you should try to keep happy!

Gambling and games sites also occupy more time than we like to think – try not to let yourself get sucked into them and set your alarm if you know you are one of those who gets easily distracted.

Internet connectivity need not be a tool for wasting time

If you are responsible with your browsing habits, you should have no reason to ever be fired for them. Employers who wish to avoid the confrontations that are the end result of such reckless browsing habits should concentrate on training, motivating and retraining employees; keeping staff turnover low has the knock-on effect of creating a stable workplace environment where employees feel valued and want to stay on.

Why lose your most experienced and best workers simply because they do not understand the nature of consequence? Showing them their browsing habits encourages them to see their usage for themselves and its effects on the workplace, without the need for blocking, which can also be incredibly depressive for morale.

Internet connectivity need not be a tool for wasting time. Maintain an Acceptable Usage Internet Policy and monitor your usage wisely.

This concern was also raised in our recent “Notes on E-Security Development” blog.

Majority of schools and educational institutions in developed countries are investing in sophisticated security solutions to help protect their internet resources. However, by using public proxy sites students are able to bypass security solutions and disguise their inappropriate activity from being detected.

When a proxy server is used a student will appear to be visiting only one site, the proxy itself, and not the blocked or banned target site. Any internet surfing they do after that is effectively invisible.

IBT also states that the number of public proxy sites has increased dramatically over the past few years. In 2006 M86 Security estimated the number of proxy sites to be 7,111. By 2009 the new estimate had risen to an amazing 91,490.

Using proxy sites to access blocked sites puts both the schools and the students at risk. The schools are at risk because the virus ridden proxy sites can contaminate their entire network and enables students to access the blocked sites already deemed as high risk. Students (and teachers for that matter) can personally suffer if a proxy site hosts malware, such as a trojan. Once a trojan has spread to computers, hackers can access them remotely and steal data, log keystrokes, and thus easily grab personal passwords and credit card numbers.

So what to do then? A spokesman for JANET, which carries data traffic between many local school networks in the United Kingdom, said: “I would agree that using proxy servers to get around security systems is indeed a problem. Technical solutions need to be used as one aspect of a wider approach to protecting users, including educating children, teachers, and parents in how to use the web safely.”

Education is certainly the key but it is also important to make the most out of security and monitoring solutions. Yes, it is impossible to effectively identify and manage all the 91,490 proxy sites out there. However, you might find that out of those 91,490 around 20-40 are commonly used and shared among the students at a specific school. Many public proxies use IP addresses (as opposed to site names) to avoid easy detection, so a spike in a certain IP address could be an indication that it is a popular proxy site.

Read our “How to Improve Public Proxy Management and Control” blog for tips on detecting public proxy usage.

These courageous words were uttered by Richard Schaeffer, information assurance director at the NSA. He also added that simply focusing on adhering to common best practices would substantially raise the bar.

Wired reports that the Senate Judiciary Subcommittee on Terrorism, Technology and Homeland Security recently heard from a number of experts offering commentary on how the government should best tackle securing government and private-sector critical infrastructure networks.

Larry Clinton, president of the Internet Security Alliance, declared that public indifference and unawareness played as much a role in the current state of cyber security as the unwillingness of corporate entities to take responsibility for securing the public’s data.

Clinton, whose group represent banks, telecoms, defense, technology companies and other industries that rely on the internet, said that corporate and government entities that collect and store the public data “do not understand themselves to be responsible for the defense of the data.“ He added that, “The marketing department has data, the finance department has data, etc, but they think the security of the data is the responsibility of the IT guys at the end of the hall.”

Clinton does not believe federally mandated cyber security standards are the answer as they can be seriously counterproductive to national economic and security interests. To improve cyber security, the public sector would have to institute sufficient market incentives to motivate companies to protect the public’s interests. His group plans to release a proposal next month laying out some recommendations.

What do we think?

Although Schaeffer and Clinton are discussing cyber attacks and security on a national level they make it painstakingly obvious that solving the problem requires joint efforts from the government, the public, network administrators and ALL OTHER members of an organization.

At WebSpy we continuously preach that Internet and Network monitoring should not just be the network administrators’ responsibility. In fact, Vantage Ultimate was developed exclusively to increase the effectiveness of IT policy adherence while taking the pressure off the IT department. Vantage Ultimate enables secure distribution of organizational Internet and network reports across an entire organization, whilst protecting employee privacy. Why does this matter? Because distributing the responsibility for IT security starts by efficiently distributing IT security information.

Find out more about Vantage Ultimate

Related Links:
Wired Article
NSA
Internet Security Alliance

In the mean time, I thought I’d share a way to re-brand the main elements in the Web Module by editing a few files and replacing a few images.

The only issue with this technique is that any future auto-updates for the Web Module will overwrite your edited files, so you just need to keep a copy of your customized files, so that you can restore them again after the auto-update.

Before you begin

In order to edit anything, you first need to know where your Web Module is located on your web server’s hard drive. This can be found by opening IIS Manager (Start | Control Panel | Administrative Tools | Internet Information Services (IIS) Manager) expanding the left hand server/site tree to find your Web Module.

• In IIS7, select the Web Module and click Basic Settings… in the right hand ‘Actions’ panel. The location is specified in ‘Physical Path’.
  

  Finding the Web Module's physical path in IIS7

• In IIS6, right-click the Web Module and select Properties… then go to the Home Directory tab. The location is specified in ‘Local Path’.

Windows may also prevent you from editing these files directly due to permissions issues. I’ve found a good technique is to copy the files you want to edit to your desktop, edit them, and then copy them back into the Web Module’s physical path. Windows will then prompt you to elevate to administrator and the copy/replace will succeed.

Ready To Go…

There are a few places where the WebSpy logo and WebSpy Text is presented.

• The login page
• The header bar
• The welcome Page
• Report cover pages

The Login page

The logo displayed on the login page can be found at /images/logo.png. Replace this image with your own logo. Then open Default.aspx in the Web Module’s root folder in a text editor such as notepad, and replace the following line

<img runat=”server” alt=”WebSpy” src=”~/Images/Get.ashx?image=Logo” />

with

<img runat=”server” alt=”WebSpy” src=”~/Images/logo.png” />

Before

Web Module's Login Page Before logo.png Change

After

Web Module Login Page After logo.png change

The header bar

The header bar utilizes the image located a /Images/bauble.png. Replace this image with your own custom image.

Then open Navigation.Master in the Web Module’s root folder in a text editor such as notepad, and replace the following line

Also look for the text:

Before

Web Module's Header Bar Before Bauble.png and Text Changes

After

Web Module's Header Bar After Bauble.png and Text Change

The Welcome Page

When you first login to the Web Module, you are presented with a Welcome Page. The first line on this page reads “Welcome to the WebSpy Vantage Web Module. You can change this by editing the first line in the Welcome.aspx file located in the Web Module’s root folder. Edit the section in bold below:
<%@ Page Language=”C#” MasterPageFile=”~/Navigation.Master” AutoEventWireup=”true” CodeBehind=”Welcome.aspx.cs” Inherits=”WebSpy.Vantage.WebModule.Welcome” Title=”Insert Custom Text Here” %>

Before

Web Module's Welcome Page Before Text Change

After

Web Module's Welcome Page After Text Change

The Report Cover Pages

The Image used on the cover page of reports is much easier to change.

1. Login to the Web Module as Administrator
2. Go to the Options Tab
3. Click ‘Report Logo’ under Web Module Options
4. Click Choose File, and select the image or logo you would like displayed on your report cover page
5. Click Upload

Before

Web Module's Report Cover Page Before Report Logo Change

After

Web Module's Report Cover Page After Report Logo Change

Summary

The changes above cover a majority of the areas your users will come into contact with in the Web Module. There may be a few more instances of the word “WebSpy” but for the most part, it should just be a matter of opening the relevant .aspx file and editing the html.

As I mentioned, if you auto-update the Web Module (via the system tray icon on the Web Module server), your edited files will be overwritten. I recommend keeping a copy of your edited files in a safe place outside the Web Module’s physical folder, so that you can copy them back in after the update. If the only changes you make are the ones above, then you’ll need to keep a copy of:

• /Navigation.Master
• /Default.aspx
• /Welcome.aspx
• /Images/logo.png
• /Images/bauble.png

We will also be adding the functionality to make these changes ‘properly’ in a future build, so follow us on Twitter, or subscribe to our RSS feed for updates!

Cheers!

Most frequently suggested bypass methods includes the use of public proxies, circumventors and http tunneling. I don’t wish to go into details on any of these methods as their use is NOT recommended. However, it does prove a point: The main reasons organizations block certain websites is to prevent security risks and unproductive internet usage. Although, it is an indisputable fact that employees’ use of virus ridden public proxies, and other elaborate methods, to overcome blocking efforts can in fact increase security risks and unproductive behavior – making matters even worse.

Obviously all employees do not take these measures, but isn’t it enough that some do? Yes, the same high risk and time consuming bypassing “techniques” could be used when trying to stay anonymous from internet monitoring software. However, there are two main differences:

1. Using internet monitoring software reduces the need to block. Employees will be able to access the legitimate sites that often end up blocked thanks to a “block worthy” word in a corporate blog, or something of similar virtuousness. Not blocking means less time and effort spent trying to bypass blocking solution. After all, my mailbox is not full of alerts on how to bypass internet monitoring software.
2. Using internet monitoring software will allow employers to detect who is up to no good trying to bypass blocking rules or browse anonymously. For example, if an employee continuously use public proxies or tunneling, an internet monitoring solution (or at least a good internet monitoring solution) can assist the employer in tracking down the offender. (Please have a look at “How to Improve Public Proxy Management” blog for more info.)

This blog simply adds to the convincing case against organizations’ excessive use of blocking and filtering solutions. Porn sites, known malicious virus and phishing sites – by all means, block the living daylight out of them. But as for the rest, as for news site, online shopping sites, social networking and general interest sites – Don’t block, monitor.

I want to avoid repeating myself so please have a look at previous blog for the full story on “The Cost of Blocking Employee Internet Usage”

Gallup has reported that Christmas spending intentions are currently down, compared to last year. Americans are planning to spend an average of $740 on gifts this year; somewhat less than the $801 recorded the same time last year.

Despite this monetary shopping decrease there are no sign that there will be a decrease in the amount of time spent shopping online. According to ISACA, a non-profit association of 86,000 IT professionals, employees plan to spend nearly two full working days (14.4 hours) on average shopping online from a work computer this holiday season.

Ho, Ho, How will this affect your workplace?

ISACA reports, “The potential danger of shopping online is that it can open the door to viruses, spam and phishing attacks that invade the workplace and cost enterprises thousands per employee in lost productivity and potentially millions in destruction or compromise of corporate data.”

Lost Productivity

This almost goes without saying – 14.4 hours spend online shopping means 14.4 hours being unproductive. The survey estimates that employers lost $580 million in lost productivity on Cyber Monday (Nov. 30) in 2008. This year, Cyber Monday is anticipated to be the number one online shopping day. The busiest hours are from 2pm to 4pm, while people are supposed to be working.

Increased Vulnerability

ISACA reports that “employees who shop online using a work computer are also likely to engage in other high-risk behaviors. Survey participants also bank online (51%), click on e-mail links redirecting them to shopping sites (40%) and click on links from social network sites (15%). Yet nearly one in five says they are not concerned that their online shopping habits may affect the safety of their organization’s IT infrastructure. “

Recommendations:

• Monitor – Don’t Block

Don’t start blocking online shopping sites in an attempt to circumvent the problem. Depriving the internet needs of a professional workforce can cause resentment and increase costly turnovers. Blocking Internet access also has potential to reduce productivity by complicating or delaying accomplishment of tasks. In addition, blocking sites may also encourage employees to seek out less secure ways to access block sites.

• Monitor Employee Internet Usage Openly and Respectfully

It is not 1984, overly intrusive practices can easily create the negative perception that Big Brother is watching and make employees feel frustrated and uncomfortable. The effectiveness of Internet monitoring directly relates to employees’ awareness of the content of the policy, their perception of its fairness, and corresponding breach consequences. Please have a look at previous blog “Internet Monitoring Best Practices – 10 Valuable Tips” for a complete list of best practices.

• Educate, Update, Iterate

Educate employees about the risks of online shopping in relation to viruses, malware, spam and phishing attacks. The online threat is constantly changing so ensure your online safety training and education is up to date. Iterate to ensure employees always have internet security at the top of their minds.

If you are using WebSpy’s software you can easily see who may be in need of further online safety training by viewing who is clicking on phishing links and who is trying to access unsafe websites blocked by your organization. Read recent user case: “How to Educate your Workforce and Strengthen Security with Internet Monitoring“.

• Join the Spirit of Christmas

It is Christmas after all. Being more lenient with your Acceptable Internet Usage Policies during the holiday season, as long as work related expectations are met, can significantly assist your employees during this busy time of year. This day and age many employees are working longer hours and have less time for personal tasks, such as holiday shopping. Increasing their online shopping allowance can save employees’ time and let them achieve a better work-life balance.

Related Links:

ISACA
Gallup

Unfortunately, the WebSpy Twitter account fell victim to a phishing scam, and as a result sent phishing spam to all our Twitter followers. We are embarrassed by the incident and we apologize to all of our followers, especially the ones that clicked the link in the DM and were caught by the phishing scam themselves.

Here’s a rundown of the event in the hope that it will help others know what to look out for.

What Happened?

The phishing scam works like this:

1. You receive a strange yet intriguing Direct Message from someone you follow and likely trust. This is the key element to the scams success.
2. The DM contains a link using a shortened URL such as dwarfurl.com/blah. In our case, most of them were using dwarfurl.com, wapurl.co.uk, and 3.ly
3. You click the link and get taken to what appears to be the Twitter login page. But if you look at the URL it is actually something like blogs.videos.dsfasdc.com or videos.twitter.dsfasdc.com. Checking the URL is the key to making sure the scam doesn’t get you too!
4. You enter your Twitter login details. Reports of what happens after this login page vary. You may see the Twitter fail whale, or a blank page, or a random blog.
5. Now that the phishing site has your login details, the same Direct Messages is sent to all your Twitter contacts.
6. You eventually discover what happened. You feel like a violated idiot and start scrambling to fix everything.

What to do if it happens to you

If the above sounds familiar, you need to login to Twitter right now and change your password to make sure the phishing site can no longer access your account. You also need to go to the Connections tab and disable any third party applications that look suspicious. You’ll then need to update the credentials in all the twitter clients, website/blog plug-ins, and anything else that may be using your old Twitter credentials.

Fortunately, we were still able to login to our Twitter account and change our password and disable third party connections. Thankfully there were not any new suspicious connections that we needed to worry about.

Lessons Learned

Now that we’ve fixed everything and regained control of our Twitter account, it’s good to sit back and reflect on what just happened and how to avoid it in the future.

You’ve probably heard all of this before. We had too. But it takes an incident like this to really think about and address any shortfalls in your own organization. Some of our followers were also caught out by the scam and these are people that are in the tech industry and generally know about these sorts of scams. We were definitely surprised that we fell for it! So take a moment of your time to imagine your own Twitter account was compromised in the same way, then imagine all the possible ways it could have happened. Now go and take every precaution to ensure it doesn’t happen.

Having now been through it, here are some tips to help you avoid the same fate in the future.

1. Just because a Direct Message comes from someone you trust, does not mean it is trustworthy. Always use caution!
2. Educate your employees – especially those that know your company’s Twitter credentials. The main goal you want to achieve here is getting your employees into the habit of glancing at the URL in the address bar of their browser before entering ANY login details. We used our own log analysis software (Vantage) to find out who ended up on the websites in question, and then spoke to them directly to ensure they understood what to look out for.
3. Use a Twitter application that can display the actual URL behind a shortened URL before clicking on the link. For TweetDeck users, go to Settings | General, and check ‘Show preview information for short URLs’. Please note, however that this function only works for a few specific URL shortening services.
4. If you’re using the Twitter web page directly, use a browser and plug-in that can expand shortened URLs such as Mozilla Firefox with Long URL Please.
5. Use a browser with integrated anti-phishing security (such as Firefox or Google Chrome) and keep it up to date, or ensure you have good third party anti-phishing / anti-malware software installed.
6. As always, keep your security software and OS up to date.

Our friends at Sophos also have some good information about the scam that you may like to read: http://www.sophos.com/blogs/sophoslabs/?p=7366

Sorry!

An event like this makes you realize how important Twitter is to the overall public perception of a company. Our followers trust us to deliver relevant and useful content about our key areas of expertise – log file analysis and reporting. We spend a large amount of effort researching and writing content to ensure our tweets provide our followers with a good source of information. Having a breach like this certainly degrades this public perception that we work so hard at trying to maintain.

I would therefore like to thank all our followers who have kept with us and not clicked the ‘Unfollow’ button. Now that everything is under control again we will continue to bring you the best content we can provide about the log analysis and surrounding industries.

Once again, many many apologies to all of our followers, especially those that were affected.

New research from Robert Half Technology indicates that over half of chief information officers (CIOs) do not allow employees to visit social networking sites for any reason while they’re at work. This information comes from a survey of 1,400 CIOs from companies around the US with 100 or more employees.

Is this strictness justified? Amber Naslund makes some good points in a WebProNews interview. Among other things she says that instead of employers telling their staff how they should not be using social media, they should try balancing that by giving them some ways that they should use it.

What I say – People are not robots, employers need to find a middle-way

In some parts of the world, bandwidth is charged at exorbitant rates, so some sort of safeguard and watchfulness are required from a pure cost standpoint.

I feel it is unreasonable, however, to expect employees to work without some sort of social interaction, especially in a client-facing role. Humans are social creatures and being social stimulates all sorts of wonderful brain activity; production CAN be increased as long as employees understand what REASONABLE use of the Internet entails.

“Phone bills are itemized, and web browsing should be too!”

More often than not, this distinction is not explained or understood. We all have telephones on our desk these days, but it would be unthinkable for us as employees to spend all day chatting to our wives, husbands, girlfriends, boyfriends. Phone bills are itemized, and web browsing should be too.

While this may come across as a plug for our products, it is far more than that. I truly believe that blocking defeats the purpose and the spirit of the Internet in general. I’ve dealt with companies that lock down everything except for 50 sites. I’ve also dealt with companies that perform no monitoring, reporting, or any such user governance. Both forms of policies (if you can call the latter example a policy) are extreme and do no good in the long term.

At WebSpy we have found the majority consensus of our customer base to be alike: they return to us and tell us that their bandwidth usage, and their employee browsing habits have all changed for the better when their employees have it explained to them that before work, after work and during lunch, no one really cares what you do or where you go – within reason (and seriously, browsing for porn at work is unbelievably stupid) – and that everything you say and do online bears some reflection back to your employer.

Education always triumphs over draconian measures!

Rather than sneer at your user base with some ideological feeling of moral superiority, educate and explain to them what the consequences of their browsing habits are. Ultimately, they will still appreciate their pay cheque arriving and will avoid the chance of jeopardizing such things. Inserting Internet usage policies into contracts protects the employer and informs the employee exactly where he stands.

Would love to hear YOUR thoughts on this subject!

In an attempt to prevent the use of public proxies, it is common practice to subscribe to, or collect, regularly updated lists of public proxy sites. Many of these lists are freely available on the Internet (see examples at the bottom of the page).

We recommend the following procedures, using WebSpy’s solutions, to improve management and control of public proxies use

CREATE PUBLIC PROXIES PROFILE

• In your Summaries section, run an analysis, then go to Sites / Site Name
• Right click on any site known to be a public proxy site and chose `Include in profile’
• Either create a new profile called `Public Proxies’ or add to your existing `Public Proxies’ profile
• If you suspect a site to be a public proxy, simply right click and choose `Browse’ to investigate further

UPDATING PUBLIC PROXIES PROFILE

• Locate and copy a public proxy list published online
• In your Profile section, open your `Public Proxy’ profile and paste the list
• Even though a little bit time consuming, maintaining a list of the most common public proxies will increase your chances of easily locating novice public proxy culprits

INVESTIGATING INDIVIDUAL USERS

• In your Summaries section, run an analysis, then click on `Users’
• Right click on your selected user, then choose Drilldown | Sites / Site Name
• Many public proxies use IP addresses (as opposed to site names) to avoid easy detection, so a spike in IP address visits could be an indication that an employee or student may be using a public proxy
• Right click any IP address and choose `Browse’ to investigate further

(Please ensure that comprehensive Acceptable Usage Policies, prohibiting the use of public proxies, and breach consequences are explicitly communicated to employees)

There are numerous websites publishing updated public proxy lists online that can easily be located through search engines. Below are just a few examples:

http://www.publicproxyservers.com/

http://www.proxy4free.com/

http://bestproxy.info/

http://tools.rosinstrument.com/proxy/

http://www.fresh-proxy-list.net/

Don’t hesitate to contact us for further information.

Here’s a short summary of the main topics discussed during the event. For more information please follow the related links at the bottom of the page.

Event Summary

Sophos’s Asia Pacific Managing Director initiated the event by discussing organized cyber crimes. He highlighted that online organized crime rates are escalating rapidly. Online criminals are becoming increasingly sophisticated in the techniques they use to try and scam private people and businesses alike. Unfortunately their techniques are evolving much faster than legislation and community awareness, estimated to be at least 12 months behind.

Police Cyber Crime Unit

The representative from WA’s Cyber Crime Unit expanded on the safety of children, and how Internet predators are becoming increasingly Internet savvy and often avoid getting caught by engaging in their illegal activities at work. Nevertheless the audience was happy to learn about a recent case where the Cyber Crime Unit successfully tracked down an online predator, who had managed to stay anonymous for a long time by exclusively using his employer’s Internet resources. He worked for a very large organization, but thanks to the employer’s internal security and monitoring system he was identified before he had the chance to commit further crimes.

Hiding in a Wireless Hotspot

Wireless hotspots, today free at many airports, coffee shops and fast food chains, was another concern raised by the police’s Cyber Crime Unit. More often than not, the companies providing this free access to customers do not have a system in place to monitor and alert on any inappropriate or illegal activities. When this is the case it is virtually impossible to prevent predators using these networks to stay anonymous. Unfortunately, legislation, or public outcry, to address the issue is not likely to occur until an illegal activity, enabled by the anonymous use of wireless hot spots, takes place and receives media attention.

At School

The representative from the educational sector continued to discuss online safety related to children and students. He highlighted that security system at schools and universities are essential, but not always enough. On many occasions students bypass the school’s firewall by using virus ridden public proxies to access blocked sites. Even students with studious intent occasionally use public proxies to access legitimate sites that have been blocked thanks to a “block worthy” word in a corporate blog, or something of similar virtuousness.

The importance of educating children about the dangers of social networking was also emphasized. Children are often overconfident in their abilities to spot a predator among their peers. However, in reality, they do not fully comprehend the psychological techniques used by online prowlers to gain their trust.

Best Practices

To sum up, the event focused the changing landscape of the internet and internet security. In our internet dependent world everyone is at risk, whether at work, at home, or at school. The best practices, when working towards a safer e-environment, keeps changing but the proactive theme throughout the event emphasized a combination of security systems, system monitoring, education of workforce (parents, teachers, students), and an increased involvement from all levels within organizations and community.

Related Links:
Western Australia Internet Association
Australian Communications and Media Authority
Wise up to IT
Cyber Smart Kids
Stay Safe
Think U Know
Virtual Global Task Force
Sophos

The attack was quickly under control and IT updated firewall rules to prevent any employees from accessing the particular site again. However, instead of immediately blocking further emails from the malicious sender, the IT department saw this as an opportunity to educate the workforce about phishing attacks. They used WebSpy Vantage to report on their firewall and identified all employees who tried to access blocked phishing sites. As employees were identified, IT started organizing informal meetings to inform them about phishing, how to recognize attacks and the most common phishing techniques used.

Because IT was able to pinpoint exactly which employees needed more information about phishing attacks, these smaller meetings showed to be very effective. The employees attending the meetings naturally realized their phishing knowledge was insufficient and were eager to find out more as they didn’t want to make the same mistake again, at work or at home. The government department is still the target of phishing attacks but their employees are now educated enough to identify them before any harm is done.

If you want to share your monitoring and reporting experiences either comment below or email me directly at asa@webspy.com.

To use Windows Authentication, there are just a few things you need to do.

1. Set the Web Module’s Authentication type to IIS Integrated, and add your administrators in the form of domain\username
2. Enable Windows Authentication and disable Anonymous authentication in IIS.
3. Ensure all users in your Organization screen have a login name in the form of domain\username. Use the ‘Prefix’ option to prefix “domain\” (without the quotes) to your usernames names when importing your Organization from LDAP or LDIF.
4. Connect Vantage and the Web Module using the new authentication details and synchronize your Organization.

1. Set the Web Module’s Authentication type to IIS Integrated, and add your Administrators.

When you first install the Web Module, the first screen you see is the ‘Initial Configuration Wizard’ that guides you through the process of selecting your authentication type and specifying your administrator(s). If you have already been through this Wizard and are currently using Vantage In-Built or Client Certificate authentication, you can easily reset this initial configuration wizard. Simply login to the Web Module with your current administrator details and go to Options | Maintenance | Reset Initial Configuration Wizard.

Note: You can also change your authentication and administrator options individually using the Authentication and Administrator options on the Options tab of the Web Module.  However, for ease of demonstration, I’ll use the Initial Configuration Wizard method.

Now that you’re at the Initial Configuration Wizard, proceed through the wizard, selecting IIS Integrated authentication and entering your administrators in the form of domain\username (replace domain with your organization’s AD domain, and username with the sAMAccountName of your administrator.

Initial Configuration Wizard - Welcome Page

Initial Configuration Wizard - Authentication Page

Initial Configuration Wizard - Delegate Administrators Page

Initial Configuration Wizard - Summary Page

Click Finish, and if the authentication was successfully changed, you should get a message saying ‘The specified credentials were not accepted”.

The 'Specified credentials were not accepted' message.

Don’t panic at this point. This message is an indication that the authentication was successfully changed and that the Web Module is now listening for IIS to pass through Windows Usernames. The reason you’re getting this message is because IIS is not yet passing through Windows Usernames to the Web Module. This is configured in the next step.

2. Enable Windows Authentication and disable Anonymous authentication in IIS.

Now that the Web Module is expecting IIS to authenticate your users, you need to set up  IIS  to do this.

1. Open IIS by navigating to Start | Control Panel | Administrative Tools and double-clicking on Internet Information Services (IIS) Manager.
2. Navigate to the Web Module site or virtual directory in the left hand ‘Connections’ Panel. It will be located under <Server Name>\<Sites>. For example, MyServer->Sites->Default Web Site->webmodule.

• If you’re running IIS7 ( Windows Server 2008, Vista or Windows 7)
  1. Select the Web Module site and ensure the ‘Features’ tab is selected at the bottom of the middle pane.
  2. Double-click the ‘Authentication’ feature.
  3. Right-click ‘Anonymous Authentication’ and select Disable
  4. Right-click and ‘Windows Authentication’ and select Enable
  5. Restart IIS by selecting your server in the right hand connections pane, and clicking Restart in the ‘Actions’ pane on the right.

• If you’re running IIS6 or 5.1 (Windows Server 2003, Windows XP)
  1. Right-click Web Module site and select Properties.
  2. Go to the Directory Security tab
  3. Under ‘Authentication and access control’ click the Edit button.
  4. Uncheck ‘Enable anonymous access’ and check ‘Integrated Windows authentication’
  5. Restart IIS by right-clicking the local server, select All Tasks, and then click Restart IIS.

If you added your own Windows login name as an administrator in step 1, you can now test the authentication is working. Go back to the Web Module in your browser and click Refresh. You will be presented with an ‘Authentication Required’ dialog where you can enter your username and password.

Authentication Required Dialog

Again, ensure your username is in the form of domain\username. Click OK, and you should log straight into the Web Module using Windows Authentication.

3. Ensure all users in your Organization screen have a login name in the form of domain\username

Now your administrator account can log into the Web Module using Windows Authentication, but all other users will not be able to log in unless they have their login name specified in the form of domain\username. This is done in Vantage Ultimate on the Organization screen.

Organization Screen showing correct login name for Windows Authentication

If you’re importing your users from LDAP or LDIF, make sure you use the ‘Prefix’ option on the User Details page to prefix domain\ before your imported usernames. For example:

4. Connect Vantage and the Web Module using the new authentication details, and synchronize your Organization.

In order to publish information to the Web Module, you need to add a connection between Vantage and the Web Module. This is done on the Web Module screen in Vantage Ultimate.

1. Click Add Web Module (or if you already had a web module before changing the authentication details, select it and click Properties)
2. Enter the server & virtual directory of the Web module, and enter the correct credentials ensuring domain is specified.
3. Click OK to connect.

Connect to Web Module dialog

Once connected, synchronize Vantage with the Web Module by clicking the Synchronize link in the Web Module task pad. You may also want to provide permissions for your users in the Permissions section on the Web Module screen.

That’s it. You can now test that everything is working by getting one of your users to access the Web Module’s URL. They should sail straight in with no username/password prompt.

I hope this helps! Please let me know your feedback by emailing me, or leaving a comment below.

Something of great importance is taking the users of the network you intend to monitor into consideration. Overly intrusive practices can easily create the negative perception that Big Brother is watching and make employees feel frustrated and uncomfortable. Effective Internet monitoring requires a two-pronged approach; intuitive monitoring software AND workforce education / consideration.

This time around I would like to expand on the best ways of monitoring your organizational Internet usage whilst maintaining a harmonious working environment between employers and employees.

1. Allow for a certain amount of personal / recreational usage

Prohibiting all personal use is usually both impractical and virtually impossible to enforce in most work environments. Allowing a certain amount of (monitored) online recreation can enhance many workplaces and ultimately make employees more productive. Recent research by Dr Brent Coker at the University of Melbourne shows that people who do surf the Internet for fun at work – within a reasonable limit of less than 20% of their total time in the office – are more productive by about 9% than those who don’t.

2. Allow for a certain amount unmonitored usage

Employee privacy is a recurring concern when monitoring Internet usage at work. Employees are working longer hours than they ever before and might need to be able to deal with urgent personal matters online, during work time. By specifying and ensuring, that there will be no monitoring during lunch hours (for example), your employees can trust that their privacy is important to you, and you know it won’t affect their productivity.

3. Establish Acceptable Internet Usage Policies

Establish policies around Internet usage and:

• Explain the business-related reasons behind monitoring
• Clearly state what is considered productive and unproductive activity
• If you allow a certain amount of personal use, and / or unmonitored use during certain hours of the day, ensure you outline the exact specifications of these privileges
• Clearly state what is absolutely prohibited, for example, sending or accessing discriminatory, harassing, defamatory, or pornographic material, downloading or distributing copyrighted material without permission etc
• Include consequences for policy violations

4. Use an honest and open monitoring approach

The effectiveness of Internet monitoring directly relates to employees’ awareness of the content of the policy and corresponding breach consequences. Once your crystal clear policies have been developed, ensure you actively distribute, publish and communicate them so employees understand exactly what is expected of them and the conditions of their working environment.

5. Allow employees to view their own Internet usage

This is one of the best recommendations we can give you. More often than not, employees tend to underestimate the time they spend browsing non-work related sites. Allowing employees to view, for example, their productive and non-productive activity can help foster and drive responsible Internet usage behaviour. Employees who understand the organizational costs of their personal unproductive activities are more likely to accept your monitoring activities and modify their own behaviour accordingly.

6. Monitor the whole organization (even managers)

If you ensure everyone knows the whole organization is being monitored, as opposed to individual users or departments, you will decrease the likelihood employees feeling singled out or treated unfairly. Employees feel affirmed if procedures are adopted to treat them with respect and dignity and the likelihood of Internet monitoring acceptance and effectiveness is increased.

7. Help employees sticking to the rules

If you have set a limit of, for example, no more than 10 hours of recreational surfing per month, then ensure you alert employees when they are approaching that limit. Again, this will give the employees another opportunity to modify their own behaviour before they actually violate your Acceptable Internet Usage Policy.

8. Distribute reports – distribute responsibility

Frequently IT managers and administrators are given the ultimate responsibility of managing, enforcing and communicating acceptable Internet usage for an entire organization. Take some of the pressure off the IT department and distribute organizational Internet activity reports to responsible managers or department heads. This will enable them to see how Internet usage affects the security and performance of their own department and distributes the responsibility of enforcing acceptable usage with the managers themselves.

9. Protect employee privacy

If distributing Internet usage reports across your organization it is important to protect employees’ personal data. Make sure you use monitoring software designed to protect privacy rights by only allowing authorized users to see the employee’s identity. For instance, Network Administrators may need to investigate all traffic going to a particular site but should not need to know the user names – in this case user names should be anonymous for them but available for HR.

10. Automation

Find a monitoring solution that easily lets you customize and automate the majority of these guidelines for you. Right here might be a good place to start looking…

What do you think of these tips? Please feel free to comment below and share Internet monitoring tips that have assisted you in creating a productive and balanced working environment.