This video covers registering Vantage and the Web Module.

This video will cover the Organization section of the software, showing you how to import your users and groups from Active Directory / LDAP.

This video follows on from #3, and will cover the Profiles section of the software, and how to use site categorization.

WARNING: With storage locking disabled, it is possible to import into a storage while running a report, and doing this may cause storage corruption. It is therefore very important if you decide to enable these features to ensure that a storage is not written to while running reports.

Due to the experimental nature of these features, they can only be enabled by including a config file next to Vantage’s executable. To enable multi-instance capabilities and disable storage locking:

1. Download the following config file:
   WebSpy.Vantage.exe.config
2. Close Vantage
3. Extract downloaded zip file into Vantage’s installation folder (usually c:\Program Files (x86)\WebSpy\Vantage <flavour> 2.2). If you already have a file of the same name in that location, make a backup of it before overwriting it with the  new file.
4. Run Vantage.

You can now run Vantage again to launch another instance of the application.

Be aware of:

Simultaneous reading and writing, and multiple writes

I just want to be very clear that if you run reports while importing, or import into the same storage simultaneously, storage corruption can occur. Storages are not designed to be unlocked for these reasons. The only reason we’ve provided this ability is so that you can READ from the a single storage  simultaneously (i.e. run two or more reports). Reading and writing, and multiple writing is NOT supported, but Vantage will attempt to do it if you ask it to, with undefined behavior.  Check your Tasks configuration and note when any import jobs are likely to occur to avoid running reports at these times.

Configuration Changes

When Vantage closes it writes all of it’s state to a series of files under c:\users\<user profile>\AppData\Roaming\WebSpy\Vantage <flavour> 2.2). When Vantage opens, it loads these files into memory. When  running multiple instances, these instances will be reading and writing the same files. So if you open two instances of Vantage, make a change to a report template in one instance, then close the application, the Vantage.Templates file will be updated. But when you close the second instance of the application, the Vantage.Templates file will be overwritten with a version that doesn’t include the change.

When making configuration changes (templates, tasks, aliases, organization etc), make sure only one instance is running (check Task Manager for the WebSpy.Vantage.exe process).

It’s Experimental!

There may be other undefined behaviors that we are yet unaware of, so we advise running this configuration in a test environment.

We’re providing these feature on an “as-is” basis, meaning we will not be providing technical support for issues that arise as a result. That said, we are certainly interested to hear about any issue to help us improve the feature.

Let us know how you go!

Of note, this release includes full support for Microsoft Exchange 2010 Tracking logs (previously supported with the Exchange 2007 loader, but missing a few fields), as well as JunOS (Juniper) Traffic Logs, IronPort Traffic Monitor Logs and Squid Syslog.

We’ve also included an experimental feature to allow multiple instances of WebSpy Vantage to run on the same operating system. The goal here is to run multiple reports at the same time using multiple instances of the application. To do this, we have also included a second experimental feature to disable storage locking. This allows multiple instances of Vantage to read from the same storage at once. These features can only be enabled by including a config file next to the Vantage’s executable. More on this feature here.

Here’s the full list of changes:

Application Changes

• New: Added suffix option to Import Windows Users wizard in Aliases.
• New: Date modifiers now supports h for hour and n for minute, e.g. %[-2h,yyyyMM - HH].
• New: Added tracing to storage publish task.
• Experimental: Multiple instances of Vantage can now be run simultaneously, by adding the multipleInstance key to the application config file.
• Experimental: Storage locking can be turned off to allow multiple instances of Vantage to run reports on a single storage simultaneously. This is done by adding the storageLocking key to the application config file.
• Fix: Import Organization merge options now appends attributes if keep existing user details is selected, and replaces attributes if update user details from the directory is selected.
• Fix: Import Organization merge no longer replaces user’s passwords.
• Fix: Fixed issue where no results were returned when filtering on time less than one day – such as past n hours.
• Fix: Storages are no longer duplicated in the Import new hits task dialog.
• Fix: Fixed issues where the Site Domain summary included sub-domains for European domains (.fr, .be etc).
• Fix: SQL server inputs now commit correctly if the user edits the input and only changes the port number.

Loader Changes

• New: IronPort Traffic Monitor Logs.
• New: Juniper JunOS Traffic Logs (SRX).
• New: Microsoft Exchange 2010.
• New: Squid Syslog.
• Improved: Astaro Security Gateway: Added support for an additional different syslog header.
• Improved: SonicWall: Split syslog format into Web and Firewall schemas, added support for User field, string-type Category field and split Protocol field.
• Fix: Microsoft FTMG: Changed type of Object Source field in from Int32 to String. Users will need to clear/field select/reload their storages before this change will apply.
• Fix: Astaro Mail Gateway: Improved format detection, fixed negative size issue, and Index out of bounds errors.
• Fix: IronPort WSA: Improved format detection.

How to update

To update your software, simply click Tools | Check for updates. Vantage Ultimate users will also need to update the Web Module in order to use the new loader formats that have been added. To update the Vantage Web Module, right-click the WebSpy system tray icon and select ‘Check for updates’. If you have issues with the Web Module update process, please see: http://www.webspy.com.au/forums/viewtopic.php?f=4&t=29

Let me know if you have any questions or issues!

This time around I thought I share some tips focusing on how individuals can ensure they’re being smart and safe when engaging in social networking activities. Hopefully these tips can assist you, or the people working in your organization, in avoiding being exposed to the dark side of social networking where hackers, identity thieves, fraudsters and stalkers are lurking in the shadows.

I’ve picked five tips I thought were quite helpful and perhaps not as obvious as not sharing personal information such as birth date and address, keeping passwords safe, be selective whose friendship requests you accept, use appropriate virus/phishing software etc.

1. Limit work history details on LinkedIn

Although tempting, it is not advisable to use LinkedIn as an online resume displaying all information about time and place with previous employers and educational institutions.

Too much personal information publicly available simply makes it too easy for identity thieves to use the information to fill out loan applications or guess password security questions. A hacker recently used social reverse engineering to find information, such as post code, home town, high school graduated from etc, in order to hack into VP candidate Sarah Palin’s web mail account.

If you really feel the detailed information will help you when looking for a new job then expand the details during the job hunting process and cut back later after you have a position.

LinkedIn also offers some capabilities to restrict information. You can close off access by others to your network of contacts, something you don’t have to share if you don’t want. This is a common practice by sales professionals and recruiters not wanting to expose their valuable network to others who might poach customers or prospects from them.

2. Avoid sharing location information

Hopefully you’re already aware that sharing phone and address details online is a very bad idea. You might also think twice before posting “Away for the weekend…returning on Monday” on public sites like Twitter and LinkedIn, to avoid attracting attention from potential stalkers or burglars.

What you might not have considered is that, over time, seemingly innocent and non-specific information can be pieced together, giving lurkers a much more complete and rich picture of you, your family, your habits and other personal information.

Software like Twitter and Foursquare are often used at conferences, parties and other social scenes where alcohol is consumed, increasing chances of personal and location information slipping out. Travel plans and experiences while on holiday is often communicated via Twitter, giving clue to others that you’re not at home, leaving your family or possessions at risk for intruders.

Your Foursquare check-ins to local restaurants and shops can also make it fairly simple to determine what area you live in and your habits. Something as innocent as a Foursquare airport check-in is a tell-tale sign you’ll be away from home for at least one night.

3. Search yourself

Sometimes referred to as vanity search, it is a very good idea to search your name on Google and check out your profile as others see it on social networking sites. Understand where you show up and what information is available about you, and then adjust your profile, settings and habits appropriately.

If you unexpectedly see your name in locations you don’t frequent, it could give you a heads up someone else is using your identity online. Set up a Google alert with your name. Google Alerts will email you weekly, daily or immediate notifications when the Google robots comes across your name online.

4. Setup an OpenID account

OpenID is an open source standard for creating a single sign-on to multiple online services and applications. With OpenID, your password is only given to your identity provider, and that provider then confirms your identity to the websites you visit. Other than your provider, no website ever sees your password, so you don’t need to worry about an unscrupulous or insecure website compromising your identity

As a framework, OpenID accounts are available from multiple providers. Companies like AOL, Microsoft, Sun, and Novell are beginning to accept and provide OpenIDs. It is estimated that there are over 160-million OpenID enabled URIs with nearly ten-thousand sites supporting OpenID logins.

OpenID is making inroads into the SaaS application market to better manage user accounts. We’re also likely to see OpenID used in online social networking sites to help verify users identities and reduce impersonators and false identities. If the social networking sites you frequent don’t use OpenID or a similar technology, e-mail the site creator and lobby for adding it.

5. Don’t violate your company’s social networking policies

As blogging and social networking sites enter the workplace corporate acceptable use policies (AUP) are being updated to define boundaries for employees, contractors and the company.

Data leakage incidents (loss of corporate, confidential or customer information), making inappropriate public statements about the company, using corporate resources for personal uses and harassing or inappropriate behavior toward another employee can all be grounds for reprimand or dismissal. Social networking sites are another way those things can happen and they create an easy digital paper trail to investigate.

Data leakage (or loss) prevention is currently one of the hottest areas in security. Companies are looking for ways to prevent company confidential and proprietary information from slipping through the firewall. Most incidents probably occur via email or file transfers but IM chat tools, blog posts, Twitter messages and even online resume content could disclose proprietary company information.

Even using social networking sites on company time or using company resources could be a violation of the company’s acceptable use policy. Before you become the corporate poster child for some publically humiliating episode from using social networks at work, check your corporate AUP to make sure you aren’t violating the policy.

Other Resources
http://www.microsoft.com/protect/parents/social/socialnet.aspx
http://www.networkworld.com/community/tips-for-safe-social-networking

You can also read through these steps on this page: Analyzing SonicWALL log files with WebSpy.

Creating and Importing SonicWALL log files

Analyzing SonicWALL log files

We intend to make some SonicWALL specific report templates available on our SonicWALL how to page soon.

Until then, feel free to create your own templates, or modify our existing web reports to include the extra goodies contained in the SonicWALL logs.

TIP: To modify an existing web report, right-click the report and choose ‘Duplicate template’. Then choose the “SonicWall Web” schema. You’ll then have a report template that you can modify to include all the SonicWALL summaries, such as Categories, and Source and Destination Interface.

If you need some assistance getting the report(s) you need, feel free to contact me, or support@webspy.com.

Take a look at our dedicated Astaro pages to get an idea of what can be achieved when analyzing Astaro Web Gateway log files with WebSpy Vantage.

I’ve created some quick videos to show you how to enable the correct logging options on the Astaro Security Gateway appliance, how to import these log files into Vantage, and analyze the data on the Summaries screen.

Configure Logging

The best way to configure logging is to setup a 3rd party syslog server (such as Kiwi Syslog) on a machine in your network, then configure the Astaro Security Gateway to send syslog messages to that server. The syslog server then creates log files that can be imported into WebSpy Vantage. This video takes you through that process.

Importing and Analyzing Astaro logs

Once you have successfully configured syslogging on your Astaro Security Gateway, you can import the log files into WebSpy Vantage and analyze activity on the Summaries screen. This video takes you through that process.

We intend to make some Astaro specific report templates available on our Astaro How To page soon.

Until then, feel free to create your own templates, or modify our existing web reports to include the extra goodies contained in the Astaro logs.

TIP: To modify an existing web report, right-click the report and choose ‘Duplicate template’. Then choose the “Astaro Security Gateway – Filter with category” schema. You’ll then have a report template that you can modify to include all the Astaro summaries, such as Actions and Categories.

If you need some assistance getting the report(s) you need, feel free to contact me, or support@webspy.com.

Cyber bullying is legally defined as repeated harassment online, although in popular use, it can describe even a sharp-elbowed, unwarranted swipe online. We all know kids can be cruel because they often lack the maturity and empathy to understand the emotional ramifications their words or actions can have on others. Adding the anonymity of the Internet, cyber bullying can be more psychologically savage than schoolyard bullying. The Internet erases inhibitions and adolescents often take the bullying much further online than in person.

The article describes a number of bullying cases played out in the US over the last few years. It also goes into detail on how the bullies, victims, parents, schools, and the authority responded in each instance.

D.C. and the forged Facebook profile

The main story focus on Marie and her son D.C. The kids at school started to avoid D.C. as he allegedly was posting horrible comments about other kids on his Facebook profile. As a matter of fact D.C. didn’t even have a Facebook account and it turned out someone had forged his identity on Facebook, and was bullying others in his name. Marie was desperate to make it stop as the ongoing online bullying had detrimental effect on, not only her son, but also the kids targeted on D.C’s forged Facebook account.

When D.C’s mum contacted school officials to help track down the students who was making her son miserable she was told there was nothing they could do. It was an off-campus matter.

Finally, after months and months of continued harassment, the police was able to subpoena Facebook for the address of the computer linked to the forged profile and much later subpoena Comcast, the Internet service provider, for the home address of the computer’s owner. Three boys were identified to be behind the scheme.

The Common Thread

The lawlessness of the Internet, its potential for casual, breathtaking cruelty, and its capacity to cloak a bully’s identity all present slippery new challenges for kids and parents. This is a dark, vicious side of adolescence, enabled and magnified by technology.

The article covered a variety of online bullying scenarios and the common thread running through all of them is the parents’ helplessness and frustration about the school’s inability and reluctance to intervene and proactively protect the students.

Yes, a large chunk of the responsibility lies with the parents. It even starts outside the borders of technology by simply raising mindful and responsible children. However, even if parents got that part right, in addition to restricting, monitoring, discussing, and educating themselves and their children about safe and responsible Internet use, the schools also have responsibility to ensure their internet resources are not used for cyber bullying. The negligent approach of passing off D.C’s case as an off-campus matter is simply unacceptable. If even one of the malicious Facebook updates happened during school hours, on the schools network, identifying the culprits could have taken a few minutes, as opposed to months!

What the school should have done

Majority of schools require students to use individual login details to access the Internet. This means an IT administrator could have easily match students’ Facebook activity with the timing of the malicious Facebook updates on D.C’s public profile and quickly narrowed down the list of suspects. Even better, as Facebook generates a specific URL every time someone updates their Facebook profile, a report filtering out all other Facebook activity would have been an even more efficient option.

Here’s how you filter based on Facebook updates using Vantage:

Let’s say you want to track down the source of malicious Facebook updates made during a specific time period.

• Make sure you have access to the log files created during the period you which to investigate. You either need to import them to a new storage or apply a date filter when running a new analysis.
• In the ‘Summaries’ screen click ‘New Analysis’ and make sure your storage is selected.
• Click through the wizard and add a date filter if your storage include dates you are not interested in investigating.
• In the ‘Filters’ section click ‘Add’ and select ‘Field value filter’.
• In the ‘Summary’ drop down select ‘Site URL’.
• Click ‘Add’ and enter value: http://www.facebook.com/ajax/updatestatus.php (This is the URL recorded every time someone updates their facebook profile).

   

  

   

• Click OK in all wizard windows until the new analysis starts running.
• Once the new analysis has been completed click ‘Users’ and you will be able to see which users/student posted Facebook updates during your selected time period and drilldown (right click) into each user and select ‘Individual Record’ to get the exact time of the Facebook updates.

   

  

   

  

  Click to enlarge

• Alternatively you can start by looking at the dates in question, drilldown (right click) into each user for a specific date and drilldown again into ‘Individual Records’ for the exact time of the specific user’s update

Every organization is different with regards to the volume of logs it creates, but I’ve averaged three data sets submitted to us by customers to produce the following estimates.

You will create roughly 0.9 MB of IronPort WSA access logs per user per day.

Once imported into a WebSpy Storage, the Storage will be about 90% of your original log file size. If you apply NTFS compression to the storage folder, the actual size on disk of the WebSpy Storage will be about 30% of your original log data.

So an organization with 1000 users will produce about 900 MBs of access logs per day. The default WebSpy Storage  will be 810 MB, but with NTFS compression, the size on disk will around 270 MB.

As I said, this is a rough guide based on the average of three sets of sample logs we have in house, so please run your own tests and if you can, let us know your values in the comments below.

But there is another advantage to switching to text. They take up considerably less disk space. Here are some figures:

Number of Records in 235 MBs of log data:

235 MB of TMG’s W3C text logs contains 326,824 records. An SQL Express database of the same size (mdf and ldf files) contains only 40,308 records. In other words, w3C text logs can store over 8 times as much data in the same amount of disk space.

A rule of thumb:

By switching to W3C text logs, the disk space taken by your log files will be roughly 12% of the SQL Express or MSDE log files. This can be reduced even further by compressing your text logs.

• MSDE/SQL logs: budget for 5 KB per record
• W3C Text logs: budget for 0.71 KB per record

How many records your ISA or TMG server creates per day will depend on the number of users in your organization and how much traffic they generate, but about 16,000 records per user is a reasonable estimate.

A real world example

If you are hitting 500 GB of SQL Express/ MSDE logs per month (about 86,128,205 records), simply switching to W3C text logs will reduce this down to 61 GB.

Once imported into a WebSpy Storage, the storage size would be roughly 53 GB (87% of the original W3C text logs).

With NTFS compression applied to the Storage folder, the WebSpy Storage would be roughly 13.4 GB (22% of the original W3C text logs).

Applying NTFS compression to your WebSpy Storages folder is certainly a good idea. This does not impact performance. If anything, it may improve performance slightly as there is less disk fragmentation within the storage.

Disadvantages and Alternatives

Please be aware that by changing your logging to text, the default reporting functionality within TMG will no longer work. However, the reporting supplied by WebSpy Vantage should more than adequately replace this feature.

If you are still concerned about changing the logging method, you can utilize a script published by Microsoft to convert your SQL Express logs to W3C text.  You can then keep the text logs and set some more stringent data retention policies on the SQL Express logs, such as clearing logs every week. You can download this script as part of the Microsoft Forefront Threat Management Gateway (TMG) 2010 Tools & Software Development Kit.

Additional Resources

• Here’s a great article by Marc Grote at isaserver.org on the pros and cons of the different logging options in ISA and TMG. It also takes you through how to exclude fields to reduce the amount of data being logged:
  http://www.isaserver.org/tutorials/Microsoft-Forefront-TMG-Logging-options-Forefront-TMG.html
• Also take a look at Richard Hicks’ blog regarding MSDE performance with ISA Server 2006:
  http://tmgblog.richardhicks.com/2009/10/31/msde-performance-with-microsoft-isa-server-2006/
• Here’s another article on isaserver.org by Richard Hicks on the logging enhancements in TMG 2010
  http://www.isaserver.org/articles/Logging-Enhancement-Microsoft-Forefront-Threat-Management-Gateway-TMG-2010.html

The figures above were produced using some sample logs received from customers with similar (but not exactly the same) logging settings. If you have changed to text logging, I’d be very interested to hear the sort of disk savings you are seeing, and I’m sure others would to. So please leave a comment below.

The Claim

According to network security firm, Arbor Networks, traffic to Google sites broke a new record this month, and now accounts for 6.4% of all Internet traffic around the world.

The 6.4% includes all sites owned by Google, including Google’s search engine, YouTube, GMail, Google Maps, AdWords and Google’s office suite of products like Google Docs and Spreadsheets. The data was obtained through more than 110 ISPs in 17 countries.

Our Test

Using Vantage I ran an analysis on web proxy traffic for July – September 2010. I am aware the Google data is for September 2010 but wanted to test a larger set of data since I’m reporting on traffic from a smaller amount of users, compared to 110 ISPs in 17 countries.

How much of your traffic is going to Google sites?

To find out simply create an Alias and add all Google related sites.

• Click on the ‘Alias’ tab in Vantage.
• Double-click the ‘Web sites’ alias on the left-hand side and make sure ‘Use wildcard matching’ is selected. Click ‘OK’.
• Click ‘Add Group’ in the Groups task pad.
• Name the group something intuitive…I named mine ‘GOOGLE SITES’.
• Click ‘Add’ and type ‘*google*, youtube.com, ytimg.com, gmail.com and adwords.com’. The *google* will add all domains with the string google in it. ytimg.com is YouTube’s DNS and also needs to be added (see previous blog post for more information).
  
• Click ‘OK’
• Click on ‘Summaries’ tab
• Click ‘Site Domain’ to get a listing of sites
• Enable the ‘Web site’ Alias on the left hand side.
  
• Click the top of the column called ‘Size’ to sort domains in descending order
• Click the pie chart icon on your right-hand side
  
• Have a look at the pie chart to identify what percentage of your traffic is going to Google related sites.
  

  % of WebSpy traffic going to Google related sites

It is apparent that WebSpy traffic to Google sites makes up more than twice than the reported average. I’m also fortunate enough to have access to one of our larger client’s (approximately 25,000 users) most visited websites. The below pie chart is from one of their reports on site domain traffic for a random period in October.

% of traffic to Google sites from Enterprise size organization

How much of your traffic goes to Google websites?

We really encourage you to add a Google sites alias and have a look at your Google traffic. Please comment below on the percentage of your traffic going to Google sites, approximately how many users you are reporting on and the time period (about a month or more) you are reporting on.

Many thanks.

If you are, or considering, blocking Twitter at your workplace please have a look through these amazing numbers and think again. Seems like Twitter is here to stay doesn’t it? The distinction between personal and professional Twitter usage is diminishing as it is being adopted by the masses as an every day tool to communicate with friends, provide networking opportunities, locate critical contacts, leads and receive timely industry news. Blocking access to Twitter, or other social media sites, means blocking an online revolution and depriving the needs of a professional workforce. This in turn can cause resentment, increase costly turnovers and reduce productivity by complicating or delaying accomplishment of tasks.

Big thank you to the guys at www.website-monitoring.com for putting together this handy infographic. Website Monitoring monitors the availability of your site, not a WebSpy competitor .

After reading the article I continued with some quick and dirty research. It is pretty clear, and not very surprising, that only a minority of aged care facilities around the world offer Internet access to their residents. An even smaller minority actually promotes and educates their residents in the use of Internet and social media.

The article mentions that research, carried out in Melbourne, Australia, has shown that while there are many challenges, older people can learn how to use computers, email and social media, and derive huge benefits from doing so. The oldest participant in the research is a 99 year old man who is currently learning to Skype, to keep in touch with relatives in France.

Internet Use per Age Group

A study titled Generations Online in 2009, confirms that surfing the web is still primarily a young person’s game – 18 to 44 year olds accounting for 53% of the total number of users – but in recent years older generations have started closing the gap. The chart below that has been borrowed from the report shows the overall breakdown by age:

So, why is the Internet still a young person’s game? Dr Murnane, involved in recent Melbourne based study, says, “The way we talk about the internet, for example by referring to digital natives and immigrants, helps to build a culture of fear among the non-computer literate. We need to stop thinking about the internet as the preserve of the young; indeed, the way the World Wide Web enables us to explore, learn and communicate might have been especially designed for the elderly or disabled.”

According to Latest Research…

Sociologist Shelia Cotten, currently researching the ability of computer use and social media networking to enhance the quality of life of elderly, says “Many elderly are increasingly isolated and grapple with depression, loneliness and declines in physical health. With an increasing number of elderly living in long-term care facilities and declines in quality of life as people age, we need innovative ways to lessen these negative impacts and to enhance quality of life”

“Internet access provides an important opportunity for mental stimulation, which is closely tied to older people’s health,” said Dr Murnane. “It is also a liberating outlet for those confined to a single building on a day-to-day basis. Everyone living in retirement facilities deserves to experience these benefits.”

Within a few decades there will no doubt be a natural increase in pressure on aged care facilities to make internet easily accessible, as a growing number of computer-literate residents will move in. However, aged care facilities should not wait for residents to demand Internet usage but rather be proactive ensuring existing residents, that didn’t grow with computers and Internet as a part of their daily life, can access the many benefits the Internet provides.

Benefits for the Elderly

• Access to a wealth of knowledge:From health resources to breaking news to how-to-guides, the internet is a robust informational tool providing an important opportunity for mental stimulation, which is closely tied to older people’s health.
• Help decrease inequalities in access to health information due to age-related declines in mobility. An increasing amount of health information is available electronically, says Cotten. “Once older adults cross the digital divide, they can access health information much more easily using the Internet than they can go to the library or visit a health-care professional”
• Increased social opportunities: The Internet allows people worlds apart to communicate with ease through video conferencing, email, social networks, chat rooms, and discussion forums. Elderly can, with the use of the internet, easily connect with sources of social support, and stay in touch with family and friends regardless of their physical abilities.
• Improved mental health: The Phoenix Center, a non-profit organization that studies public-policy issues, found that “spending time online reduces depression by 20 percent in senior citizens” after examining survey responses from 7,000 retired Americans over the age of 55.
• Enhanced brain function: A study conducted at the Semel Institute for Neuroscience and Human Behavior at UCLA discovered that “surfing the web for only a week stimulated areas of the brain that control decision-making and complex reasoning in middle-aged and older adults with little internet experience.”
• Allow families to monitor their loved one’s health and quality of care through video conversations

You CAN Teach Old Dogs New Tricks

According to the National Institute of Aging, older age is not in itself a hindrance to computer or Internet use. However, older adults’ use of electronic technology may be affected by age-related changes in vision and in cognition—for example, the ability to remember, learn, think, and reason. Cognitive abilities that change with age and that are likely to affect computer use include working memory, perceptual speed, text comprehension, and spatial memory.

Saying that, recent research clearly disputes the widely-held beliefs that residents of aged care facilities and other elderly people are too old to learn to use the internet.

I know from personal experiences that someone who is a little bit older and completely technologically unequipped can, and want to, learn how to use new technology, especially when they can see the immense benefits that arise.

At a ripe age of 60 my dear mother is still far from an elderly person at an aged care facility. However, I’d say she started off on a very similar level including lack of basic technological understanding and even fear of this perceived complex and scary technological evolution. I am very proud to say that over the years, as I (her only daughter) have travelled and lived abroad, she has gone from making phone calls to sending emails, sending text messages, chatting on msn, talking on Skype and believe it or not, I recently convinced her to get an iPhone 4. Needless to say, she loves it! She is making Skype calls on her phone, downloading applications, taking and sharing photos and videos. She is amazed and ecstatic how technology can help her stay connected with dispersed family members and instantaneously share experiences with someone on the other side of the world.

To Sum Up…

We are the computer and Internet literate generation so it is our responsibility to make it available for those who grew up without the Internet as a part of their daily life. To sum up, there’s a few key things we can do to improve the life of elderly.

• Make Internet access mandatory at aged care facilities
• Take time to educate elderly in the use of Internet, email, social media and technologies that comes with it. This is everyone’s responsibility, children, grandchildren, aged care facilities, local governments; the list goes on and on.
• Keep the elderly in mind when developing websites and applications. I came across this great document (developed by National Institute on Aging and the National Library of Medicine) full of tips on making your website senior friendly

Other Resources

• www.holidaytouch.com
• National Institute of Aging

The default LDAP query when you first run through the Import Organization wizard should filter these computers objects out. The query is:
(&(objectCategory=person)(objectClass=user))

In Active Directory, computers do not generally have an objectCategory equal to Person. Computers usually have the objectCategory ‘Computer’.

If by chance your computers are not being excluded by this filter, you could exclude all objects without an email address. This of course assumes that all users you want to import have an email address populated in Active Directory. To exclude objects without email addresses, the filter becomes:

(&(objectCategory=person)(mail=*)(objectClass=user))

Another useful addition to the query is to exclude users that have been disabled in Active Directory. You usually disable an account when a person leaves the organization, but you still need their user profile in Active Directory for whatever reason. This query is slightly less obvious:

(&(objectCategory=person)(mail=*)(objectClass=user)(!(userAccountControl:1.2.840.113556.1.4.803:=2)))

For information on what the numbers mean in the query above, see How to query Active Directory using a bitwise Filter

Another question I’m often asked is how to exclude specific OUs from a query. Unfortunately LDAP does not support this concept and the only way to do this is to run multiple queries on different root level DNs. This means running through the Import Organization wizard multiple times with a different Root Distinguished Name each time, and the ‘Merge’ options set to ‘Keep users that are no longer in the directory’ and ‘Keep existing user details’.

If you have other helpful LDAP queries, please leave a comment below.

Suite 14242, 2nd Floor
145-157 St John Street
London, EC1V 4PY
Find us on Google Maps

Phone: +44 (0) 208 133 9004
Fax: +44 (0) 1509 380 036
Email: Email: europesales@webspy.com

I wanted to share this clip with the rest of you to brighten up your day AND provide you with a short an unobtrusive break, enabling your mind to rest, leading to a higher total net concentration, and as a result, increased productivity.

After all, Dr Brent Coker, from the Department of Management and Marketing, says that workers who engage in ‘Workplace Internet Leisure Browsing’ (WILB) are more productive than those who don’t.

“People who do surf the Internet for fun at work – within a reasonable limit of less than 20% of their total time in the office – are more productive by about 9% than those who don’t,” he says.

“Firms spend millions on software to block their employees from watching videos on YouTube, using social networking sites like Facebook or shopping online under the pretense that it costs millions in lost productivity, however that’s not always the case.”

According to the study of 300 workers, 70% of people who use the Internet at work engage in WILB. Among the most popular WILB activities are searching for information about products, reading online news sites. Playing online games was the fifth most popular, while watching YouTube movies was seventh.

The attraction of WILB, according to Dr Coker, can be attributed to people’s imperfect concentration. “People need to zone out for a bit to get back their concentration. Think back to when you were in class listening to a lecture – after about 20 minutes your concentration probably went right down, yet after a break your concentration was restored.

Invitation to participate in Dr Brent Coker’s continued web usage study

An issue was introduced in build 2.2.0.42 when a change was made to the underlying data type of the Url Category field in the Microsoft TMG Web Schema. Unfortunately this change resulted in nulls being imported into the Url Category summary when importing from TMG’s SQL log files. Fortunately, importing W3C text logs were unaffected by this change.

This issue has now been fixed and released as an auto-update. To update your software simply select Tools | Check for updates.

If you have existing storages that have not been importing the URL Category, go to the storages screen and click ‘Reload All’. This will re-import your log files and populate the Url Category summary.

For all the customers affected by this issue, thank you for your patience and understanding.

Here’s the full list of changes:

• New: Juniper SA Series. Vantage can import and report on web traffic and VPN connections.
• New: Microsoft Forefront Protection for Exchange 2010 format.
• New: Avencis SSOx.
• Improved: IronPort WSA: Department and Message fields were sometimes returned as null. Fixed.
• Improved: Microsoft FTMG: Removed usage of deprecated “FilterInfo” field from W3C Web format.
• Improved: Microsoft IAS Radius: Added support for Source/Destination IP and port (field code 5000).

To update your Vantage application simply select Tools | Check for updates.

To update the Vantage Web Module, right-click the WebSpy icon in the Web Module server’s system tray, and select Check for updates. If you have any issues with the Web Module update process, please see my previous blog regarding Web Module Errors and Workarounds.

This release will be welcomed with open arms by many customers for the following reasons:

• General usability improvements in the Web Module
  Multi-select / delete options, Ajax progress indicators to avoid page refreshes, export from Dynamics Report tab and more (see below)
• Fixes to the Microsoft Forefront TMG loader
  See my other post: Microsoft Forefront TMG logs size fields the wrong way around. Also fixed ‘value cannot be null’ error when importing SQL logs.
• Fixes to storage corruption issues
  This build should prevent ‘Normalization Index’ storage corruption issues from occurring. This often occurred after importing data, editing some log inputs and reimporting.
• New loaders and more fixes
  See below for the full list

To update your Vantage application, simply choose Tools | Check for updates. To update the Web Module, right-click the WebSpy icon in your system tray and select ‘Check for updates’. If you have any issues updating the Web Module, please see my previous post Web Module Update Errors and Workarounds.

Web Module Changes:

• New: Task progress is now updated without refreshing the page
• New: Added multi-select / delete functionality to Reports, Analyses and Storages tables.
• New: Added export functionality to Dynamic Reports view.
• New: Added Performance section on the Options tab to enabling multi-processing (improves Analysis speed)
• Fix: Dynamic Reports view now supports Trend reports.
• Fix: Organization selector on Dynamic Reports view now always reflects updated data under IE6/7/8.Fix: Fixed javascript errors in IE when expanding the organization filter.
• Fix: Report template names are no longer truncated on the Dynamic Reports view.
• Fix: Fixed errors that may occur when collating reports on the Dynamic Reports page.
• Fix: Authentication errors are now logged with stack trace.

Vantage Changes

• Fixed: ‘Normalization index’ storage corruption problems.
• Fix: Report collation: Added support for collation of Min/Max aggregates on DateTime columns (time of first hit etc). Also added support for arrayed fields (for example, category fields with a comma separated list of categories)
• Fix: Import windows wizard now remembers settings for Import all or selected users
• Fix: Organization: Filtered LDIFs may now be imported when references to some users are missing (for example, if a user’s manager does not exist in the LDIF)
• Fix: Improved connection and error handling between Vantage and the Web Module.

Loader Changes

• New: BlueReef Sonar Total Management Module
• New: Microsoft Sharepoint 2007
• New: SmoothWall Guardian 7.0 format
• New: Sun One Proxy (Supported under Sun One Webserver)
• Fixed: Astaro: Improved format detection
• Fixed: Cisco: Strings in the IP fields of 113019 lines are now imported
• Fixed: IronPort WSA: Improved log format detection
• Fixed: Microsoft Exchange 2007: No longer raises issues regarding total-bytes or internal-message-id fields
• Fixed: Micorosft FTMG (Web) SQL: No longer encounters value could not be null errors
• Fixed: Microsoft FTMG: Added option to reverse bytes received/sent fields. See Microsoft Forefront TMG logs size fields the wrong way around
• Fixed: Microsoft IIS W3C: Now imports cs-method and connection ID
• Fixed: Sophos Web Appliance: Switched the outgoing and ingoing sizes so that they are now the correct way around
• Fixed: Fixed import new hits issue associated with W3C formats. You must reload your logs before this change will take affect. Formats affected include: BlueCoat, Clearswift, Microsoft Exchange 2007, Microsoft FTMG, Microsoft Windows Media Services, WebSpy Live Tracking Log

Enjoy!

Microsoft TMG Log showing Bytes Sent consistently larger than Bytes Received

This issue has been confirmed by the Microsoft Forefront TMG team, and unfortunately there is no ETA for a fix.

We obviously don’t want our reports showing incorrect usage figures, so we’ve fixed our TMG loader so that it imports the ‘bytesrecvd’ field into the Bytes Sent aggregate, and the ‘bytessent’ field into the Byte Received aggregate.

But what if Microsoft release a fix? What we’ve done is implemented a loader property to allow you to turn off this behavior. This will allow you to import your old logs with the fields reversed, and your new logs with the fields the right way around.

To access the loader property:

• On the import wizard, select the Microsoft FTMG format and click the Properties button on the toolbar
• Select Microsoft FTMG from the drop down list
• Notice the option to ‘Reverse Bytes Sent and Received to compensate for bug in TMG’s logging’. Leave this checked until Microsoft issue a fix.

Microsoft Forefront TMG Loader Option to Reverse Bytes Sent and Received

This fix is available in Vantage build 2.2.0.48 (and above) which has been released as an auto update. So simply select Tools | Check for updates to ensure you have this fix.

So let’s use WebSpy Vantage to drill into that Anonymous user and find out what is going on.

One way to do this is to run an Ad-hoc analysis on the Summaries screen and drilldown into the Anonymous user to view all the information about that user. However, TMG and ISA tend to log a lot of information that may not be relevant to this particular investigation, so I’ve created some report templates (one for ISA and one for TMG) and a set of Aliases that pull out some relevant information.

Download our Anonymous Traffic Investigation Report

If you’re running WebSpy Vantage download the Anonymous Traffic Report Templates & Aliases

Then open the .Templates file on the Reports tab, and the .Aliases file on the Aliases tab. Once you have both files opened, go to the Reports tab and click either the ‘Anonymous Traffic Investigation (ISA)’ or the ‘Anonymous Traffic Investigation (TMG)’ report. Then click the ‘Generate report’ link and run the report template on your ISA or TMG storage.

The report gives you the ability to drill into the Allowed, Denied and Failed traffic to see a list of the unauthenticated IPs, Sites, Rules responsible for blocking or allowing the traffic, unauthenticated Applications and Result Codes.

Main causes of anonymous traffic

What you will probably find is that most of the Anonymous traffic is being denied by your TMG or ISA firewall. When a client first requests a web page, the proxy will challenge the client for authentication. These events are often logged with the result code 12209 meaning ‘authorization is required to fulfill the request’. These requests are therefore denied by the proxy until the client’s credentials are authenticated.

Have a look at the amount of traffic being denied and then checkout the Result Codes associated with the denied traffic. Chances are you’ll see ‘proxy authentication required’ appear predominantly.

If you also look at the Applications section you may also find that Windows Updates are sailing through your TMG or ISA firewall unauthenticated.

Filter out unauthenticated traffic from Reports

The most logical next step is to filter out the information you do not want in your reports. You’ll probably still want to include Windows Update traffic in your reports, but you’re probably not so interested in the ‘proxy authentication required’ information. So let’s filter that out.

To do this:

1. Go to the Reports tab and select the report you want to filter (such as your Organization report)
2. Click ‘Edit Template’, then click ‘Template Properties’.
3. In the filter section at the bottom of the dialog, click Add | Field value filter.
4. Select the ‘Result Code’ summary and select the Status Code Names (ISA-FTMG) alias.
5. On the toolbar, search for Authorization, and check the following two items:
   • The server requires authorization to fulfill the request. Access to the Web Proxy filter is denied.
   • The server requires authorization to fulfill the request. Access to the Web server is denied. Contact the server administrator.
6. Ensure the ‘Exclude’ radio button is selected and click OK.

If you decide that you don’t care about seeing ANY unauthenticated traffic in your reports, you can always simply filter out the Anonymous user from your reports.

To do this:

1. Go to the Reports tab and select the report you want to filter (such as your Organization report)
2. Click ‘Edit Template’, then click ‘Template Properties’.
3. In the filter section at the bottom of the dialog, click Add | Field value filter.
4. Select the ‘Username’ summary.
5. On the toolbar, click Add and type ‘anonymous’. Click OK.
6. Ensure the Exclude radio button is selected and click OK.

Hopefully this article improves your understanding of the ‘anonymous’ user, and gives you some actions to take for your specific reporting situation.

If you have any questions, please leave a comment below.

• How to import your log files and explore the information recorded by IronPort using the Summaries screen
• How to open the customized IronPort Report Templates and Aliases
• How to generate reports
• How to import your organizational structure and report on departments
• How to setup the Web Module and publish reports

PART 1: Importing log files & exploring your IronPort summaries

Once you have exported your IronPort access logs (see http://www.webspy.com.au/vendors/ironport/howto.aspx#ftp), this video takes you through importing your logs into WebSpy Vantage and analyzing data on the Summaries screen.

PART 2: Opening the customized IronPort Templates & Aliases, and running reports

This video takes you through opening the IronPort-specific report templates and aliases and generating a report that provides an overview of your organization’s Internet usage.

PART 3: Importing your Organization structure & generating department reports

This video shows you how to import your organizational structure into WebSpy Vantage from a directory server (such as Active Directory) using LDAP, and then generating a report that contains information on your newly imported departments.

PART 4: Using the Web Module.

This video takes you through configuring and using the WebSpy Vantage Web Module. Specifically, it takes you through the following tasks:

• Configuring the Web Module for Windows Authentication
• Adding a Web Module to Vantage
• Publishing reports to the Web Module
• Adding permissions for a user
• Synchronizing the Web Module
• Using the Dynamic Reports tab

PART 5: A quick word about tasks & conclusion

This video summarizes the actions taken in the previous four videos and also briefly discusses how to automate the reporting processing using scheduled tasks.

I hope this helps! Let me know if you have any questions by leaving a comment below.

The log databases are stored in an SQL Express instance named MSFW. By default these databases cannot be accessed by a remote computer. I’d first like to say that we recommend changing TMG’s logging to W3C text files, as these logs are about 5-6 times faster to import, and you don’t need to worry about the steps below.

But if you need to stick with the SQL Express logging, here are the basic steps to enable access to the logs from a remote computer:

Enable TCP access to the MSFW instance

To do this:

1. Log into your Forefront TMG server using administrator credentials.
2. Select Start | All Programs | Microsoft SQL Server 2008 | Configuration Tools | SQL Server Configuration Manager.
3. Expand SQL Server Network Configuration and select Protocols for MSFW
4. Right-click TCP/IP and select Enable
5. Click OK on the Warning dialog informing you that “changes will not take effect until the service is stopped and restarted.”

Enabling TCP/IP on the MSFW instance

Set the listening Port on the MSFW instance

Once TCP/IP is enabled on the MSFW instance, you need to set it to listen on port 1433

1. Select Protocols for MSFW under SQL Server Network Configuration
2. Right-click TCP/IP and select Properties.
3. Click the IP Addresses tab and scroll to the IPAll section at the bottom of the list.
4. Change the TCP Port to 1433 and ensure nothing is entered in TCP Dynamic Ports (Delete the ‘0′ value  if present). Click OK and click OK on the Warning dialog.

Setting the Port on the MSFW instance

Change the listening port on the ISARS instance

The ISARS SQL instance also listens on port 1433 and this can cause connection issues. Change this instance to use port 1434:

1. Still in SQL Server Configuration Manager, select Protocols for ISARS under SQL Server Network Configuration
2. Right-click TCP/IP and select Properties.
3. Click the IP Addresses tab and scroll to the IPAll section at the bottom of the list.
4. Change the TCP Port to 1434 and ensure nothing is entered in TCP Dynamic Ports. Click OK and click OK on the Warning dialog.

Changing the port on the ISARS instance

Restart the Services

For the above changes to take effect, you need to restart the SQL Server (ISARS) and then the SQL Server (MSFW) services in that order.

1. Go to Start | Administrative Tools | Services
2. Right-click the SQL Server (ISARS) service and select Restart.
3. Right-click the SQL Server (MSFW) service and select Restart.

Test the connection from the WebSpy machine

You should now be able to connect to the MSFW databases from a remote computer. To test the connection, we recommend that you install SQL Management Studio on the machine running WebSpy and try to connect to <TMGservername>\MSFW, 1433 (replace <TMGservername> with your actual server name or IP address). For example TMGServer\MSFW, 1433 or 192.168.0.10\MSFW, 1433.

As long as you are logged into Windows with a user account that is a local administrator on the TMG server, you should be able to connect without issue.

Importing the TMG Log files into WebSpy Vantage

Once you have established a connection, you can import your logs using WebSpy Vantage like so:

Create a new Storage

Select Database Connection

Select the Microsoft FTMG Loader

Click Add

Enter TMGServer\MSFW and port 1433

Successfully Imported WebProxy Logs

The screenshots above also illustrate using a database mask of *WEB* to only import the WebProxy logs. If you only want to import the Firewall logs, set the database mask to *FWS*. If you want to import both the WebProxy and Firewall logs, leave the database and table masks set to *.

Now that you have your log files imported, you can run a quick ad-hoc analysis on the Summaries screen or generate any of Vantage’s default web of firewall reports. M

Make sure you also download our Forefront TMG specific Aliases and report template. For more information, see our Forefront TMG How To page.

If you have any questions or encounter any hurdles, please leave a comment below.

“I would like to know how much of my bandwidth is being eaten by each protocol. I will then use this information to determine if circuit may need to be increased due to increased traffic”.

This customer was collecting syslog messages from a Cisco Firewall, then using WebSpy Vantage to generate reports. In theory, this sounds like a fair plan. Unfortunately, the Cisco Firewall logs many different types of messages. Some to do with denied packets, some to do with authentication, some for vpn and so on. The information contained within each message changes. Some events include the size information that is required for any type of bandwidth assessment and some don’t. Correlating the required events to get any sort of accurate ‘bandwidth’ representation is a bit of a nightmare.

Fortunately, there’s a simpler method. If you search the Cisco website or the Internet for bandwidth utilization reporting, you’ll no doubt be pointed in the direction of NetFlow.

NetFlow is a network protocol developed by Cisco Systems to run on Cisco IOS-enabled equipment for collecting IP traffic information [Source Wikipedia http://en.wikipedia.org/wiki/Netflow]

There are a couple of commands to enter on your router to turn NetFlow on, and then you just need a NetFlow collector to receive the Netflow information and generate reports.

Fortunately WebSpy has developed a little tool called FlowMonitor that collects the Netflow information and writes a log file that can then be imported into WebSpy Vantage and reported on.

The FlowMonitor Management Console

Once your FlowMonitor logs are imported into WebSpy Vantage, you can run the default FlowMonitor report to see the size of traffic flowing between IP addresses, subnets, router interfaces or protocols. Alternatively you can create your own custom reports to see exactly what you want to see.

NetFlow doesn’t record usernames or URLs so it’s not great for reporting on the web sites your users are visiting, but it is great for network administration and trouble shooting. Identify chatty IP addresses, protocols that are chewing too much bandwidth, the times throughout the day when incoming or outgoing links become heavily utilized and so on.

For information on how to configure your router and deploy FlowMonitor, see the FlowMonitor Installation and User Guide. You can also download a free trial here.

FlowMonitor is a handy little tool. Ask your friendly WebSpy account manager about it today!

Vantage uses multiple threads to perform certain tasks. As general rule, the more threads being used simultaneously, the higher the CPU utilization. There are a few situations where Vantage uses multiple threads simultaneously:

CPU usage when importing log files

When importing more than one log file, each log will be imported with a separate thread. As CPU usage increases when more threads are used, importing a single log file won’t push your CPU, but importing a folder full of logs will.

CPU usage when generating reports

CPU performance can also be affected by the structure the report you are running. Report templates have what we call ‘Nodes’ in them. You can go into a report template, right-click | add node. Think of each node as an SQL query. When generating a report, each node gets processed in a separate thread, and if the nodes are ‘at the same level’ they get processed at the same time.  Here’s a screenshot showing what I mean by nodes at the same level.

A report template with two 'levels' of nodes

The three ‘red’ nodes will be processed at the same time, and then the three ‘green’ nodes will be processed at the same time. The green nodes won’t be processed until the red nodes have been processed. The more nodes being processed at the same time increases the number of simultaneous threads and the amount of CPU being used.

CPU usage when filtering reports

CPU usage is also affected by the number of records being processed from your storage. If you are running a report on your entire storage with no filters, then Vantage will be pushing all records in your storage through the reporting engine. If you run the same report but with a filter for a specific user, then Vantage will seek through the records in the storage until it finds a record for that user, then push that record through the reporting engine. This results in a ‘trickle’ of records being pushed through the reporting engine so it doesn’t get a chance to really push your CPUs.

A filter that excludes a lot of information that exists in your storage is the most common reason for low CPU utilization while running a report.

In Short

The number of logs, report template structure and filters can all have an effect on the way Vantage utilizes your CPUs.

We also have some exciting ideas on our roadmap to ensure Vantage utilizes as many CPUs as you can throw at it. Until then, I hope the above information helps you understand when and why your CPU usage will and won’t be pushed.

This is a great update for Vantage Ultimate users as we’ve introduced a new feature/tab into the Web Module called ‘Dynamic Reports’.

If you’re publishing the same report to the Web Module each day, you can use the Dynamic Reports tab to select a date range and a department (or whatever organizational groups you have defined) and the Web Module will collate all the daily reports that match that filter into one report. This allows you to report on entire week, month or year by simply ‘reporting on reports’, rather than reporting months of raw storage data.

Here’s the full list of changes since the last auto update (2.2.0.32 on the 14th April 2010).

Application Changes

• Added Dynamic Reports feature to the Web Module.
• Rewrote the Web Module transfer protocol. New protocol adds version checking, connection checking, and integrity checking for high latency environments.
• Purge data from storage task no longer prevents importing new hits when all data is removed from an input within a storage.
• IPv6 addresses now show IPv4-mapped addresses as plain IPv4 addresses in summaries.
• IPv6 and IPv4 addresses are now freely interchangable in filter expressions.
• Fixed IPv6 drilldowns on the Summaries screen
• SQL inputs can now be resumed from the previous position. Previously any input that was partially imported would be skipped when importing new hits.
• Template-based analysis has been fixed, no longer results in blank/non-existent analysis.
• Added new string manipulation functions to expression language; Contains, StartsWith, EndsWith, IndexOf.

Loader Changes

• Astaro: Now checks that the ID field is present in a line before attempting to read it.
• Barracuda Web Filter: Added this format to replace Spy Filter.
• BlueCoat Proxy SG W3C: Added support for gmttime, timestamp, x-bluecoat-surfcontrol-is-denied and x-bluecoat-transaction-id.
• ClearSwift: Added a new loader group for ClearSwift that includes the MimeSweeper loaders
• ClearSwift SECURE Web Gatway: Now supported with the Web Appliance loader
• Clearswift Web Appliance: User summary displays Source IP if Username is blank.
• IronPort WSA: Fixed memory usage issues.
• Microsoft FTMG: Added category name lookup to SQL loader.
• Microsoft FTMG: No longer fails to import lines where the rule field contains square brackets.
• Microsoft FTMG: URL Category field is now a string instead of an integer. Added URL Categorization Reason field.
• Microsoft FTMG: Fixed memory usage issues.
• Microsoft IIS W3C: No longer hangs or crashes when loading a file that isn’t IIS W3C.
• NetAsq: Added support for srcname field. The Username summary is populated with user first, and then srcname if user is blank. The User summary is also now populated with Source IPs if the Username summary is blank.

To update WebSpy Vantage, simple select Tools | Check for updates.

To update the Web Module, login to the Web Module server, right-click the WebSpy system tray icon, and select Check for updates.

As always, please contact us if you have any issues or questions.

One of Forefront TMG’s major strengths is obviously its URL categorization and filtering abilities. Since TMG now takes care of the threat management aspects, clients converting from other solutions, such as ISA Server, no longer need a third party filtering solution and will most likely save a considerable amount of money.

However, the reporting functionality included in Forefront TMG are not much different from ISA Server 2006, i.e. very little flexibility or customization for those with reporting requirements beyond general overviews cluttered with irrelevant information.

We’ve blogged a lot about TMG reporting in the past and have now uploaded new and dedicated WebSpy Vantage and Microsoft Forefront TMG pages outlining:

• 10 Reasons to Use WebSpy Vantage to Report on Forefront TMG
• How to:
• Set-up TMG Logging for WebSpy
• Import TMG Logs into WebSpy Vantage
• Forefront TMG Report Templates and Aliases (created to make your life a lot easier)
• Run Reports
• Analyze and Drilldown into Data

Have a look at WebSpy Vantage and Microsoft Forefront TMG.

Hopefully it can assist you in your quest for sophisticated Forefront TMG reporting.

Check your version in Help | About. If you are running 2.2.0.27 or above, then you already have this update. If not, make sure you update to your software by selecting Tools | Check for updates.

Here’s a quick video outlining some of the differences between TMGs Reporting, and what can be achieved using WebSpy Vantage. The video does not illustrate all the limitations outlined below, so please read on.

Whats is in the Forefront TMG report?

The default TMG report contains the following sections

• Summary
• Web Usage
• Application Usage
• Traffic and Utilization
• Security
• Malware Protection
• URL Filtering
• Network Inspection System

Each section contains overviews such as ‘Top users’ and ‘Top Sites’.

If your reporting requirements can be satisfied with these overviews – that’s great! Unfortunately, when you start thinking about what system administrators and other people in your organization actually need to make informed decisions, this report is quite limiting.

The 8 Limitations of Microsoft Forefront TMG’s Reporting

Here is what I consider to be the 8 main limitations of Microsoft Forefront TMG’s reporting functionality.

1. No Drilldowns

Want to see the sites that the top 5 users accessed? Want to see the users that downloaded the most traffic from youtube? These are fairly standard reporting requirements that simply cannot be achieved using the inbuilt TMG reporting.

WebSpy Vantage lets you either interactively drilldown into a user or site, or produce a regular report that includes further details about what your top users have actually been up to.

2. No Filtering

When you generate a report in TMG, you can only filter the report by a date range. There is no way to filter out anonymous (unauthenticated) traffic or exclude traffic coming from advertising servers (such as doubleclick and 2mdn.net) that tend to dominate most of the top 10 sites.

This can easily be achieved using WebSpy’s software. Check out my video on how to remove clutter from your web reports.

3. No Customization

Customization of each overview in the TMG report is limited to the number of items to show (e.g. top 5 or top 50 users), and the sort order (Incoming Bytes, Outgoing Bytes, Requests and Total Bytes).

What about the time a user spent browsing the web, or the number of users that visited a specific site? There is no way to add custom columns such as total browsing time, average session time, or number of users/sites/IPs to the report tables.

Or say you simply want to change your top users chart from a bar to pie to easily see the percentage used. Nope sorry!

If you do make one of the two available customizations in a TMG report, you then get the annoying Apply / Discard message to save changes to the configuration database.

All of these customizations can be achieved using WebSpy Vantage, and it doesn’t touch your TMG server to apply a change to a report.

4. Limited Report Distribution

When you generate a report, you get the option to email it to a specific email address. What if you would like to create a report for every department, and then email it to the managers of each department? Or better yet, host the report on a secure web server where department managers can log in and view their reports?

WebSpy Vantage Ultimate comes with a secure ‘Web Module’ specifically for this purpose and managers still receive a link to the report via email.

5. Cluttered ‘Top Sites’ List

The ‘Top sites’ list can become particularly cluttered due to the inclusion of sub-domains. I don’t want to mentally add up the size values from farm1.static.flickr.com, farm2.static.flickr.com, and farm3.static.flicr.com – I just want to know how much was downloaded from flickr.com.

This is compounded by the inability to exclude sites that are merely placing advertising banners on the actual sites users are visiting (as mentioned in the ‘No Filtering’ limitation above).

WebSpy Vantage breaks URLs down into separate components and lets you analyze each part separately. Look at the Site Domains summary to remove sub-domains and see only flickr.com. Or perhaps you want to see the keywords a user entered into search engines like Google? Or perhaps the top pages accessed within a website? No problem. Just include the Site Keywords or Site Resource summaries in your Vantage reports.

6. No Grouping or Aliasing

There is no way to group users into departments or locations, or IP addresses into subnets, or extensions such as .html, .pdf or .exe into file types. The ability to group and represent raw log data in more meaningful ways, as offered by WebSpy Vantage, can increase the value of a report tremendously.

7. No Productivity Assessment

One of the major features introduced in TMG since ISA Server 2006 is the included URL categorization technology.

Although the TMG report gives you an overview of the categories that have been visited, the report does not use this information to display a productivity assessment for your users.

WebSpy Vantage not only provides this assessment, but also the ability to customize the categories that are deemed productive as this can vary wildly depending on the industry and organization.

8. Not browser independent

This is a minor limitation that can be a major annoyance. The report that TMG produces is a HTML report that only displays correctly in Internet Explorer. As Forefront TMG is a Microsoft product, this is not exactly surprising, but still very annoying if IE is not your default browser.

How to get awesome reports from Forefront TMG

If you have had personal experience with any of the above limitations, you’ve probably been hunting for an alternative solution. I strongly recommend checking out the WebSpy Vantage range of products, and if you would like secure report distribution via the ‘Web Module’, Vantage Ultimate is what you are after.

If you agree or disagree with anything in this article, I encourage you to leave your thoughts in the comments.

Cheers!

Scott

We take great pride in this versatility and the fact that our software is virtually vendor neutral, or universal. At the time of writing our software support 128 different vendors, and more than 250 log file formats. But what does all this actually mean? What’s all this log file format gibberish and why is better to use a universal log file analyzer than a reporting solution that can only analyze a limited set of log files?

Let’s break it down…

What is a Log File?

A log file is a set of data that is automatically created and maintained by a security or network device of activity performed by it.

Web proxy servers maintain log files listing details on every request, from outgoing traffic, made to the proxy server – who is accessing external sites, what sites are being accessed, when the sites were accessed, what page or search phrase referred the user to the sites, and the type and size of data downloaded from the sites. Email servers store log files containing data about the sender, the receiver, timing of delivery or receipt, subject line, and size of attachment. Firewalls, and other security devices, normally contain data about network activity and the external and internal traffic that has been blocked or filtered.

Log files contains bundles of information and are usually not very structured or easy to decipher. Here’s what a  log file can look like:

Can you see the need for a log file analyzer?

Who are the Network and Security Device Vendors?

There’s a bunch of them to say the least. A recent WebSpy customer survey showed that Microsoft, Novell, Squid, IronPort, Blue Coat and CheckPoint were the top vendors whose products our clients wanted to analyze.

Checkout the whole list (128) of vendors we support at http://www.webspy.com.au/resources/logformats.aspx

What is a Log File Format?

When we state we can analyze more than 250 different log file formats we take into account the different log files formats produced depending on the vendors’ product, product version and log type

For example: Microsoft develops products such as Exchange, IIS, Proxy Server and ISA Server. ISA Server comes in different versions (2000, 2004 and 2006). Each version can log and store different types of log files (file, MSDE Database and SQL Database). So, ISA Server MSDE Database 2004 is one log file format we support, ISA Server file 2000 is another.

Benefits of a Universal Log Analyzer

The 250 something log file formats WebSpy analyze and report on are simply the most common ones. It is very rare that we come across a client who need to analyze a log file that is not already on our list of supported log files. On the odd occasion this does occur, the client can simply request support for their specific log file format, our developers work their magic and wham – we support one more log file format.

Most competing log file analyzers are hard-coded to analyze a particular log file type. When this is the case their clients will need a different log analyzer to achieve each individual reporting requirement, increasing the time and costs involved to produce all the required reports.

WebSpy’s clients, on the other hand, reap the rewards of using one application to achieve all their reporting requirements, spending less on software maintenance, hardware and administration.

TMG or UAG? What is the difference?

TMG is an outgoing proxy that protects your internal users from malware, viruses and the like. TMG generates some great web proxy log files to import into WebSpy Vantage allowing you to monitor where your users are going on the Internet, how much they’re downloading etc.  TMG, unlike ISA, now has deep packet inspection for HTTPS traffic, plus a bunch of other new features.

UAG is an incoming proxy that provides employees, partners and vendors secure remote access to corporate resources such as Outlook Web Access (OWA) and Sharepoint (MOSS). It utilizes the TMG engine, but this is mainly just to protect the UAG server (more on this topic here http://technet.microsoft.com/en-us/library/ee522953.aspx).

TMG can also publish your OWA and MOSS sites, but this is no longer recommended by Microsoft. They recommend using a dedicated UAG server to perform this function.

Upgrading to TMG

If you’re thinking about migrating your ISA server (2004 or 2006) to TMG, you may like to check out this migration guidance video with Mohit Saxena (Senior Technical Lead) and Jim Harrison (Program Manager). http://edge.technet.com/Media/ISA-to-TMG-Migration-Guidance/

Microsoft Forefront TMG Migration Video

Reporting on TMG

If you’re using TMG at the moment, we invite you to analyze your web proxy and/or firewall logs using WebSpy Vantage and tell us what you think!  Download your copy of WebSpy Vantage here, and import your logs using the Microsoft FTMG format:

Microsoft Forefront Threat Management Gateway

TWiST features entrepreneur Jason Calacanis and a rotating group of guest experts who bring you a weekly take on the best, worst, most outrageous and interesting from the world of Web companies.

The entire episode is well worth watching. Jason chats to Jack about his background, WebSpy, internet monitoring, and gives his inventive start-up insights and opinions during the “Ask Jason”, “Shark Tank”, “Dead Pool” and “News with Lon” segments. Due the length of the episode (2h 40min) I’d thought I’ll provide you with a short summary and selected clips of WebSpy’s and Jack’s golden TWiST nuggets.

Selected Clips & Short Summary

Talking about Monitoring

Jason starts of chatting about his own experiences when it comes to Internet monitoring, or should I say the lack thereof. Jason’s acquaintance was running a start-up and one of the employees was downloading copyrighted materials to his work computer. Lo and behold, one day a letter from a large entertainment conglomerate arrives holding the start-up company responsible of stealing.

Jack also recollects an incident where a large number of government employees were let go because they had been doing inappropriate things on the internet during a six month period. Jack clearly states his, and WebSpy’s, view: No employee should be fired for misusing the internet. It is tragic when this happens, and it is not the reason to get rid of people, hiring and firing should be based on if you have a good or bad employee. It is the employer’s responsibility to ensure Acceptable Internet Usage Policies are put in place and adhered to. If clear rules are communicated and monitored no one should have to be sued or fired.

Jason enquires about the best way for an employer to implement a monitoring solution without seeming like a draconian maniac boss. Jack highlights that all the information (logs) of what internet and email are being used for are already there. WebSpy simply make sense out of the vast amount of raw data and turn it into clear, searchable, categorized reports. Jack says that using monitoring solutions makes you less of a draconian boss. You are opening up what is already there while taking control of the valuable information.

Although blocking is a way some companies are trying to control their employee’s internet usage both Jason and Jack agrees that blocking is ineffective, gets in the way, and it is not what the internet is about. The internet is there to be used and by monitoring employers help employees to use it productively.

WebSpy Soho

Jack officially announces the upcoming launch of WebSpy Soho – WebSpy’s brand new home and small office dashboard application. Soho is designed for small offices, home offices, families, shared student households and even a single computer home user. It will display download and upload activity and history for all computers on your network and alert you when the network load is high, and which computer is causing the overload. Soho will also alert you when an unknown machine is using your network, which is great if you use an open wireless connection.

I will blog MUCH more about Soho over the next few weeks. For now, though, please register your interest in the product at www.webspy.com/soho for the chance to get a free copy of the full version AND vote for you favorite Soho logo (before the 11th of December).

If you get a chance I do recommend you watch the full episode (below). You can also download the video and/or audio version free from iTunes.

Watch the full TWiST episode

Marketboomer provides supply chain management software to help businesses (such as InterContinental Hotel Group, Emirates Airline Catering, Hyatt and more) take cost out of their supply chain. The solution enables businesses to trade with each other more effectively using the Internet, allowing purchasers to buy from suppliers at the best possible price at a given point in time, and by improving processes. Marketboomer typically reduces clients’ procurements costs by between 8 and 20%, and additionally creates process savings and efficiencies of 20% or more within the procurement function.

The completed acquisition is great news for all stakeholders and will provide significant synergy benefits and strategic expansion opportunities. Marketboomer’s software is highly complementary to WebSpy’s, in the areas of cost management and productivity improvement, and its customer base represents a strategic and natural market for WebSpy’s software. Both WebSpy and Marketboomer products are B2B, based around Internet and large data manipulation, and can be applied to any industry sector and any size business.

The acquisition will also enable WebSpy to gain additional operational and development capability in Europe, the Middle East, China and South East Asia. The Marketboomer Group has established marketing presence in Australia, Ireland, China, Indonesia, Thailand, Saudi Arabia, the UAE and Singapore. These additional marketing centers will strengthen WebSpy’s own marketing resources. WebSpy on the other hand has been operating in the United States for nearly 10 years and will be able to assist Marketboomer’s entry into the region.

ASX Announcement – Marketboomer Acquisition Completed
ASX Announcement – Details of Proposed Acquisition

In the mean time, I thought I’d share a way to re-brand the main elements in the Web Module by editing a few files and replacing a few images.

The only issue with this technique is that any future auto-updates for the Web Module will overwrite your edited files, so you just need to keep a copy of your customized files, so that you can restore them again after the auto-update.

Before you begin

In order to edit anything, you first need to know where your Web Module is located on your web server’s hard drive. This can be found by opening IIS Manager (Start | Control Panel | Administrative Tools | Internet Information Services (IIS) Manager) expanding the left hand server/site tree to find your Web Module.

• In IIS7, select the Web Module and click Basic Settings… in the right hand ‘Actions’ panel. The location is specified in ‘Physical Path’.
  

  Finding the Web Module's physical path in IIS7

• In IIS6, right-click the Web Module and select Properties… then go to the Home Directory tab. The location is specified in ‘Local Path’.

Windows may also prevent you from editing these files directly due to permissions issues. I’ve found a good technique is to copy the files you want to edit to your desktop, edit them, and then copy them back into the Web Module’s physical path. Windows will then prompt you to elevate to administrator and the copy/replace will succeed.

Ready To Go…

There are a few places where the WebSpy logo and WebSpy Text is presented.

• The login page
• The header bar
• The welcome Page
• Report cover pages

The Login page

The logo displayed on the login page can be found at /images/logo.png. Replace this image with your own logo. Then open Default.aspx in the Web Module’s root folder in a text editor such as notepad, and replace the following line

<img runat=”server” alt=”WebSpy” src=”~/Images/Get.ashx?image=Logo” />

with

<img runat=”server” alt=”WebSpy” src=”~/Images/logo.png” />

Before

Web Module's Login Page Before logo.png Change

After

Web Module Login Page After logo.png change

The header bar

The header bar utilizes the image located a /Images/bauble.png. Replace this image with your own custom image.

Then open Navigation.Master in the Web Module’s root folder in a text editor such as notepad, and replace the following line

Also look for the text:

Before

Web Module's Header Bar Before Bauble.png and Text Changes

After

Web Module's Header Bar After Bauble.png and Text Change

The Welcome Page

When you first login to the Web Module, you are presented with a Welcome Page. The first line on this page reads “Welcome to the WebSpy Vantage Web Module. You can change this by editing the first line in the Welcome.aspx file located in the Web Module’s root folder. Edit the section in bold below:
<%@ Page Language=”C#” MasterPageFile=”~/Navigation.Master” AutoEventWireup=”true” CodeBehind=”Welcome.aspx.cs” Inherits=”WebSpy.Vantage.WebModule.Welcome” Title=”Insert Custom Text Here” %>

Before

Web Module's Welcome Page Before Text Change

After

Web Module's Welcome Page After Text Change

The Report Cover Pages

The Image used on the cover page of reports is much easier to change.

1. Login to the Web Module as Administrator
2. Go to the Options Tab
3. Click ‘Report Logo’ under Web Module Options
4. Click Choose File, and select the image or logo you would like displayed on your report cover page
5. Click Upload

Before

Web Module's Report Cover Page Before Report Logo Change

After

Web Module's Report Cover Page After Report Logo Change

Summary

The changes above cover a majority of the areas your users will come into contact with in the Web Module. There may be a few more instances of the word “WebSpy” but for the most part, it should just be a matter of opening the relevant .aspx file and editing the html.

As I mentioned, if you auto-update the Web Module (via the system tray icon on the Web Module server), your edited files will be overwritten. I recommend keeping a copy of your edited files in a safe place outside the Web Module’s physical folder, so that you can copy them back in after the update. If the only changes you make are the ones above, then you’ll need to keep a copy of:

• /Navigation.Master
• /Default.aspx
• /Welcome.aspx
• /Images/logo.png
• /Images/bauble.png

We will also be adding the functionality to make these changes ‘properly’ in a future build, so follow us on Twitter, or subscribe to our RSS feed for updates!

Cheers!

Unfortunately, the WebSpy Twitter account fell victim to a phishing scam, and as a result sent phishing spam to all our Twitter followers. We are embarrassed by the incident and we apologize to all of our followers, especially the ones that clicked the link in the DM and were caught by the phishing scam themselves.

Here’s a rundown of the event in the hope that it will help others know what to look out for.

What Happened?

The phishing scam works like this:

1. You receive a strange yet intriguing Direct Message from someone you follow and likely trust. This is the key element to the scams success.
2. The DM contains a link using a shortened URL such as dwarfurl.com/blah. In our case, most of them were using dwarfurl.com, wapurl.co.uk, and 3.ly
3. You click the link and get taken to what appears to be the Twitter login page. But if you look at the URL it is actually something like blogs.videos.dsfasdc.com or videos.twitter.dsfasdc.com. Checking the URL is the key to making sure the scam doesn’t get you too!
4. You enter your Twitter login details. Reports of what happens after this login page vary. You may see the Twitter fail whale, or a blank page, or a random blog.
5. Now that the phishing site has your login details, the same Direct Messages is sent to all your Twitter contacts.
6. You eventually discover what happened. You feel like a violated idiot and start scrambling to fix everything.

What to do if it happens to you

If the above sounds familiar, you need to login to Twitter right now and change your password to make sure the phishing site can no longer access your account. You also need to go to the Connections tab and disable any third party applications that look suspicious. You’ll then need to update the credentials in all the twitter clients, website/blog plug-ins, and anything else that may be using your old Twitter credentials.

Fortunately, we were still able to login to our Twitter account and change our password and disable third party connections. Thankfully there were not any new suspicious connections that we needed to worry about.

Lessons Learned

Now that we’ve fixed everything and regained control of our Twitter account, it’s good to sit back and reflect on what just happened and how to avoid it in the future.

You’ve probably heard all of this before. We had too. But it takes an incident like this to really think about and address any shortfalls in your own organization. Some of our followers were also caught out by the scam and these are people that are in the tech industry and generally know about these sorts of scams. We were definitely surprised that we fell for it! So take a moment of your time to imagine your own Twitter account was compromised in the same way, then imagine all the possible ways it could have happened. Now go and take every precaution to ensure it doesn’t happen.

Having now been through it, here are some tips to help you avoid the same fate in the future.

1. Just because a Direct Message comes from someone you trust, does not mean it is trustworthy. Always use caution!
2. Educate your employees – especially those that know your company’s Twitter credentials. The main goal you want to achieve here is getting your employees into the habit of glancing at the URL in the address bar of their browser before entering ANY login details. We used our own log analysis software (Vantage) to find out who ended up on the websites in question, and then spoke to them directly to ensure they understood what to look out for.
3. Use a Twitter application that can display the actual URL behind a shortened URL before clicking on the link. For TweetDeck users, go to Settings | General, and check ‘Show preview information for short URLs’. Please note, however that this function only works for a few specific URL shortening services.
4. If you’re using the Twitter web page directly, use a browser and plug-in that can expand shortened URLs such as Mozilla Firefox with Long URL Please.
5. Use a browser with integrated anti-phishing security (such as Firefox or Google Chrome) and keep it up to date, or ensure you have good third party anti-phishing / anti-malware software installed.
6. As always, keep your security software and OS up to date.

Our friends at Sophos also have some good information about the scam that you may like to read: http://www.sophos.com/blogs/sophoslabs/?p=7366

Sorry!

An event like this makes you realize how important Twitter is to the overall public perception of a company. Our followers trust us to deliver relevant and useful content about our key areas of expertise – log file analysis and reporting. We spend a large amount of effort researching and writing content to ensure our tweets provide our followers with a good source of information. Having a breach like this certainly degrades this public perception that we work so hard at trying to maintain.

I would therefore like to thank all our followers who have kept with us and not clicked the ‘Unfollow’ button. Now that everything is under control again we will continue to bring you the best content we can provide about the log analysis and surrounding industries.

Once again, many many apologies to all of our followers, especially those that were affected.

I’ve been in touch with ComGuard’s Product Manager, Amir Mohsen Abedinifar, who was present at the event, and he said,

ComGuard's busy GITEX stand

“Overall GITEX proved to be a great networking and interactive marketing experience. The turnout this year was not as high as the previous year, most likely due to the economic downturn. However, we did notice that the majority of visitors we talked to had a sincere interest in the products we where showcasing.”

Mr Abedinifar added,“Security information vendors, such as WebSpy, are in high demand during these budget slashing times thanks to the software’s productivity improvement and cost savings abilities.

WebSpy’s solutions add tremendous business benefits to organizations that are struggling with unproductive Internet usage, unjustifiable bandwidth costs and other network management issues. We are excited to follow-up on our new contacts and encourage all GITEX visitors to take advantage of ComGuard’s and WebSpy’s GITEX offer.”

"WebSpy's software in high demand"

Here at the WebSpy lair we have indeed noticed a sharp increase in software downloads and inquires from the Middle East. This is in addition to the large amount of free 30 day software trial CDs that visitors picked up at ComGuard’s GITEX stand. It is apparent that investment in technology, that reduces costs and increase productivity, is more sought after than ever before.

About Gitex

GITEX is an annual trade show taking place in Dubai, United Arab Emirates. It is held at Dubai Trade Center exhibition halls with a typical turnout of 120,000 people visiting from all over the Persian Gulf each year. GITEX presents a great opportunity to source from over 3,000 suppliers, network with thousands of ICT professionals, see the latest launches and hear about the latest hot topics.

Related Links
GITEX
ComGuard

If you’re considering upgrading your ISA Server to TMG, this means that you can start your deployment using the Release Candidate, and simply switch it to a licensed version with no additional configuration changes once the full release is available. At least, that is what Vladimir Holostov (Lead Program Manager, Release Manager for Forefront TMG 2010) states on the Forefront TMG (ISA Server) Product Team Blog:

“The final product will be released later this year and you can expect it to behave exactly like the Release Candidate. You can install Forefront TMG 2010 RC today and upgrade to a licensed version once available without changing the configuration of your deployment.”

To offer some peace of mind for organizations considering the deployment, Vladimir also mentions that “Forefront TMG 2010 RC is deployed at three major Microsoft sites located around the world in Haifa, Bellevue and Redmond. More than 20,000 employees are already protected by TMG and these deployments have already accumulated more than 5,000 hours of runtime, performing extremely well under heavy load”.

No major features have been added to the Release Candidate since Beta 3, however there have been improvements geared around tightening up security, reliability and performance and telemetry. For more information about the release candidate, please visit the
Forefront TMG (ISA Server) Product Team Blog.

You can also download the release candidate here

I mentioned in my last blog posting that WebSpy has introduced support for reporting on Microsoft Forefront TMG log formats in the Vantage product range. To try it out, please make sure you have installed Vantage 2.2 (any flavour – Premium, Giga or Ultimate), and then select Tools | Check for updates to download build 2.2.0.10 or above. You can then import your TMG log files by selecting the Microsoft FTMG loader in the import wizard.

Importing Microsoft Forefront Threat Management Gateway Log Files

We’re very interested to hear your thoughts on the reporting functionality, so please go ahead and give it a go!

So, who is Marketboomer and why has WebSpy proposed to acquire them?

Marketboomer provides supply chain management software to help businesses (such as InterContinental Hotel Group, Emirates Airline Catering, Hyatt and more) take cost out of their supply chain. The solution enables businesses to trade with each other more effectively using the Internet, allowing purchasers to buy from suppliers at the best possible price at a given point in time, and by improving processes. Marketboomer typically reduces clients’ procurements costs by between 8 and 20%, and additionally creates process savings and efficiencies of 20% or more within the procurement function.

The acquisition is great news for all stakeholders and will provide significant synergy benefits and strategic expansion opportunities. Marketboomer’s software is highly complementary to WebSpy’s, in the areas of cost management and productivity improvement, and its customer base represents a strategic and natural market for WebSpy’s software. Both WebSpy and Marketboomer products are B2B, based around Internet and large data manipulation, and can be applied to any industry sector and any size business.

The acquisition will also enable WebSpy to gain additional operational and development capability in Europe, the Middle East, China and South East Asia. The Marketboomer Group has established marketing presence in Australia, Ireland, China, Indonesia, Thailand, Saudi Arabia, the UAE and Singapore. These additional marketing centers will strengthen WebSpy’s own marketing resources. WebSpy on the other hand has been operating in the United States for nearly 10 years and will be able to assist Marketboomer’s entry into the region.

The proposed acquisition needs to be approved by shareholders, more information at: www.webspy.com/newsroom/announcements/Marketboomer-Acquisition.pdf

You should be prompted to update your software on startup, but if you’ve turned off that feature, simply go to Tools | Check for Updates.

New Features

This new build sports the following new features:

• Support for Microsoft Forefront Threat Management Gateway (Beta)
  Microsoft Forefront Threat Management Gateway (FTMG) is still currently in Beta, and is due to be released around November 2009. For those that do not know, FTMG is the next version of Microsoft’s popular ISA Server. Information and downloads for FTMG can be found here http://www.microsoft.com/forefront/edgesecurity/isaserver/en/us/tmg-beta.aspx. We have added support for FTMG beta 2 and 3 for both the W3C text logs (recommended) and the internal SQL Server Express Database logs. If you are currently trialling FTMG, we are very interested to hear your feedback. Let us know how you go!
  

  Now Supported - Microsoft Forefront Threat Management Gateway

• Data purge
  You can now purge data from a storage, and schedule this purge to occur on a regular basis using Tasks. Purge options include data between a date range, data before a date, data after a date, data older than a date relative to now, and all data. This feature will let you easily maintain a single storage that only includes data for the last month or day.
  

  Options for Purging data from your storage

• Import Organization from CSV can now be scheduled using Tasks
  If you are importing your organizational structure from CSV, you can now schedule this action using Tasks. This enables you to update your organizational structure before any reports are run.
  

  Import Organization from CSV via Tasks

• Added Support for ExoServer Web
  If you’re running ExoServer Web, you can now analyze it’s logs using WebSpy Vantage.

Fixes

We also fixed some things that may have been bugging you:

• Improved the start time for the application by improving the logic to check for Storage damage.
• Fixed the IronPort loader (Fixed out of range issues on excessive size fields).
• “Having” filters no longer override the sort order of a Report Template node.
• Fixed an issue that may result in duplicated storages after migrating settings from earlier versions.
• Fixed the inability to remove invalid entities from web module permissions list (users that no longer exist).
• Fixed a timeout issue when publishing storages to the web module.

Why are you still reading? Go update now!

Have fun!

To use Windows Authentication, there are just a few things you need to do.

1. Set the Web Module’s Authentication type to IIS Integrated, and add your administrators in the form of domain\username
2. Enable Windows Authentication and disable Anonymous authentication in IIS.
3. Ensure all users in your Organization screen have a login name in the form of domain\username. Use the ‘Prefix’ option to prefix “domain\” (without the quotes) to your usernames names when importing your Organization from LDAP or LDIF.
4. Connect Vantage and the Web Module using the new authentication details and synchronize your Organization.

1. Set the Web Module’s Authentication type to IIS Integrated, and add your Administrators.

When you first install the Web Module, the first screen you see is the ‘Initial Configuration Wizard’ that guides you through the process of selecting your authentication type and specifying your administrator(s). If you have already been through this Wizard and are currently using Vantage In-Built or Client Certificate authentication, you can easily reset this initial configuration wizard. Simply login to the Web Module with your current administrator details and go to Options | Maintenance | Reset Initial Configuration Wizard.

Note: You can also change your authentication and administrator options individually using the Authentication and Administrator options on the Options tab of the Web Module.  However, for ease of demonstration, I’ll use the Initial Configuration Wizard method.

Now that you’re at the Initial Configuration Wizard, proceed through the wizard, selecting IIS Integrated authentication and entering your administrators in the form of domain\username (replace domain with your organization’s AD domain, and username with the sAMAccountName of your administrator.

Initial Configuration Wizard - Welcome Page

Initial Configuration Wizard - Authentication Page

Initial Configuration Wizard - Delegate Administrators Page

Initial Configuration Wizard - Summary Page

Click Finish, and if the authentication was successfully changed, you should get a message saying ‘The specified credentials were not accepted”.

The 'Specified credentials were not accepted' message.

Don’t panic at this point. This message is an indication that the authentication was successfully changed and that the Web Module is now listening for IIS to pass through Windows Usernames. The reason you’re getting this message is because IIS is not yet passing through Windows Usernames to the Web Module. This is configured in the next step.

2. Enable Windows Authentication and disable Anonymous authentication in IIS.

Now that the Web Module is expecting IIS to authenticate your users, you need to set up  IIS  to do this.

1. Open IIS by navigating to Start | Control Panel | Administrative Tools and double-clicking on Internet Information Services (IIS) Manager.
2. Navigate to the Web Module site or virtual directory in the left hand ‘Connections’ Panel. It will be located under <Server Name>\<Sites>. For example, MyServer->Sites->Default Web Site->webmodule.

• If you’re running IIS7 ( Windows Server 2008, Vista or Windows 7)
  1. Select the Web Module site and ensure the ‘Features’ tab is selected at the bottom of the middle pane.
  2. Double-click the ‘Authentication’ feature.
  3. Right-click ‘Anonymous Authentication’ and select Disable
  4. Right-click and ‘Windows Authentication’ and select Enable
  5. Restart IIS by selecting your server in the right hand connections pane, and clicking Restart in the ‘Actions’ pane on the right.

• If you’re running IIS6 or 5.1 (Windows Server 2003, Windows XP)
  1. Right-click Web Module site and select Properties.
  2. Go to the Directory Security tab
  3. Under ‘Authentication and access control’ click the Edit button.
  4. Uncheck ‘Enable anonymous access’ and check ‘Integrated Windows authentication’
  5. Restart IIS by right-clicking the local server, select All Tasks, and then click Restart IIS.

If you added your own Windows login name as an administrator in step 1, you can now test the authentication is working. Go back to the Web Module in your browser and click Refresh. You will be presented with an ‘Authentication Required’ dialog where you can enter your username and password.

Authentication Required Dialog

Again, ensure your username is in the form of domain\username. Click OK, and you should log straight into the Web Module using Windows Authentication.

3. Ensure all users in your Organization screen have a login name in the form of domain\username

Now your administrator account can log into the Web Module using Windows Authentication, but all other users will not be able to log in unless they have their login name specified in the form of domain\username. This is done in Vantage Ultimate on the Organization screen.

Organization Screen showing correct login name for Windows Authentication

If you’re importing your users from LDAP or LDIF, make sure you use the ‘Prefix’ option on the User Details page to prefix domain\ before your imported usernames. For example:

4. Connect Vantage and the Web Module using the new authentication details, and synchronize your Organization.

In order to publish information to the Web Module, you need to add a connection between Vantage and the Web Module. This is done on the Web Module screen in Vantage Ultimate.

1. Click Add Web Module (or if you already had a web module before changing the authentication details, select it and click Properties)
2. Enter the server & virtual directory of the Web module, and enter the correct credentials ensuring domain is specified.
3. Click OK to connect.

Connect to Web Module dialog

Once connected, synchronize Vantage with the Web Module by clicking the Synchronize link in the Web Module task pad. You may also want to provide permissions for your users in the Permissions section on the Web Module screen.

That’s it. You can now test that everything is working by getting one of your users to access the Web Module’s URL. They should sail straight in with no username/password prompt.

I hope this helps! Please let me know your feedback by emailing me, or leaving a comment below.

Something of great importance is taking the users of the network you intend to monitor into consideration. Overly intrusive practices can easily create the negative perception that Big Brother is watching and make employees feel frustrated and uncomfortable. Effective Internet monitoring requires a two-pronged approach; intuitive monitoring software AND workforce education / consideration.

This time around I would like to expand on the best ways of monitoring your organizational Internet usage whilst maintaining a harmonious working environment between employers and employees.

1. Allow for a certain amount of personal / recreational usage

Prohibiting all personal use is usually both impractical and virtually impossible to enforce in most work environments. Allowing a certain amount of (monitored) online recreation can enhance many workplaces and ultimately make employees more productive. Recent research by Dr Brent Coker at the University of Melbourne shows that people who do surf the Internet for fun at work – within a reasonable limit of less than 20% of their total time in the office – are more productive by about 9% than those who don’t.

2. Allow for a certain amount unmonitored usage

Employee privacy is a recurring concern when monitoring Internet usage at work. Employees are working longer hours than they ever before and might need to be able to deal with urgent personal matters online, during work time. By specifying and ensuring, that there will be no monitoring during lunch hours (for example), your employees can trust that their privacy is important to you, and you know it won’t affect their productivity.

3. Establish Acceptable Internet Usage Policies

Establish policies around Internet usage and:

• Explain the business-related reasons behind monitoring
• Clearly state what is considered productive and unproductive activity
• If you allow a certain amount of personal use, and / or unmonitored use during certain hours of the day, ensure you outline the exact specifications of these privileges
• Clearly state what is absolutely prohibited, for example, sending or accessing discriminatory, harassing, defamatory, or pornographic material, downloading or distributing copyrighted material without permission etc
• Include consequences for policy violations

4. Use an honest and open monitoring approach

The effectiveness of Internet monitoring directly relates to employees’ awareness of the content of the policy and corresponding breach consequences. Once your crystal clear policies have been developed, ensure you actively distribute, publish and communicate them so employees understand exactly what is expected of them and the conditions of their working environment.

5. Allow employees to view their own Internet usage

This is one of the best recommendations we can give you. More often than not, employees tend to underestimate the time they spend browsing non-work related sites. Allowing employees to view, for example, their productive and non-productive activity can help foster and drive responsible Internet usage behaviour. Employees who understand the organizational costs of their personal unproductive activities are more likely to accept your monitoring activities and modify their own behaviour accordingly.

6. Monitor the whole organization (even managers)

If you ensure everyone knows the whole organization is being monitored, as opposed to individual users or departments, you will decrease the likelihood employees feeling singled out or treated unfairly. Employees feel affirmed if procedures are adopted to treat them with respect and dignity and the likelihood of Internet monitoring acceptance and effectiveness is increased.

7. Help employees sticking to the rules

If you have set a limit of, for example, no more than 10 hours of recreational surfing per month, then ensure you alert employees when they are approaching that limit. Again, this will give the employees another opportunity to modify their own behaviour before they actually violate your Acceptable Internet Usage Policy.

8. Distribute reports – distribute responsibility

Frequently IT managers and administrators are given the ultimate responsibility of managing, enforcing and communicating acceptable Internet usage for an entire organization. Take some of the pressure off the IT department and distribute organizational Internet activity reports to responsible managers or department heads. This will enable them to see how Internet usage affects the security and performance of their own department and distributes the responsibility of enforcing acceptable usage with the managers themselves.

9. Protect employee privacy

If distributing Internet usage reports across your organization it is important to protect employees’ personal data. Make sure you use monitoring software designed to protect privacy rights by only allowing authorized users to see the employee’s identity. For instance, Network Administrators may need to investigate all traffic going to a particular site but should not need to know the user names – in this case user names should be anonymous for them but available for HR.

10. Automation

Find a monitoring solution that easily lets you customize and automate the majority of these guidelines for you. Right here might be a good place to start looking…

What do you think of these tips? Please feel free to comment below and share Internet monitoring tips that have assisted you in creating a productive and balanced working environment.

The survey only consisted of six questions and to prevent influencing answers and encouraging unique thoughts and opinions, the majority of questions were open ended. Those answers applicable for categorization have now been categorized, compared and correlated. The survey was conducted for internal purposes but I would like to share some of the findings and comments we received.

Vendor devices and log formats

The top ten vendor devices client used WebSpy’s software to analyze and report on:

1. Microsoft ISA Server (a whopping 48.6%)
2. Microsoft Exchange
3. Novell – BorderManager
4. Squid – Proxy Server
5. Microsoft Proxy Server
6. Microsoft – Internet Information Services (IIS)
7. Blue Coat – Proxy SG
8. Check Point – Next Generation
9. IronPort – Web Security Appliance
10. Microsoft – Windows Event Logs

Business Issues

The most common reason why clients started to monitor and report on their log files:

1. To achieve transparency, higher visibility, on overall internet and network resources. (No actual issue quoted)
2. To reduce issues related to lost productivity, inappropriate and illegal internet usage
3. To be able to perform detailed investigations on specific user and site levels
4. To reduce bandwidth cost and increase speed
5. To achieve monitoring requirements not satisfied by vendor device
6. To reduce viruses and other security issues
7. To produce reports required by management
8. To trend and forecast Internet and network usage
9. To investigate email usage
10. To immediately address critical issues using real-time monitoring and alerts

Most valued WebSpy software features

1. Dynamic Drilldowns
2. Ad-hoc analysis and summaries screen
3. Real-time monitoring and alerts
4. Comprehensive and detailed reports and analysis options
5. Predefined and customizable reporting and analysis templates
6. Aliases – Logical grouping of data to represent it more meaningfully
7. Profiles – Categorize web site URLs, email subject lines, instant message chats and any other logged data using customizable keyword profiling technology.
8. Extensive log file compatibility and support
9. Ease of use
10. Automated task scheduling

Suggested Improvements

The majority of answers related to suggested improvements and added software features were very specific to individual user experiences and quite difficult to categorize. However, here a list of suggested improvements that our development has taken to heart and decided to prioritize:

Increase and Improve Number of Default Report and Analysis Templates
Major changes and improvements to our reporting engine and interfaces are currently being planned. A short term goal is to provide the ability to create reports in PDF format. This should hopefully be available in the coming months.

Longer term, we will be incorporating a visual report designer (drag / drop charts and tables etc), as well as the ability to create reports that collate information from different sources. For example, using the information from Event logs, Web proxy logs and email server logs, you could produce a report that shows when a user logged on to their PC, what sites they browsed, what files they accessed, and the emails they sent – all sorted chronologically in the one table!

We currently have a large range of report templates available, but choosing the one you want can be a time consuming process, and you will most likely want to customize the report once you’ve found one that suits. We are planning to make the whole process of creating the report YOU want much, much easier.

Further Expand Profiles (Website Categorization)
When it comes to website categorization, WebSpy applications utilize keyword profiling in addition to importing category fields from log files (if available). There are upsides and downsides to keyword profiling. A simple keyword can instantly categorize thousands of sites which is much more efficient than maintaining a URL Category database. Being able to categorizing hits to your own organization’s website or intranet is also a very useful feature.

We are committed to distributing more frequent profile updates, without overwriting any customizations you have made. Longer term, we plan to provide an option for collaborative keyword customization, so that you benefit from the customization that other customers are making around the world. We also plan to integrate third party categorization services, as there are many great organizations that focus purely on URL categorization.

Improve Software Updates Correspondence
Our main mass communication channels include our newsletter, press releases blog and twitter. Here is where we inform existing and prospective clients about our software, updates, offers and much more. Starting September 2009 we will also send out emails, on a monthly basis, informing existing clients about the specific software updates implemented and are ready to download.

Further Improve Reporting Speed
Vantage 2.2 ranges (Premium, Giga and Ultimate) now use multithreading techniques to utilize the extra processing power on machines with multiple cores or CPUs to import log files and generate reports faster. In addition to the multi-processing improvements recently made in Vantage 2.2, the development team are planning to implement even more multi threading improvements in the next major version. These improvements will be available for users of both the Analyzer and Vantage range.

Thanks again for your participation!

The winner of the USB Camera Pen has been picked using a random draw and will be contacted personally via email.

Being a part of industry, it comes as no surprise that the popularity of social media is rapidly increasing. Nevertheless, many numbers and trends were surprisingly staggering.

Here’s a summary of the most interesting facts and numbers:

• Social media has overtaken porn as the #1 activity on the web
• 1 out of 8 couples married in the U.S. last year met via social media
• Years it has taken for each media to reach 50 million users:
  • Radio: 38 years
  • TV: 13 years
  • Internet: 4 years
  • iPod: 3 years
  • Facebook: 100 million users in less than 9 months
• Ashton Kutcher and Ellen DeGeneres have more Twitter followers than the entire population of Ireland, Norway and Panama
• 80% of Twitter usage is on mobile devices. People update anywhere, anytime.
• YouTube is the second largest search engine in the world with more than 100,000,000 videos.
• There are over 200,000,000 blogs and 54% of bloggers post content or tweet daily.
• 25% of search results for the world’s top 20 largest brands are links to user-generated content.
• 24 of the 25 largest newspapers are experiencing record decline in circulation

The video might even have the social media sceptics doubt. The effect social media has, and will continue to have, on society, consumers, businesses and employees is more apparent than ever before. Organizations in all industries should embrace all that the Internet has to offer, start making the most of the social media phenomenon, or risk missing the boat completely. Read previous blog covering the costs of blocking Internet usage

“We no longer search for the news, the news find us…In the near future we will no longer search for products and services, they will find us via social media…”

If you’re using WebSpy Vantage, you are probably interested in filtering your log file imports by date (only import files from the month of June for example). The obvious way to do this is to specify a date filter using the filters page in the Input Wizard. The problem is Vantage will still check every record in every log file being imported to see if it matches the date filter. If you have months or years worth of logs in the folder being imported, that’s a lot of data that Vantage has to pointlessly sift through.

The good news is, if your log files contain the date in their file name, then you can use file masks to instruct Vantage to never touch these unwanted files.

A bit about file masks…

You can specify file masks such as *, *.log, *.gzip, *WEB*.w3c, etc to import logs with specific file extensions, or with specific strings in the file name (such as WEB or FWS to import only Microsoft ISA Web Proxy or Firewall logs respectively).

But if your log file contains the date in the file name, you can also use date modifiers in the file mask to select logs from a particular month, date or year.

Say you have log files that look like this:

• 20090801.log

• 20090802.log

• 20090803.log

and so on..

You can create a simple file mask to only import log files from the month of August very easily using 200908*, or 200908*.log.

Using date modifiers in file masks

But if you’re using Tasks to automatically create a new storage each month, you don’t want to have to worry about manually changing the file mask to 200909*.log when the first day of the next month rolls around.

So intsead, you can use a date modifier in the file mask that will automatically select the logs for the current month, every time your task runs. For the above example, the file mask looks like this:

• %[yyyyMM]* (you can also use %[yyyyMM]*.log)

When the task runs, %[yyyyMM] will be replaced with actual values from the current date. So if the task runs on the 1st of August 2009, the file mask will become 200909* (or 200909*.log).

Dealing with different date formats

You can also use date modifiers for log files that look like this:

• 2009-Aug-01.log

• 2009-Aug-02.log

• 2009-Aug-03.log

In this case the file mask looks like:

• %[yyyy-MMM]* - notice the three MMM's as opposed to two MM's used previously.

Vantage uses the custom date and time format strings available in the .NET framework, so for more information on whether to use m or M or MMM, please refer to this article http://msdn.microsoft.com/en-us/library/8kb3ddd4.aspx

Importing logs from previous months

If you would like to import logs from a previous month, this can also be done by adding an additional element to the date modifier. For example, to import the previous months logs you can use:

• %[-1m,yyyyMM]*

Notice the -1m meaning ‘minus one month’. You can also use -1d (for minus one day), or -1y (for minus one year).

More examples

Here are some more examples to give you an idea of what is possible using date modifiers.  Assuming the date is 14th of August 2009:

• %[-1y,yyyyMM]*.log will create a file mask of 200808*.log

• %[yyyy-MM-dd]*.log will create a file mask of 2009-08-14*.log

• %[-4d,yyyyMMdd]*.log will create a file mask of 20090810*.log

• %[1-m,-4d,yyyyMMdd]*.log will create a file mask of 20090710*.log

• %[-1y,1-m,-4d,yyyyMMdd]*.log will create a file mask of 20080710*.log

• ISALOG_%[-1m,yyyyMM]*_WEB_*.w3c will create a file mask of  ISALOG_200907*_WEB_*.w3c

• *%[-1m,yyyyMM]* will create a file mask of  *200907*

Adding a file mask

File masks are configured on the Input Selection page of the Input Wizard, when you select Add | Folder.

Adding a File Mask

There is also an option to save the literal date into the file mask when the task is run.  For more information on this option, please see my previous blog about this feature.

Other uses for date modifiers

Date Modifiers are a great way to speed up log file imports, but you can also use them when specifying storage names as well as report names. For example, if you specify a storage name of %[yyyyMM]_storage, this will create storages with the names 200907_storage, 200908_storage and so on. When selecting the storages to report on, you can click the Add button on the storage selection toolbar in the Report Wizard, and specify storages such as %[-1m,yyyyMM]_storage, to report on the previous month’s storage.  For more information, please see Automatic Importing and Reporting using Tasks.

I hope this helps someone out there. Let me know how you go!

We’ve got a great deal at the moment where you get 20% off Live and Sentinel if you purchase them online with Analyzer Standard.

If you convert your logs to text using this script, they won’t import into WebSpy Vantage or Analyzer due to an extra line break in the header of the file (after #fields:).

We’ve therefore created a modified version of the script that creates compatible log files for WebSpy software.

Download the modified MSDEToText script:
MSDEToText.zip -26 KB

Also make sure the file names of your output log files contain the word WEB (for Web Proxy logs) or FWS (for Firewall Logs) as Analyzer and Vantage use these strings to automatically detect the type of ISA log file.

Happy converting!

Watching a single video on YouTube will probably generate a list of about three to five sites such as lax-v41.lax.youtube.com, www.youtube.com, img.youtube.com, and so on. Your list of top sites also probably contains hits to ad servers and tracking servers, such as doubleclick.net, google-analytics.com and imrworldwide.com. All this clutter gets in the way of determining what sites were ‘intentionally’ visited.

Fortunately there are a few simple steps you can take to exclude this information from your reports. Watching is much easier than reading, so I thought I’d create a video demo to walk you through the process.

By the way, this is the first video demo of what I hope will be many more to come. I created it using TechSmith’s Camtasia Studio which is by far the best screen recording software I’ve used. All the zooming you see throughout the demonstration is completely auto-magical! It’s a brilliant piece of software that has saved me hours of time. Props to the guys at TechSmith! The one pitfall of Camtasia is that it seems to make me sound like a geek with a raw Aussie accent… I hope they fix that in the next version.

Anyway, I hope you find this useful.

The concern for Internet misuse and costly drawbacks in the business arena were gradually rising and proactive Jack could already visualize an increasing need to monitor organizational web resources. Back then the software focused solely on analyzing and reporting on log files produced by proxy servers. Names such as Proxy Analyzer, Log File Reporter, Proxy Log Hog and Web Analyzer entered Jack’s brain but were quickly dismissed. He needed something better, something with more impact, a name that would draw attention, a descriptive name without the dullness, one name that could encompass the software’s functionality and create a conceptual understanding of what the company was all about….one name to rule them all…

WebSpy at trade shows

To no one’s surprise the name of choice was WebSpy. The name was almost perfect. It got a tick for all aforementioned qualities, it received the attention sought for and proved to be very controversial from a marketing point of view. I can personally vouch for the attention and curiosity we generate at events such as trade shows.

It is now more than a decade since the birth of WebSpy. The softwares’ features and capabilities have changed dramatically. Now allowing organizations to monitor and report on, not only web usage, but also; email usage, instant messaging, event logs, routers, website visitor traffic, firewalls, anti virus and anti-spam applications, the name WebSpy remains the same. Why you might ask. Why keep a name that actually understates the software’s capabilities? Why keep a name that may be perceived as associated with unethical spying activities?

Yes, WebSpy has experienced a few instances when organizations have been suspicious. Where initial cooperation has been resisted based on a misconception about us stemming from the name. However, the name comes with great weight and legacy. WebSpy was one of the industry pioneers in the 90’s and has many years of brand and reputation building. This is something that can’t be tossed out the window and replaced with a generic or uncontroversial name, just to prevent potential misunderstandings. Needless to say these misunderstandings are generally easy to clear and invalidate.

Instead we make an extra effort to prevent misconstructions. We use our website, marketing communication and workforce to always clearly state our mission and conviction. So for the record, just in case someone has missed it:

We do not by any mean promote or condone spying on employees. We are pro-internet access and provide businesses, government departments and educational organizations an alternative to blocking and filtering software. We emphasizes that organizational internet usage should be managed using an honest and open monitoring approach where acceptable internet usage policies are clearly communicated to employees and students.

The End

All three products in the Vantage range (Premium, Giga and Ultimate) have been updated with the following features:

• Multi-processing
  Vantage now utilizes the extra processing power on machines with multiple cores or CPUs to import log files and generate reports faster.
  
  

  The new Performance tab in Vantage

• Improved report styles
  HTML (including MHT) reports are now sporting an updated look and feel.
  
  

  New Report Style

• Updated Aliases and Profiles
  Added support for the latest search engines, social networking sites, operating systems and user agents.
  
  

  Profiles Aliases support the latest Internet tools and technologies.

• Storage Repair Utility
  Storages can become damaged if there is a system crash or if WebSpy.Vantage.exe process is ended when an import is in progress. Damaged storages are now detected automatically and you have the option of repairing them. Storages can also be manually checked and repaired on the Storage properties dialog.
  
  

  Storage Repair Utility

• Save / Open Tasks
  Scheduled Tasks can now be saved to an external file (.Tasks) for backup and migration purposes.
  
  

  Save and Open Tasks

Obviously, the multi-processing feature is the big one! Many of you (Vantage customers) have been running Vantage on multi-core or multi-cpu machines and just dying for Vantage to grab hold of the CPUs and make them work. You should be happy with this new build. Import a folder full of log files and watch 6 to 8 logs import simultaneously instead of watching them import one after the other. Run a full analysis or a comprehensive report with no filters and watch your CPU jump into gear to generate the report in 30% – 50% of the time. We’re expecting this to be a very popular update, especially among our larger customers.

Multi-processing is a great new feature, but be aware that there are some circumstances where Vantage will not be able to utilise more CPU power when generating reports. For example, if your report contains filters that significantly cuts down the amount of data in your storage that needs analysis, Vantage’s analysis engine and your CPUs will not be significantly pushed. Also see my earlier blog for more information on how the report structure can also affect the performance.

To get hold of this build, simply download the latest version of Vantage from our website. It will run side by side with your existing Vantage 2.1 installation, which minimizes any downtime, and makes it easy to transfer all your settings across (use the Migration Wizard in the Tools menu). Then you can uninstall Vantage 2.1 when you’re ready.

Let us know how you go by leaving comments below, or by tweeting us.

Cheers!
Scott

Certain U.S. Army bases, that formerly blocked access to Web 2.0 sites, now permit users to surf to sites such as Facebook and Flickr. The Army has ordered its network managers to give soldiers access to social media and thereby reversing a years-long trend of blocking web 2.0 sites on military networks.

SCMagazineUS.com interviewed Marcus Sachs, director of the SANS Internet Storm Center and a retired Army officer, who says, “It’s a recognition that soldiers are using Facebook and Twitter as part of their jobs, not just for recreation.”

Sachs explains that the decision to allow social networking sites requires a balance between allowing technology that the soldiers are familiar with and want to use versus the concerns related to social network sites. Sachs says ”Keeping these sites blocked is a good defense to keep networks free of malware infection, but such a move cuts down on how efficient soldiers can be.” Sachs also points out that confidential data leakage is another concern that will be tackled by educating soldiers how to appropriately use social networking sites.

This order is truly a step in the right direction. Not only does the army recognize soldiers’ need to communicate with family and friends but also the efficiency issues arising when blocking Internet access. As clearly indicated there are several issues that may arise by providing access to social media sites. However, most of these can be tackled by educating soldiers (or workforce) on what is accepted and what is not and ensure online privileges are used as intended. Hopefully more government, educational and commercial organizations will follow suit.

Related Articles:
SC Magazine – Army ends ban on Facebook, Flickr, other social media sites

Well, it’s now been released and can be downloaded from the Microsoft Download Center:
http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=e05aecbc-d0eb-4e0f-a5db-8f236995bccd

One of the major improvements in Beta 3 is URL filtering which leverages Microsoft Reputation Services (MRS). With regards to this, Microsoft says:

“At the time of this release, the MRS database content is being populated and updated continuously as part of the initial beta service offering. As this process continues, URL filtering categorization accuracy and comprehensiveness will increase. A telemetry package designed for improving the quality of URL filtering database and collecting your feedback is planned to be released soon. Please check back for updates in August.”

Support for importing Microsoft TMG log files into your favorite WebSpy product is coming soon so stay tuned! Subscribe to the WebSpy blog RSS feed or follow us on twitter if you want to be notified as soon as it’s available.

Dr Brent Coker is seeking organizations who would be interested in participating in a study to examine how WILB affects their employees’ productivity. Initial evidence from Dr Coker’s research suggests WILB in certain circumstances can result in improved productivity.

Participating in this research will offer organizations a chance to understand the extent to which their employee productivity is being affected by WILB. The degree to which organizations participates is optional, though the research may include analysis of data before and after internet access was controlled, or employee/supervisor interviews/surveys and analysis of employee performance data after internet access was monitored (e.g., using WebSpy).

This research is to be completely anonymous; data will be de-identified before analysis, and is subject to the privacy conditions provisioned in the University of Melbourne research statute. University of Melbourne ethics approval (#0828412.1) has been granted for this research.

We strongly encourage our clients to get involved in this unique opportunity to get a concrete insight into how WILB affects their organization’s operations specifically and:

• Get a better understanding of your employee’s loyalty and motivation to excel in the workplace.
• Proactively take measures to create a more harmonious and productive working environment.
• Use Dr Coker’s expertise and findings to ensure your Internet investment is used to its full potential.

To find out more information or to register your interests to participate in the study please contact:

Dr Brent Coker
Department of Management and Marketing
Level 9, 198 Berkeley Street
The University of Melbourne Victoria 3010 Australia
Tel:  +61-3-8344-1933�
Fax: +61-3-9349-4293
Email: bcoker@unimelb.edu.au

Read “Freedom to Surf” media release from the University of Melbourne
Read WebSpy’s media release discussing findings from previous study

READ RELATED ARTICLES:
CNET News: Study: ‘Leisure browsing’ increases productivity
People at Work and Play: WILB is Good for Your Employees
WA Today: Surfing the Net at Work can Boost Productivity
Fierce CIO Tech Watch: Surfing the Net Boosts Office Productivity
Security Vibes: Blocking Social Networks Decreases Productivity
ABC Net: Workplace Browsing Boosts Productivity: Study
Technology & Business: Facebook, YouTube at work make better employees: study

This Week in Startups (TWiST) and Martin Bailey Photography Podcast (MBP Podcast)

TWiST features entrepreneur Jason Calacanis and a rotating group of guest experts who bring you a weekly take on the best, worst, most outrageous and interesting from the world of Web companies. LA based Calacanis is known for his transparency and insights into the internet and media industries and his show recently ranked #12 on iTunes top video podcasts.

The MBP Podcast is hosted by Martin Bailey himself. Bailey is a renowned Tokyo based photographer specializing in nature and wildlife photography. He is an established podcaster with more than 200 episodes under his belt. Listeners will benefit from his expert advice on photo creation, explanation of techniques and talks about photography in general.

Although completely different the podcasts’ common denominators are their technological proficiency, progressive thinking and understanding of the utmost importance of workplace internet access. Something highly valued at WebSpy.

WebSpy is also offering podcast listeners a special deal when purchasing our products. Want to know what it is? Download the FREE podcasts from iTunes or below web sites to find out.

This Week in Startups

Martin Bailey Photography

Do these figures mean that employers are cutting back workplace Internet access? Is organizational URL blocking a desperate attempt to decrease Internet costs in our sluggish economic environment? Is it really the best way to decrease costs? What are the hidden costs? To block or not to block, that is the question…

Unlike any Shakespearean dilemma this question has a straightforward answer: Don’t block your employee’s Internet usage, not even social networking sites.

Blocking employee Internet use can open up a can full of nasty worms. Depriving the needs of a professional workforce can cause resentment and increase costly turnovers. Blocking Internet access also has potential to reduce productivity by complicating or delaying accomplishment of tasks. Organizations not allowing employees accesses to specific networking sites will also risk foregoing online networking opportunities, locating critical contacts, leads and receive timely industry news.

Adding to this, recent research by Dr Brent Coker at the University of Melbourne shows that people who do surf the Internet for fun at work – within a reasonable limit of less than 20% of their total time in the office – are more productive by about 9% than those who don’t. Coker says, “People need to zone out for a bit to get back their concentration. Short and unobtrusive breaks, such as a quick surf of the Internet, enables the mind to rest itself, leading to a higher total net concentration for a days’ work, and as a result, increased productivity.”

What about all genuine concerns about employee Internet usage, lost productivity, ridiculously high bandwidth costs and legal obligations? What about rightful remarks similar to Telstra Business executive director Brian Harcourt’s, “Tweeting, friending or poking your way through the working day may not be the best way to improve the productivity of those many small businesses which are battling to find a way through the challenging economy.”

Of course organizations must look after their resources and make sure the Internet is used as intended. Don’t just allow employees full Internet access, close your eyes and hope for the best. As Harcourt nicely puts it, “There is a clear need for formal policies on the use of social networking sites in the workplace and the appropriate and effective software tools that support those policies.”

So what is the appropriate and effective software tool? Internet monitoring and reporting software of course.

Allow employee Internet access, establish acceptable usage policies AND make staff aware that monitoring is taking place to ensure policies are adhered to. Even better, allow employees to view their own usage, for example, their productive and non-productive activity. This will help foster and drive responsible Internet usage behaviour. Employees who understand the organisational costs of their personal unproductive activities are more likely to accept the organisation’s monitoring activities and modify their own behaviour.

Costs are cut, employees are happy, productivity is maintained (or even increased), simply a win-win situation.

Read related articles:
Businesses chop facebook to save internet costs
A Growing Acceptance of Social Networking in the Workplace

According to the latest blog from TMG’s Product Unit Manager, David B. Cross, Beta 3 will be released in the next couple of weeks. As for the full release, David says that they are still on track for Q4 this calendar year.

Beta 3 will introduce URL filtering that is ‘fully integrated’ with TMG’s web policy rules, and also utilizes Microsoft Reputation Services.

Microsoft are also introducing Intrusion Prevention and Detection (IPS/IDS) capabilities in TMG. These systems will utilize a technology they’re calling Network Inspection System (NIS) that detects attacks using signatures of known vulnerabilities, downloaded from the Microsoft Malware Protection Center. For more information on NIS see http://blogs.technet.com/isablog/archive/2009/04/12/exercising-nis-with-test-signature.aspx

If you’re currently using ISA 2004 or 2006, upgrading to TMG will consist of exporting rules and settings from ISA, then importing them into a clean installation of TMG. TMG will also only run on Windows Server 2008.

Improving the on-box reporting has not been a focus for the TMG development team, so analyzing TMG’s web proxy and firewall logs is still the best way to go for in depth reporting.

If you’re interested in reporting on your TMG log files stay tuned! We’re currently implementing support for the SQL Express, W3C and Native text logs. WebSpy Vantage is likely to be the first application to include the feature, with Analyzer and Live soon to follow.

All going well, you can expect to see TMG support in your favourite WebSpy app within the next month or so. If you want to be notified once we’ve added support, just leave a comment below.

Cheers!
Scott.

• Windows Vista & Server 2008:
  C:\Users\<user profile>\AppData\Roaming\WebSpy\Vantage <edition> <version>
• Windows XP & Server 2003:
  C:\Documents and Settings\<user profile>\Application Data\WebSpy\Vantage <edition> <version>

When Vantage opens, it loads the information contained in these files.

To move all your settings across, you can simply copy all the files in this location on your original machine, into the same folder on the new machine (make sure Vantage is closed when you do this).

When you open Vantage on the new machine and you’ll have all your settings moved across.

If you’re using scheduled tasks, one thing to note is that copying these files will not recreate the Windows scheduled task jobs. To get your tasks functioning on the new machine:

• Go to the Tasks tab and double-click each scheduled task.
• Proceed through the wizard and make sure all the settings are correct.
• Enter your authentication details on the last page and click OK.

This will create a new Windows scheduled task job which should run as it did on the old machine.

Please note that this process only moves ’settings’ across to the new machine. If Storages and Reports were being written to a location on the old machine, then you will need to move these across to the Storages and Reports locations on the new machine (check Tools | Options | Paths for these locations).

Also take access privileges into account. If you’re old installation was set to create storages in \\server\mystorages, make sure the new machine also has write privileges to this location.

That’s pretty much all there is too it. Happy migrating!

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
	<appSettings>
		<add key="synopsisCallStack" value="true" />
		<add key="traceLog" value="true" />
	</appSettings>
	<system.diagnostics>
		<switches>
			<add name="LogOutput" value="4" />
		</switches>
	</system.diagnostics>
</configuration>

After creating the file, restart Vantage. A trace log will be created in your local application data folder;

XP; C:\Documents and Settings\%username%\Local Settings\Application Data\WebSpy\Vantage (edition)\(build)\trace.log

Vista; C:\Users\%username%\AppData\Local\WebSpy\Vantage (edition)\(build)\trace.log

The trace log shows the list of steps performed, and the details of any errors encountered. This information is useful to WebSpy when diagnosing errors reported by users.

Of particular note is a fix to the Vantage range:

• Fix: Improved partition matching when filtering by a date range

This will benefit you if you’re running reports with a date range filter such as last week or yesterday (which is pretty much everyone), and are using the default storage partitioning scheme of “Date” (again… pretty much everyone).

There was an issue in Vantage’s partition filtering technology which meant that if you specified a date range filter, Vantage would still analyse an entire storage, instead of just the date partitions you selected in the filter.

So do a Tools | Check for updates to grab build 2.1.2.12 and you should see a significant speed improvement in report generation. Woo hoo!

Here’s more details on the changes:

VANTAGE

Application Changes

• New: Added support for importing event logs from non-domain machines.
• New: Added the ability for the Troubleshoot Alias/Profiles to export results to a file.
• New: Added hash salt and substring functions to the expression language.
• New: Added support for computing a date-modified file mask at import time and storing it back into the storage.
• New: Added a file mask override to the Import new hits to existing storage task action.
• New (Vantage Ulitmate & Giga): Added support for attributes on Organization groups.
• New (Vantage Ultimate): Added custom expression-based split in Web Module publishing (Beta).
• New (Vantage Ultimate): Changed the publish report wizard to allow selection of multiple storages.
• Fix: Improved partition matching when filtering by a date range.
• Fix: Fixed an issue with registrations and trial extensions.
• Fix: Fixed zero-width rectangle exceptions in chart renderer.
• Fix: Fixed overflow exceptions on footer generation in report tables.
• Fix: Fixed XmlException error during settings load.
• Fix: Storage now clears previous data when it is overwritten when using the ‘Import logs into new storage’ task.
• Fix (Vantage Ultimate & Giga): Fixed issues with date range selection when collating reports.
• Fix (Vantage Ultimate): Web Module dock no longer checks for updates to a web module server when ‘Check for updates on startup’ is disabled.

Loader Changes

• New: Added Phion Firewall
• New: Added Watchguard Firebox X Core
• New: Added NetScreen 208
• New: Added IPCop
• New: Added Astaro Mail Gateway
• New: Added iPrism Monitor v4.2
• New: Added Cisco VPN Concentrator
• New: Added Snare for Lotus Notes
• Improved: Added string host name to Kerio Mail Server
• Improved: Improved support for NetIntact PacketLogic
• Fix: Made changes to the IronPort detection method to fix the issue when importing via FTP.
• Fixed: Fixed date format issue in event log import. You can now import event logs in any regional configuration.
• Fix: Postfix loader no longer drops session state after the first recipient line
• Fix: Updated the detection method in CheckPoint Firewall-1 Syslog format.
• Fix: Fixed an issue in iSheriff where a drilldown on the Category field would display no results.
• Fix: Various fixes to Netscreen 10
• Fix: Various fixes to Arkoon PxLog
• Fix: Vaious fixes to Sendmail MTA

VANTAGE WEB MODULE

• FIX: When sorting by key column the chart will now use the “hits” aggregate for value (prevents the chart from going wacky)

ANALYZER

Application Changes

• NEW: Added support for manual configuration of ISA block list settings (Tools | Options | Block Lists)
• Fix: Fixed an issue with registrations and trial extensions
• Fix: Fixed report wizard configuration to restore the sort column from the report template

Loader Changes

• New: Added Webroot
• New: Added Qmail Desknow Mail Server
• New: Added UserGate Proxy Server 2.7
• New: Added Netgear FVX538
• Improved: Added support for importing allowed/blocked status in Sophos Web format
• Fixed: Changed CC Proxy field count check
• Fixed: Changed ISA SQL loaders to use size delta fields instead of cumulative fields
• Fixed: Modified Exchange 2000/2003 loader to discard message cache after import

LIVE

Application changes

• Fix: Fixed an issue with registrations and trial extensions

Loader changes:

• New: Added Webroot
• New: Added Qmail Desknow Mail Server
• New: Added UserGate Proxy Server 2.7
• New: Added Netgear FVX538
• Improved: Added support for importing allowed/blocked status in Sophos Web format
• Fixed: Changed CC Proxy field count check
• Fixed: Modified Exchange 2000/2003 loader to discard message cache after import

The fix can be applied to WebSpy Analyzer Giga 2.3, Analyzer Premium 4.3 or Analyzer Standard 4.3. If you’re not running the latest version, download it now!

You can download the new loader build that we created today at either of these locations:
USA West Coast (FTP)
USA East Coast (FTP)

Then extract the zip file into Analyzer’s installation folder (usually C:\Program Files\WebSpy\Analyzer flavour 4.3\) and overwrite the existing file.

Then go to the storages screen and select your Sophos storage(s) and click ‘Reload all hits’. This will re-import your log files using the modified loader and will populated the ‘Blocked’ summary appropriately. To check it out, go to the Summaries screen and run a Full Analysis. Then go to the ‘Blocked’ summary and you should see two items – ‘Blocked’ and ‘Not Blocked’. Drilldown into whichever one you care about to analyze the sites, users, files, browsing times, size downloaded etc. Go nuts!

You can also filter out blocked hits (or Not Blocked hits) from your reports. On the Reports Screen, click Generate a new report and go through the report wizard with this filter (this example shows filtering out blocked hits).

Analyzer Report Wizard - Select Custom Filters

Analyzer Report Wizard - Selecting the 'Blocked' Summary as a Filter

Analyzer Report Wizard - Adding the items that you want to filter

Analyzer Report Wizard - final filter to exclude 'Blocked' hits

Then proceed through the report wizard to generate your report. This filter can be applied to any report as well as analyses on the Summaries screen (using the same options in the Analysis Wizard).

Happy analyzing!

Unfortunately, Exchange 2007 tracking logs are not used to simple questions, and are likely to return a complicated and / or misleading answer.

But the confusion it seems, all comes down to definitions. Once you understand these definintions, things start to make a bit more sense.

What is an Email?

If you send an email to one person, you’ve sent one email. But if you’ve sent that same email to 500 people, have you sent one email, or 500? I will take a guess, and say that a large majority of you will want to see 500 in your reports.

Microsoft Exchange 2007 tracking logs contain an excellent field called ‘Message ID’. If you send an email to someone, that message is uniquely identified by a Message ID that persists though Exchange’s various functions for the lifetime of the message.

At first glance, it seems that counting Message IDs will give us what we want. But if you send the same email to 500 recipients, all those emails get the same unique message ID. So counting message IDs will show us that only one email has been sent. No good.

Then next obvious step is to count the number of recipients that received the email.

What is a Recipient?

The definition of recipient can also get clouded when you start talking about distribution lists. If you send an email to one real person, then that is one recipient. If you send the same email to five real people then that is five recipients. If you send an email to an internal distribution list, the number of recipients is the number of people that are members of that distribution list.

If you send an email to an external distribution list (such as SalesDL@othercompany.com) this will only be recorded as only one recipient, as your Exchange box has no way of knowing how many real people are members of that DL at the other company.

How do I count Recipients?

Again, Exchange Tracking logs contain another excellent field called ‘Recipient Count’. But don’t get carried away as this too can be misleading.

Without going into specifics, Exchange has a bunch of internal functions to deal with an entire message transmission. The tracking logs files contain another excellent field called Internal Message ID that identifies each of these processes per-message.

Unfortunately, each Internal Message ID contains its own value for ‘Recipient Count’. So when you sum the Recipient Count field for a single message, the final result may be much larger than the actual number of real recipients.

To illustrate, WebSpy Vantage imports Recipient Count into a Summary of the same name. Here is a screenshot of the Recipient Count Summary for one message

The Recipient Count Summary for a Single Message

As you can see, there are multiple rows of individual Recipient Counts. The first row, is actually correct. This email was actually sent to 961 people. But there are additional entries where Exchange performed an internal operation with a subset of those messages. Therefore, summing the Recipient Count field for a message is also no good.

Counting recipients “properly”

The best way to count recipients is to use WebSpy Vantage to import your logs, then drilldown into a message to the Recipients summary and look at the total number of recipients at the bottom. Alternatively, add a Count Distinct aggregate for the Recipients summary to any report template.

Here’s a screenshot of the Recipients summary:

The Recipients Summary showing Total 'Real' Recipients

And here’s a screenshot showing how to add the aggregate to a report template:

Adding the Number of Recipients to a report template

Counting Total Number of Emails

The above screenshot will give you a count of all the recipients you have ever sent email to. However, what you really want is a count of recipients per message. You can do this by concatenating the Recipient with the Message ID, and counting the total number of rows. To do this, edit the Number of recipients aggregate column above and enter [Recipient] + [MessageID] in the ‘Custom’ edit box.

Customizing an aggregate column to concatenate Recipient and MessageID

Exchange 2007 Report Templates

You can download a WebSpy Vantage Templates file here that includes three reports (Email Overview, User Email Activity, and Email Trends) that uses columns such as Number of Emails, Number of Unique Messages and Number of Recipients.

Tip: You can convert any email template that has the schema ‘All Mail Schemas’ into an Exchange 2007 template in order to report and filter using all the fields available in Exchange 2007.

To do this:

1. Right click an ‘All Mail Schema’ email template and select Duplicate.
2. Select Microsoft Exchange 2007 from the schema drop down and click OK.
3. When you edit the nodes in your new template, you will have access to all the fields that Exchange 2007 records.

Cheers!

A common setup is to have a task that creates a storage each month, then a separate task that runs at the end of each day – say at 10pm, to imports new hits into the storage.  The issue here is that at the end of the month, hits between 10pm and 12pm will never be imported, as the next time the task runs it will be dealing with next month’s storage.

As long as your log files contain the date in the file name, you can use date modifiers in file masks to import the appropriate log files. Date Modifiers are used to create file masks such as ‘access_200902*.log’, or ‘access_20090203.log’ that will cull down the number of log files to import. This is much faster than using date filters (specified on the filters page).

The Issue:

The issue is that a log file location in a storage can only have one file mask, and this cannot (until now) not be modified by task actions.  So you cannot specify a new task on the first of each month to import yesterday’s log file into last month’s storage, if that storage already has a file mask to only import the current day’s log file.

The two solutions:

1. Save the literal file mask into the storage when the storage gets created.

This option allows you to specify a file mask such as  access_%[yyyyMM]* in the ‘Import logs into new storage’ task, but when each storage is created, the actual file mask that gets saved into the storage includes the literal month and year values such as access_200902*

This way you can specify an ‘Import new hits’ job, and even if that job runs in the next month,  ONLY log files from the desired month will be checked for new hits.

To access this option:

1. Go to the Tasks screen and edit your ‘Import logs into new storage’ action
2. Go to the Input Selection page
3. Select your log folder (not individual files) and click Edit
4. Check the ‘Save literal date into mask’ check box


Save literal date into file mask option

Downside of this option is that in my ‘monthly storage’ example, all 31 log files for the month will be checked for new hits.

2. Override a storage’s file mask using the Import New Hits task action.

The Select Storage dialog that appears when configuring an ‘Import new hits into existing storage’ action now has a new option to override a storage’s file mask when the task runs.



Override the file mask option

This allows you to configure a task that runs on the first day of each month to import only yesterday’s log file into last month’s storage (need to add last month’s storage using the Add button, and specify the appropriate date modifier to access it – e.g. “%[-1m, yyyyMM] My Monthly Storage”).

This will only check one file for new hits and will finish the import job faster than option 1 (using the monthly storage example).

So here it is (you might want to grab a coffee first!).

Unless you have a sophisticated end point security or file auditing solution in place, you’re pretty much limited to the quality of data found in your Windows Security Event log. By default, accesses to your confidential files are not going to trigger any entries to be written to the Event log. You first need to setup file or folder auditing.

WebSpy have written a nice article to help you out with this: Managing Event Logs

Personally, I’m running Windows Vista SP1.  So I first turned on Object Access auditing by going to Control Panel | Administrative Tools | Local Security Policy | Local Policy | Audit Policy and set Audit Object Access for Success and Failure.

Windows Vista Local Security Policy

In Windows Explorer, navigate to the folder or files to audit, then Right-click | Properties | Security | Advanced | Auditing and click Continue when Vista’s User Access Control gets in the way.  Here you get the option to add Users or Groups to the audit policy. So if you only want to know when Joe Bloggs access the file/folder, then only add Joe Bloggs. If you want to know when anyone accesses the file/folder then add your entire company.

Scroll….

Click OK and apply the changes. If applying this to a folder, take note of the setting to ‘apply the auditing entries to containers within this container’ at the bottom and use as required.

Congratulations. That’s the auditing setup. Once people start accessing these files(s), the auditing information will get recorded to the Security Event Log on the machine that hosts the file(s) in question.

The next step is to import the Windows Security log into your flavour of WebSpy Vantage. I’m using Vantage Ultimate, but the steps are the same for Premium and Giga.

1. Run Vantage (as Administrator if on Vista)
2. Go to the Storages tab and click Import Logs
3. Run through the Import Wizard with these settings:

• Storage: New storage
  

  Input Dialog: Storages Page

• Input Type: Windows Event Log
  

  Input Dialog: Input Type Page

• Loader Selection: Microsoft


Input Dialog: Loader Selection

• Input Selection: Add
  Select either local computer, or multiple computers, enter authentication details and Click ‘Filter Event Logs’. Check the ‘Security’ Log and click OK.
  

  Input Dialog: Input Selection Page - Adding Event Logs

• Click OK to start the import.

If there are any issues with the import process, consult these three WebSpy Knowledgebase articles to do with issues importing event logs:

• Event Log Troubleshooting (Known Issues and Fixes)
• Importing Event Logs from machines on a different domain
• Required Services for Event Log Importing

The first article came in handy for me as I’m running on Vista and in order to import from the Local Security log, you need to run Vantage as Administrator. To do this, go to C:\Program Files\WebSpy\Vantage Ultimate 2.1\ right-click the WebSpy.Vantage.exe and select ‘Run as Administrator’.

Once data has been imported into your storage, check it out on the Summaries screen.

To to the Summaries Tab, Run an Analysis on your new storage (ad-hoc analysis will do) , and go to the Category Summary. There should be some ‘File System’ items there assuming the file has been accessed since setting up file auditing. You can then drilldown to Event Type to see ‘Audit Success’ or ‘Audit Failure’. To see who has Successfully accessed a certain file, drilldown into the ‘Audit Success’ item.

Unfortunately the good stuff is buried in the ‘Message’ field, which you can only access in the Individual Records view. This is because the Message field in Event logs is free form and could vary wildly resulting in millions of unique items. A Message Summary has therefore been excluded from a default ad-hoc analysis for very good performance reasons.

Event logs can also be quite verbose, and if you drilldown to Individual Records at this stage, you’ll see lots of messages like ‘A handle to an object was requested’ which probably isn’t of any great value from a reporting perspective. One way to filter out this noise is by Event ID.

I’ve discovered that the events that correspond to ‘An attempt was made to access an object’ have the ID 4663. (One day I’ll create an alias to map Event IDs to their meaningful description. If you come across a good  resource I can use for this, let me know!).  So go to the Event ID summary and drilldown into 4463 to the Individual Records view.

Once you’re at Individual Records, you can hover over the message field to get details. You can also use the find edit box to search for a particular user or file:

Drilldown into Successful File System Accesses (Event ID 4663)

You can export this view To Word Document, HTML, Text or CSV by right-clicking the Individual Records summary and clicking Export.

You can also create a report template to access this same information, but as there is no ‘Message’ summary to choose from, you need to use the Custom expression options, both when adding a column to a node in a Template, and when specifying your filter.

To add a column to a report that displays an Event Message:

1. Go to the Reports Tab and click New Template
2. Create an Analysis template based on the ‘All Windows Event Schemas’ schema
3. Click New Node and click the Advanced button to launch the Advanced editor.
4. On the General page, delete any existing Key columns and select Add | Key. In the Custom Expression section enter [Message] (include the square brackets) and click OK.

To filter the report:

1. Go to the Filters page of the New Node dialog (alternatively you can specify this filter in for all nodes using the Template Properties dialog)
2. Click Add | Field Value Filter. Select Category from the Summary drop down, and click Add. Enter ‘File System’ (without the quotes) and click OK. Click OK to add the filter.
3. Click Add | Field Value Filter. Select Event ID from the Summary drop down and click Add. Enter ‘4463′ (without the quotes) and click OK.
4. To filter on the Message field, Select Add | Manual Filter Expression.
5. Enter the expression:
6. [Message] LIKE “text to filter for”
   Change ‘text to filter for’ to the user or file that you want to search for. If you want to search for multiple strings, repeat the above expression separated by an AND or an OR, and place brackets wherever it makes sense. For example:
   • [Message] LIKE “scottg” AND [Message] LIKE “.avi”
     Will filter for all .avi files that scottg has accessed.
   • [Message] LIKE “scottg” OR [Message] LIKE “.avi”
     Will filter for any file that scottg has accessed and any avi that anyone has accessed.
   • ([Message] LIKE “scottg” AND [Message] LIKE “.avi”) OR [Message] LIKE “andrew”
     Will filter for any all avi files that scottg has accessed and any file that Andrew has accessed.
7. You can add the individual filters using Add | Manual Filter Expression multiple times, and then using the Manual Filter Expression editor at the bottom to change ANDs to Ors and place brackets appropriately, like so:
   

   Filtering for File Access Events by particular users

8. Right-click the Manual Filter Expression edit box and select Validate to make sure everything is good with the expression.
9. Modify chart settings, sorting, etc as appropriate.

Here’s the resulting report template for you, but please note that it includes the filter above (events for the user’s  ‘Asa’ and ‘Scottw’), so you will need to modify the filter and enter the users or files you want to filter on. Just use the user’s windows login name, and/or the name of the file.  Alternatively, remove the filter altogether if you want to see all File Audit events.

That’s it! Now run your report, automate it using the Tasks screen, and your set!