Why there is so much anonymous traffic in Microsoft TMG and ISA logs

One of the most common questions we get asked by users of Microsoft TMG and ISA is why there is so much traffic attributed to the Anonymous user. Even though unauthenticated access to the web has been disabled, they still see the ‘Anonymous’ user as one of the top users in their reports.

So let’s use WebSpy Vantage to drill into that Anonymous user and find out what is going on.

One way to do this is to run an Ad-hoc analysis on the Summaries screen and drilldown into the Anonymous user to view all the information about that user. However, TMG and ISA tend to log a lot of information that may not be relevant to this particular investigation, so I’ve created some report templates (one for ISA and one for TMG) and a set of Aliases that pull out some relevant information.

Download our Anonymous Traffic Investigation Report

If you’re running WebSpy Vantage download the Anonymous Traffic Report Templates & Aliases

Then open the .Templates file on the Reports tab, and the .Aliases file on the Aliases tab. Once you have both files opened, go to the Reports tab and click either the ‘Anonymous Traffic Investigation (ISA)’ or the ‘Anonymous Traffic Investigation (TMG)’ report. Then click the ‘Generate report’ link and run the report template on your ISA or TMG storage.

The report gives you the ability to drill into the Allowed, Denied and Failed traffic to see a list of the unauthenticated IPs, Sites, Rules responsible for blocking or allowing the traffic, unauthenticated Applications and Result Codes.

Main causes of anonymous traffic

What you will probably find is that most of the Anonymous traffic is being denied by your TMG or ISA firewall. When a client first requests a web page, the proxy will challenge the client for authentication. These events are often logged with the result code 12209 meaning ‘authorization is required to fulfill the request’. These requests are therefore denied by the proxy until the client’s credentials are authenticated.

Have a look at the amount of traffic being denied and then checkout the Result Codes associated with the denied traffic. Chances are you’ll see ‘proxy authentication required’ appear predominantly.

If you also look at the Applications section you may also find that Windows Updates are sailing through your TMG or ISA firewall unauthenticated.

Filter out unauthenticated traffic from Reports

The most logical next step is to filter out the information you do not want in your reports. You’ll probably still want to include Windows Update traffic in your reports, but you’re probably not so interested in the ‘proxy authentication required’ information. So let’s filter that out.

To do this:

  1. Go to the Reports tab and select the report you want to filter (such as your Organization report)
  2. Click ‘Edit Template’, then click ‘Template Properties’.
  3. In the filter section at the bottom of the dialog, click Add | Field value filter.
  4. Select the ‘Result Code’ summary and select the Status Code Names (ISA-FTMG) alias.
  5. On the toolbar, search for Authorization, and check the following two items:
    • The server requires authorization to fulfill the request. Access to the Web Proxy filter is denied.
    • The server requires authorization to fulfill the request. Access to the Web server is denied. Contact the server administrator.
  6. Ensure the ‘Exclude’ radio button is selected and click OK.

If you decide that you don’t care about seeing ANY unauthenticated traffic in your reports, you can always simply filter out the Anonymous user from your reports.

To do this:

  1. Go to the Reports tab and select the report you want to filter (such as your Organization report)
  2. Click ‘Edit Template’, then click ‘Template Properties’.
  3. In the filter section at the bottom of the dialog, click Add | Field value filter.
  4. Select the ‘Username’ summary.
  5. On the toolbar, click Add and type ‘anonymous’. Click OK.
  6. Ensure the Exclude radio button is selected and click OK.

Hopefully this article improves your understanding of the ‘anonymous’ user, and gives you some actions to take for your specific reporting situation.

If you have any questions, please leave a comment below.

Scott

Co-founder and Chief Product Officer at Fastvue.co (WebSpy's Parent Company)
I’m a co-founder, software product designer, web developer, and UX guy from Perth, Western Australia currently living in Bellevue, Washington USA.

5 Responses

  1. Will Smothers says:

    This is a great article and the templates are really useful. However, when I go to follow the instructions on filtering out the anonymous information from the reports, my dropdown under the Filter Values screen does not have a ‘Result Code’. The only options I have are:
    Date
    Day Of Week
    Day of Year
    Hour
    MOnth
    Site Domain
    Site Extension
    Site Keywords
    Site Name
    Site Port
    Site Profile
    Site Protocol
    Site Query
    Site Resource
    Site Top Level Domain
    Site URL
    User
    Username
    Week Of Year
    Year

    I am running version 2.2.0.68 and I have installed the packages for Reporting and Aliases for TMG 2010. Any suggestions?

  2. Will Smothers says:

    A little more information. It appears that my default reports are all listed as “All Web Schemas” and not “Forefront Threat Management Gateway 2010 Web” for the Schema type. This means that I am unable to select that value for the web schema reports. We are only analyzing and reporting on TMG, so, to fix this on the default reports, I will have to follow option two and just exclude the anonymous user from the reports…

  3. Stefanie says:

    The best way to get the templates from All Web Schemas into Forefront Threat Management Gateway 2010 Web schema is – go to Reports and select/highlight the template, then click the Duplicate Template link on the left. Now you can set the schema to FTMG, and this will open up the template to all of the fields from FTMG log files, not just the base set of fields.

  4. Roberto says:

    Do you have the same reports for ISA Server 2006 logs?

    Thank you,
    Roberto

  5. Stefanie says:

    Hi Roberto,
    Thanks for your comment. I’ve updated the zip file to include an ISA 2006 version of the template.
    Regards,
    Stefanie

Leave a Response

You must be logged in to post a comment.