The next morning I woke to find several more messages from my VPS host, each with a higher and more significant excess transfer than the last. At this point it occurred to me that it was unusual for my VPS to reach its quota, let alone exceed it. The excess transfer was now enough that it was going to incur significant cost, so I set about investigating the cause.

I downloaded some firewall logs for the previous few days from the server and imported them into Vantage. The first place I looked was in an analysis at the “Source Address” summary, to see where the activity was coming from. What I found was a single host with a disproportionately larger amount of transferred data than the other addresses listed, so I drilled down to the “Destination Port” summary for this source address to see what services it was accessing. I found that all the traffic was going to port 53 – my DNS. More accurately, the large amount of data was going from my DNS to the source address. Drilling down to the “Individual records” view then showed that my server was providing a large response to a small DNS request from that source address – about 20 times per second.

Curious about why this single machine somewhere on the Internet was bombarding my server with small DNS requests at such a high rate, I set my server’s firewall to deny packets from that address and began searching around online for any information.

I quickly found out that I hadn’t configured my DNS properly, and it was set to allow recursive requests, meaning that if a request came in for a domain my server wasn’t authoritative for, it would then forward the request to another DNS that could answer, or given a blank request it would respond with the full list of root servers. Running tcpdump on the VPS revealed that every request coming in was blank, and my server was responding with the full list of root servers for each request.

It still seemed odd that a server would be constantly sending small requests to my server and receiving large responses. Then it dawned on me; I was looking at a Distributed Reflected Denial of Service (DRDoS) attack. The source address in all the requests I had looked at was forged by the attackers, so that my server – and many other servers out there also receiving the requests – would send their responses to the forged source address in an attempt to flood its connection. The source address in my firewall logs was the target of the attack. I found more information about this specific type of attack here.

Having disabled recursion on my DNS, my server’s contribution to the attack was significantly reduced. However, my server was now responding with a much smaller “request denied” packet for each incoming request. I wanted some way of preventing my DNS from responding at all, so again I headed out to the Internet to see what I could find.

I discovered a package called “fail2ban”, which dynamically updates your firewall rules to block addresses that are abusing your server’s services. I installed it using this guide, and immediately my bandwidth usage dropped off as it blocked further DNS requests. Even now the requests are still flooding in, but now my VPS contributes only a handful of packets towards the attack instead of the previous millions per day.

Unfortunately, the WebSpy Twitter account fell victim to a phishing scam, and as a result sent phishing spam to all our Twitter followers. We are embarrassed by the incident and we apologize to all of our followers, especially the ones that clicked the link in the DM and were caught by the phishing scam themselves.

Here’s a rundown of the event in the hope that it will help others know what to look out for.

What Happened?

The phishing scam works like this:

1. You receive a strange yet intriguing Direct Message from someone you follow and likely trust. This is the key element to the scams success.
2. The DM contains a link using a shortened URL such as dwarfurl.com/blah. In our case, most of them were using dwarfurl.com, wapurl.co.uk, and 3.ly
3. You click the link and get taken to what appears to be the Twitter login page. But if you look at the URL it is actually something like blogs.videos.dsfasdc.com or videos.twitter.dsfasdc.com. Checking the URL is the key to making sure the scam doesn’t get you too!
4. You enter your Twitter login details. Reports of what happens after this login page vary. You may see the Twitter fail whale, or a blank page, or a random blog.
5. Now that the phishing site has your login details, the same Direct Messages is sent to all your Twitter contacts.
6. You eventually discover what happened. You feel like a violated idiot and start scrambling to fix everything.

What to do if it happens to you

If the above sounds familiar, you need to login to Twitter right now and change your password to make sure the phishing site can no longer access your account. You also need to go to the Connections tab and disable any third party applications that look suspicious. You’ll then need to update the credentials in all the twitter clients, website/blog plug-ins, and anything else that may be using your old Twitter credentials.

Fortunately, we were still able to login to our Twitter account and change our password and disable third party connections. Thankfully there were not any new suspicious connections that we needed to worry about.

Lessons Learned

Now that we’ve fixed everything and regained control of our Twitter account, it’s good to sit back and reflect on what just happened and how to avoid it in the future.

You’ve probably heard all of this before. We had too. But it takes an incident like this to really think about and address any shortfalls in your own organization. Some of our followers were also caught out by the scam and these are people that are in the tech industry and generally know about these sorts of scams. We were definitely surprised that we fell for it! So take a moment of your time to imagine your own Twitter account was compromised in the same way, then imagine all the possible ways it could have happened. Now go and take every precaution to ensure it doesn’t happen.

Having now been through it, here are some tips to help you avoid the same fate in the future.

1. Just because a Direct Message comes from someone you trust, does not mean it is trustworthy. Always use caution!
2. Educate your employees – especially those that know your company’s Twitter credentials. The main goal you want to achieve here is getting your employees into the habit of glancing at the URL in the address bar of their browser before entering ANY login details. We used our own log analysis software (Vantage) to find out who ended up on the websites in question, and then spoke to them directly to ensure they understood what to look out for.
3. Use a Twitter application that can display the actual URL behind a shortened URL before clicking on the link. For TweetDeck users, go to Settings | General, and check ‘Show preview information for short URLs’. Please note, however that this function only works for a few specific URL shortening services.
4. If you’re using the Twitter web page directly, use a browser and plug-in that can expand shortened URLs such as Mozilla Firefox with Long URL Please.
5. Use a browser with integrated anti-phishing security (such as Firefox or Google Chrome) and keep it up to date, or ensure you have good third party anti-phishing / anti-malware software installed.
6. As always, keep your security software and OS up to date.

Our friends at Sophos also have some good information about the scam that you may like to read: http://www.sophos.com/blogs/sophoslabs/?p=7366

Sorry!

An event like this makes you realize how important Twitter is to the overall public perception of a company. Our followers trust us to deliver relevant and useful content about our key areas of expertise – log file analysis and reporting. We spend a large amount of effort researching and writing content to ensure our tweets provide our followers with a good source of information. Having a breach like this certainly degrades this public perception that we work so hard at trying to maintain.

I would therefore like to thank all our followers who have kept with us and not clicked the ‘Unfollow’ button. Now that everything is under control again we will continue to bring you the best content we can provide about the log analysis and surrounding industries.

Once again, many many apologies to all of our followers, especially those that were affected.

Here’s a short summary of the main topics discussed during the event. For more information please follow the related links at the bottom of the page.

Event Summary

Sophos’s Asia Pacific Managing Director initiated the event by discussing organized cyber crimes. He highlighted that online organized crime rates are escalating rapidly. Online criminals are becoming increasingly sophisticated in the techniques they use to try and scam private people and businesses alike. Unfortunately their techniques are evolving much faster than legislation and community awareness, estimated to be at least 12 months behind.

Police Cyber Crime Unit

The representative from WA’s Cyber Crime Unit expanded on the safety of children, and how Internet predators are becoming increasingly Internet savvy and often avoid getting caught by engaging in their illegal activities at work. Nevertheless the audience was happy to learn about a recent case where the Cyber Crime Unit successfully tracked down an online predator, who had managed to stay anonymous for a long time by exclusively using his employer’s Internet resources. He worked for a very large organization, but thanks to the employer’s internal security and monitoring system he was identified before he had the chance to commit further crimes.

Hiding in a Wireless Hotspot

Wireless hotspots, today free at many airports, coffee shops and fast food chains, was another concern raised by the police’s Cyber Crime Unit. More often than not, the companies providing this free access to customers do not have a system in place to monitor and alert on any inappropriate or illegal activities. When this is the case it is virtually impossible to prevent predators using these networks to stay anonymous. Unfortunately, legislation, or public outcry, to address the issue is not likely to occur until an illegal activity, enabled by the anonymous use of wireless hot spots, takes place and receives media attention.

At School

The representative from the educational sector continued to discuss online safety related to children and students. He highlighted that security system at schools and universities are essential, but not always enough. On many occasions students bypass the school’s firewall by using virus ridden public proxies to access blocked sites. Even students with studious intent occasionally use public proxies to access legitimate sites that have been blocked thanks to a “block worthy” word in a corporate blog, or something of similar virtuousness.

The importance of educating children about the dangers of social networking was also emphasized. Children are often overconfident in their abilities to spot a predator among their peers. However, in reality, they do not fully comprehend the psychological techniques used by online prowlers to gain their trust.

Best Practices

To sum up, the event focused the changing landscape of the internet and internet security. In our internet dependent world everyone is at risk, whether at work, at home, or at school. The best practices, when working towards a safer e-environment, keeps changing but the proactive theme throughout the event emphasized a combination of security systems, system monitoring, education of workforce (parents, teachers, students), and an increased involvement from all levels within organizations and community.

Related Links:
Western Australia Internet Association
Australian Communications and Media Authority
Wise up to IT
Cyber Smart Kids
Stay Safe
Think U Know
Virtual Global Task Force
Sophos