Fun with Google Safe Browsing

You probably have encountered it, you want to go to a website and you get a red page to say that something is wrong with the site and malware has been found on it.

Google Safe Browsing is part of your standard Mozilla Firefox and Google Chrome browser. Google isn't the only one playing this game. Microsoft has its SmartScreen filter and most major AV-solutions have something similar.

This is all fun but what if you are interested as a website owner if you have been flagged? Well actually you can get this report. If you surf to http://www.google.com/safebrowsing/diagnostic?site= you get a nice overview of what was detected for that website.

An example:
http://www.google.com/safebrowsing/diagnostic?site=google.com

It tells me that for the domain google.com in the last 90 days 903341 pages got tested:

• 484 drive-by-downloads
• 252 trojans
• 103 exploits
• 46 scripting exploits

So as you see this has some value in risk management. Personally I use this technique for information gathering when doing incident handling. You can use it in a risk management to monitor your own website and those of who you do business with in a rather cheap way.

Another cool little trick is that you can get more information on an Autonomous System (AS). 

http://www.google.com/safebrowsing/diagnostic?site=AS:15169

If you are the owner of the AS, like my current employer is the owner of the Belnet AS with the number 2611, Google has a nice little tool to generate alerts for your incident handlers

http://www.google.com/safebrowsing/alerts/

Some of us don't own AS systems. Thus I want to share with you one last toy for website owners. Enter "Fetch Like Google.  "Fetch like Google" allows you to fetch up to 500 URLs a week for the sites you own and can be very handy to figure out if the Googlebot still sees your website as infected.

More info can be found at https://support.google.com/webmasters/answer/158587?hl=en.

Some people have trouble with https but I haven't had that experience personally. I found on this video on youtube which Google's answer to people having trouble. Basically it works for Google too.

Blocking Phishing Sites

The other day I got the question how the popular browsers block you from going to malicious websites. Interesting, that you had to connect back to a database seems logical but the actual inner workings were a mystery to me so I looked it up.

Firefox, Safari and Chrome all use the same technology called the Google Safe Browsing API. Microsoft Internet Explorer uses a technology called SmartScreen Filter.

Google Safe Browsing API is a complete website with the technical details. Basically it works like this:

You type in the website you want to visit in your browser:
http://www.somehost.com/path/page.html?args

Your browser will need to check the following paths against the database:

http://www.somehost.com/path/page.html?args

http://www.somehost.com/path/page.html

http://www.somehost.com/path/

http://www.somehost.com/

http://somehost.com/path/page.html?args

http://somehost.com/path/page.html

http://somehost.com/path/

http://somehost.com/

As you can see this is quite a lot of strings and looking up strings in a database is usually a slow thing. The trick used here is that a hash is calculated and a 4-byte prefix is sent to the database. In case of a match the database will return all the matches and the client can then calculate the full hash to see if it is in the returned list. 

If the full hash matches the end user is informed about this else your page gets loaded.

As sources for their database they mention Antiphishing.org en Stopbadware.org but you can be pretty sure that it would not be the only sources.

At the Windows Live blog I could find more information on how the Microsoft SmartScreen Filter works. The example Microsoft gives is the following:

Let's say a malicious website is hosted at canada-pharmacy.us. This IP gets marked in the database as "bad", besides the URL the IP address is marked as "bad" too. SmartScreen will generalize this to IP's in the neighborhood. This is done based on ASN blocks, the way IP addresses are split up by owner.

DNS server rating are also part of the SmartScreen technology. The DNS servers that seems to know just a little too much about abusive domains is given a lower rating according to the blog. Unfortunately for a techie this is a meaningless description.

SmartScreens telemetry comes from reports from end users, third parties, traffic from URLs showing up in e-mails and logs. The feeds are fed into machine learning algorithms to either flag or pass a URL. When the algorithm is in doubt, the information is given to an analyst who will do the necessary research.

As a conclusion I would say make sure you have a browser that has such a technology enabled, it is not perfect but it is free and better then having nothing.