How Yahoo! Keeps Your Inbox Spic and Span

We know how important email is to our users because we’re Yahoo! Mail users, too. We use it to share photos of holidays, birthdays, communicate with our families and a whole host of other activities as well. And because we’re Yahoo! Mail users, too, we know that we only want to see emails from friends and family in our inboxes, from the people who matter the most… not from spammers and phishers.

In light of this, with the roll out of the new Yahoo! Mail last May, we recently launched more advanced anti-phishing defenses and enhanced spam protection to help protect your inbox. And we’re happy to report its success. The Yahoo! team put together new technology that’s now blocking nearly 550 billion spam messages from hitting your inboxes each month – that’s approximately 1,800 emails for every Yahoo! Mail user.We think the technology we’re using to keep your mailboxes safe is quite awesome and we wanted to show you how we do it, so today we’re going live with the Yahoo! Mail Visualization Project – a view of what no one has seen before using live data…how we use cloud computing and Apache Hadoop technology to filter spam and re-route email for the 300 million mail users we have across the globe.

We won’t bore you with the techy details in writing because it’s much more compelling to see…

So why is it important for us to reveal how Yahoo! Mail works? Because we’re proud to show the science and technology powering the products you use every day. Through the visualization project, we can demonstrate the technology behind the Yahoo! Network and show the impact that the tech has on you, the consumers of Yahoo!’s great technology.

As a quick example, the anti-spam protection we mentioned earlier uses Hadoop to aggregate anonymous data from the billions of emails sent and received each day and as a result has helped us reduce spam reports by 65%. Our technology analyzes all this anonymous data (with the help of Hadoop) and identifies spam patterns so we can then use really intelligent algorithms to predict future email patterns that will differentiate “good” and “bad” senders.

To give some perspective on how powerful our computing technology is, think about these fun facts:
• For every loving message from mom, Yahoo! Mail blocks 4 spam messages from the people she warned you about.
• In the time it takes you to read this, our Hadoop software (the brain behind Yahoo! Mail) has learned from more than 4 million user actions.
• In the time it takes sunlight to reach earth, Hadoop processes 332 million user actions, helping reduce spam in Yahoo! Mail.
• Hadoop processes 1 terabyte of data (equivalent to 4,000 years of continuous typing) in the time it takes the fastest human to run a mile.
• We add over 70 million Yahoo! Mail accounts per year – that’s the combined population of London, NYC, Paris and Tokyo.
With that we’ll leave you with the link to the Mail Visualization project. It’s super interactive and ready for you to check it out for yourself: http://visualize.yahoo.com/