website statistics

Today at 9:47 UTC CloudFlare went down due to a wrong protocol rule in all their edge router, made by Juniper. Which took down all of their client's site. Here we have detail story.

What is the next Gadget you want?

View Results

Loading ... Loading ...
1 Star2 Stars3 Stars4 Stars5 Stars (Rate This)
Loading ... Loading ...

Popular DNS service provider CouldFlare went down today due to an issue with Flowspec protocol rule update in their edge Juniper router. Which took their whole system down for about an hour and almost all of their 785,000 customer (Including 4chan, Wikileaks, Metallica.com & our TheTechJournal) went down with it. CloudFlare tried to act quick, now everything is operational. CloudFlare CEO Matthew Prince explained the whole thing, keep reading to understand more about the issue.


CloudFlare

What Happened With CloudFlare Today?

CloudFlare is not only a DNS service, but it also provide DNS level security, caching service, which helps website to fight with spam to DDoS attack, as well as serving content faster. Here at TTJ we use it as DNS service in some lever of our cloud system. So we are entirely depending on them. But the whole CloudFlare system went down today at around 9:47 UTC (1:47 AM in California). And all of the site its serving went down with it. The number of website use their service in some level is quite high. About 785,000 websites went down use to this issue. Due to the network-wide router fails in their edge network all of their 23 data center around the world was disconnected from internet. Some of the router came back online within few min but as all network traffic were directed to those available few, they also went down immediately after reboot. CloudFlare operation team tried to act fast and as their 23 data center is distributed in 14 countries it took them more 30 minute to get some one in all data center to manually hard reboot all router. Then they figured out what went wrong and which rule caused the issue and reverted the rule change. Within an hour whole network started to act in full power.

How All This Happened?

According to CloudFlare CEO Matthew Prince, it all started from a DDoS attack on one of their client’s site. They have a automated attack profiling system, which detected a very unusual size of data packets of between 99,971 and 99,985 bytes long, but the largest packets sent across the Internet are typically in the 1,500-byte range. And they had a limit of maximum packet size set to 4,470 on their entire network. So when the profiler distinguish the attack patter, their operation team wrote a rule like this below
CloudFlare Rule

Flowspec accepted the rule and relayed it to their edge network, and caused all Juniper router consume all their RAM until they crashed.

Overall:

All site seems running smoothly now. Most of the site came back online within an hour. As far in our experience we seen a downtime of around 15 minute for TheTechJournal. CloudFlare promised to look more deep into this matter so they could avoid this in future. They will be giving credit back to all their paid customer.

cloudflare_outage

Now it all seems good, and they way CloudFlare reacted is nice. But still I personally feel DNS is a very important for internet, it should be more redundant. Most of the big names relies on service like CloudFlare, maybe we should have some sort of backup or DNS lever load balancing to avoid this type of incident.

Buy Cheapest Related Product From Amazon.com


Texas Declares War On Drones

Rumor Says T-Mobile May Move To No-Contact Model
You can also press the left/right arrow key on your keyboard to go to previous/next post
  On March 4, 2013(1 year, 1 month ago.)
  • Jim

    It’s physically impossible to have IPv4 packets between 99,971 and 99,985 bytes, in fact anything over the 16-bit word limit of 65535 bytes is impossible. Their software that generates FlowSpec rules malfunctioned and they’re blaming the vendor.


Recent Search

Recent Tutorials

At the beginning level, many people don't know how to use Gmail properly. For them, a here's a helpful guideline.
Skype now allows you to call directly from your Outlook account. Find the step-by-step installation process from this tutorial.
This tutorial helps you quickly resolve four of the most common problems that occur with Windows XP.
After releasing Look Back, many people didn't like it. So Facebook planned to add an Edit tool to the feature and now you can edit your Facebook Look Back video.
Do you want to record Skype calls but don't know how to do that? Then this guide is absolutely for you. Get inside the article for details.
Do you know that many companies can track you on Facebook? Do you want to know who they are? Do you want to block them?
So you've been planning on replacing your old hard drive with the new drive? This tutorial helps you with a few simple steps.
Bitcoin is a digital currency that has gained extraordinary momentum in the last few months. This tutorial helps you get started with the basics.
Android lockscreen can be secured in a number of ways, some of which are described in this brief tutorial.
This review of iSkysoft Video Converter tells you everything you want to know about this Win 8/7 compatible video/DVD tool. Check the below to learn more details.
Close You Have To Login
User:
Pass:
Login With »Login With TwitterLogin With Facebook