If you noticed a slight hiatus in accessing Google on November 5, it was real and not just your internet. Google was indeed inaccessible for 30 minutes to a significant portion of the internet users, thanks to a faulty configuration by an Indonesian ISP. However, an engineer who doesn’t even work for Google, was able to identify it and had it rectified.
The problem started when Moratel, an Indonesian ISP, routed the traffic which was meant for Google to an incorrect address. Apparently, it was a result of an erroneous configuration on Moratel’s end.
What is rather intriguing is that the issue soon spread further. Given the fact that ISPs are usually inter-connected, routing configurations are shared often. And so, when Moratel configured the route to Google incorrectly, the false route propagated across other ISPs too.
A networking engineer at Cloudflare, Tom Paseka, was the first to notice the problem and discern the cause of it. Paseka was quick to contact a colleague at Moratel. According to him, “I contacted a colleague at Moratel to let him know what was going on. He was able to fix the problem at around 2:50 UTC / 6:50pm PST. Around 3 minutes later, routing returned to normal and Google’s services came back online.”
At its peak, the hiatus affected the ability of nearly 3 to 5 percent of Internet population to access Google. Paseka says that the incident is a reminder that even if a company is as huge as Google, it must have a vigilant team of engineers to keep their eyes on the web because factors outside of their control can affect them.