Categories: Internet

Colossus: The Beast That Powers Google

When Google launched Google File System (GFS), it was considered a revolution for the web. GFS helped Google use the hoard of machines at its data center as a single unit by well synchronizing their functions. As Google added huge amounts of data to its servers, it was evenly distributed across all machines through GFS. And GFS would create a new search index on regular basis.


So popular was GFS that soon other web giants such as Yahoo, Facebook and others built their own version of it. Google released research papers detailing how GFS works and soon, which led to an open source platform called Hadoop built along the same lines.

Move to Colossus
However, Google has been evolving and recently, the company devised a new way of significantly improving its foundation. It is basically a revamped file system and is called Colossus. Currently, more or less all of Google’s products are based on Colossus. From Gmail to Google Docs and YouTube, all these services run on top of Colossus.

So what made Google move from GFS to Colossus and why is Colossus significant? GFS was more suited for batch operations in which first the changes occurred to the whole system in the background and eventually, those changes were applied to the actual system. With Colossus, this has changed. Colossus is more suited to real-time operations. Colossus makes use of a new search infrastructure called ‘Caffeine’ which enables Google to update its search index in real-time, rather than first do it in the background and then apply to live system.

Another very important feature of Colossus is that whereas in GFS, there was only one master node, in Colossus there are many. So, for instance, if the node went down in GFS, the whole system would go down temporarily. This is not the case in Colossus where multiple master nodes operate at the same time.

Naturally, others on the web know of this transition by Google and they also know that Colossus is far more useful than GFS. The result is that a number of changes have already been made to the open-source Hadoop to make it look more like Colossus. Hadoop developers are actively working to bring concepts of multiple nodes to Hadoop. And, the framework’s adoption is also growing which now includes two more tech giants – Twitter and eBay.

So in a way, Google’s Colossus is driving innovation all across the web.

Courtesy: Wired

Share
Published by
Salman

Recent Posts

How to Select a Facebook Legacy Contact Who Will Inherit Your Facebook Account

You Facebook Legacy Contact cannot log into your account, read your messages, remove any of your friends or make new…

2 weeks ago

CleanseBot – World’s First Bacteria Killing Robot

CleanseBot is a smart robot with artificial intelligence and 18 sensors built in. It uses FOUR UV-C lamps to kill…

3 weeks ago

Top Apps to Watch Live Sports From Your Smartphone

With these apps to watch Live Sports, you will get full coverage of NFL, NBA, NCAA, MLB, NHL, English Premier…

3 weeks ago

Ninebot MAX: Ultimate Electric Scooter by Segway

The Ninebot by Segway KickScooter ES2 speeds up to 25 km/h, has front and rear wheel shock absorption and solid…

3 weeks ago

Best Google Cardboard Apps That You Would Love

Using the Google Cardboard Platform, users can either build their own Headset from simple, low-cost components using specifications published by…

4 weeks ago

How to Turn Off Amber Alerts on Android Phones

The WEA system is used to warn the public about dangerous weather, missing children, and other critical situations through alerts…

4 weeks ago