Less than a year back, the Twitter engineering team announced their move from Apache Storm to Heron for faster real time analysis of the data the platform gets every day. Last night Twitter open sourced Heron. Both Storm and Heron are distributed stream computation systems.
Everything that happens anywhere in the world finds a mention on Twitter. Considering the massive amount of data generated every day and the amazing job Twitter does of processing this data, analyzing trends and providing relevant details to users, all in real time, we can say Heron has been pretty successful. And with its open sourcing, this can only get better.
Twitter has partnered with Microsoft among other Fortune 500 companies for Heron. A set of startups is also working with Twitter and using Heron for other real-time use cases including fraud detection, IoT apps, virtual reality and augmented reality, social media data analysis.
This is a giant step ahead in the field of Big Data and data streaming & analysis - it will enable organizations of any size to deploy real-time data analysis systems at Twitter’s scale.
We are excited to see how Heron will impact Big Data streaming solutions and have also started working on a few ideas of our own around Heron.
If you have an idea which you think is the perfect fit for this level of real time data analysis, get in touch and we would be happy to help you with a solution.
If you are a developer interested in getting dirty with Heron, you can find more information here
