Value Creation Through Big data Analytics at Bombay Stock Exchange

Abstract

Ashishkumar Chauhan, the MD & CEO of the Bombay Stock Exchange (BSE) was reflecting on how to leverage the Hadoop Big Data framework that BSE had implemented. The BSE, Asia’s oldest stock exchange had historically used proprietary technologies in their operations, and the shift towards open-source based big data infrastructure was path-breaking. The priority of the BSE was always to be fault-tolerant and never lose any data. However, the BSE was facing issues with their existing data warehouse platform. Data was growing at a rate of 350 GB per day with around 300 million orders and 30 million trades. Owing to this large volume of data, execution of queries, loading of data, and scalability were becoming a concern with the existing data warehouse setup. Around the end of 2013, the BSE had partnered with technology firm Datametica Solutions to create the enterprise data hub. This exercise helped the BSE reduce its cost by 20 times with an overall savings of INR 20 million annually. Later, in 2016 the BSE leveraged the Big Data platform for market surveillance and created the social media rumour detection analytical module. The case delineates how an organisation migrated from a traditional data warehouse setup to the new Hadoop based Big Data platform. The case also highlights how the organisation utilized the new infrastructure to create the social media rumour detection module. Some questions remained unanswered. Many users in the organization were yet to adopt the Big Data framework for their processes. How could BSE convince more people to use the platform? How could it convert short-term wins to a long-standing competitive advantage? How could the BSE become an analytical innovator through its use of Big Data?