Covering Disruptive Technology Powering Business in The Digital Age

Home > DTA news > News > Big Data: Hype or Substance?
Big Data: Hype or Substance?
December 8, 2017 News

Big Data is certainly the rage these days. As IT continues to evolve with the rise of big data analytics, deep learning, artificial intelligence, IoT, and digital transformation, organisations in every industry are going through a disruptive change. But how much of it is just hype? Can big data truly have a significant long-term impact on industries within ASEAN, or is the excitement starting to wane as we look to the next emerging technology?

Kamal Brar

To answer these questions, Big Community was given the opportunity to interview Kamal Brar, Vice President and General Manager of Hortonworks in APAC, who shared his insight on the subject of big data and analytics, why it’s so important and how it can impact or help drive innovation in the region.

The following is the full text of the interview:

Big Community: For a while Big Data was as much as hype as substance – is that still the case?

Kamal Brar: The arrival of the term Big data ushered in an era of high hopes and higher expectations. Early hype surrounding big data technologies promised a steady stream of real-time insight that would transform customer experience, marketing pitches, and operational efficiency. The reality is that, after all, every business is a data business and often times this data comes in high volumes. Transactions, interactions, and observations generated from sensors, the web and applications sum to big data. Organisations will need to become more data driven to compete in today’s market.

Big Community: Is Hadoop starting to be replaced by other technologies or does it have a long future?

Kamal Brar: Apache Hadoop continues to innovate at a rapid pace and the next generation of Hadoop is being built today. Some of the reasons organisations use Hadoop are its ability to store, manage and analyse vast amounts of structured and unstructured data quickly, reliably, flexibly and at low-cost. Additionally, it can handle a wide variety of workloads and is interoperable with a large number of technologies.

We do not see Hadoop being replaced as the value it brings to organisations is substantial. It provides businesses with the scalability, reliability, flexibility as well as affordable for business of any size to adopt it successfully. Technologies like Spark plug right into the Hadoop platform and provide another kind of processing engine on a vast common data repository.

Big Community: Is implementing a big data project built on Hortonworks resource, skill and time intensive?

Kamal Brar: Hortonworks is in the business of helping enterprises achieve their desired business outcomes with big data as effectively as possible. We’ve done it over 1000 times. Case studies here. As you can imagine, the resources and skills required as well as the time needed are dependent on the specific use case and business outcome that is desired. Naturally there are many factors involved. Generally speaking there is a need for more skills in the area of big data from a systems level, data and integration as well as data science and AI.

Big Community: Just how much data do you need to be working with to derive value from Hortonworks?

Kamal Brar: We see data projects ranging from a few terabytes to hundreds of petabytes. Sometimes a small amount of data can require complex parallel processing which is ideal for the Hadoop platform to process efficiently. Other times it can be the sheer scale of the data that requires the Hadoop framework for processing. The good news is that the platform works at any scale and size of data or any complexity of compute.

Big Community: Can you provide real world business examples of innovation that has been driven from Big Data?

Kamal Brar: There are definitely some great examples we can share in terms of how businesses have grown or innovated through the use of Big Data

Take for example a customer we have in the telecommunications industry – Pinsight. It uses verified, network-level data to fuel intelligent brand decisions. Its data management platform ingests over 60 terabytes of data coming from 65 million devices across multiple mobile network operators. Network-level data provides right-time advertising, capturing those with the highest propensity to respond.

This has tremendous business value for brands, as it helps them gain a much clearer understanding of how users travel throughout their day, and what type of people are interacting with their brand. This greatly aids important long-term decisions, such as determining where to build new store locations and place advertisements.

Another example, the healthcare industry, our customer Geisinger, one of the largest health service organisations in the United States, was able to utilise big data to make data quickly and easily available to caregivers. They needed to connect the system’s hospitals, 40 community practice sites, and the primary and specialty care physicians and extenders who serve patients throughout the Geisinger network.

Through our solutions, Geisinger now runs queries on its unstructured data to derive analytical insights. Clinicians and non-clinicians are able to search through 200 million patient records in seconds to find relevant conditions and medications, which helps them analyse the success of treatments, identify areas of improvement, and determine ways to save time and money for both patients and providers.

Another example is around predictive maintenance. Transport is a major concern and challenge for countries across ASEAN, including Singapore and Malaysia. Having a smart transport system is a vision that the national governments are pushing for.

How can big data play a role? Our customer, Metro Transit of St. Louis (MTL) leverages on big data to understand their fleet of buses better. MTL turned to Big Data analysis to better predict when a component on a particular bus will fail, allowing them to proactively service the bus prior to any component failures.

The results have been spectacular – the average time between bus failures has improved by a factor of five. MTL was able to run the buses for much longer, thereby increasing the return on investment in their fleet. Previously, buses were being retired after 35,000 miles per year at 12 years, but now MTL is able to continue using the buses up to 60,000 to 70,000 miles per year at 15 years. This is a 2x improvement on mileage and 30% increase in bus lifespan.

These improvements in vehicle maintenance have saved St Louis area taxpayers more than US$2.5 million per year.

Big Community: What are the reasons why Singapore businesses and government need to focus on Big Data?

Kamal Brar: The ASEAN region is rapidly moving towards a digitally-enabled economy, with the ASEAN ICE Masterplan 2020 (AIM2020). The plan encompasses eight strategic thrusts that focus on enabling an innovative, inclusive and integrated ASEAN community.

On this front, big data plays a central role in enabling innovation for ASEAN governments – starting with supporting open and big data application development.

Singapore is leading the world in many areas, and government and private initiatives such as smart cities make big data technology essential to these efforts. As a hub for many companies, increased efficiency and competitive advantage are core values that Singapore based companies gain from using big data technologies. In order to continue to lead they must adapt to use data to their advantage.

In addition to the rise of the digital disrupters that is changing the business landscape, businesses need to stay ahead of the competition in understanding what their customers want and have the ability to predict the customer journey and make smarter decisions using technology such as big data analytics.

-End of Interview-


Note from Big Community: Our take from this is that big data is definitely here to stay (but we knew that anyway!). The term has been tossed around frequently by those who are looking to cash in on the buzz. But if businesses truly understand how to effectively implement and harness the power of big data, they will discover that it offers substantial benefits for the whole organisation. The main issue is not whether it needs to be implemented, but how and where it is implemented. Perhaps there was a time when big data could be considered as “just hype”. However the dramatic impact that it has had on businesses across every industry worldwide means that it will soon become the new normal – albeit just a part of a bigger whole in the next step of the technological evolution.