I’ve had some interesting conversations with my colleagues Christine Livingston and Arvind Murali. Christine knows AI. Arvind knows data, especially big data. As we spoke it became obvious that the two are deeply interconnected. Innovation today seems to be driven by data. AI relies on data. Just implementing any kind of artificial intelligence means you need two data sets:
- A training data set
- A testing data set
The Future of Big Data
With some guidance, you can craft a data platform that is right for your organization’s needs and gets the most return from your data capital.
Even more, both spoke about best practices and the overlap was kind of funny.
Take some of these best practices:
- Align to the business. Neither a big data lake nor a machine intelligence can justify itself based on being cool. Most companies make investments based on perceived value
- Start small. Both of these efforts require a journey. That journey needs a first step, first mile, and a first milestone. The key is to start
- Commitment. Both Big Data and AI represent continuous effort. You can’t just do one little project and be done. You just won’t realize value from that. That’s like rolling out something new to just 10% of your stores and stopping there.
- Trust. Arvind would call this data governance. Recently we’ve seen a number of our clients move from, “Big Data is really cool, we should stick a bunch of data in there.” to “Which of these fields can I trust? Where did this data come from?” In other words, some of the best practices from the data warehouse era continue on in our big data era. In the same vein, AI needs trustworthy data on which to train and evolve. Without it, any AI initiative will end in failure
- Data has value: AI is just the realization of that value. It’s not the only realization. API Management is another side of that coin. But the value needs to be unlocked. Big Data wants to get the data in a form where you can begin to gain insights. However, that’s just the first step in that journey.
But you get the idea. AI cannot realize its value without AI. Big Data is just a lot of data sitting there. AI is one key set of tools that allow Big Data investment to be realized.
