Forget volume and variety, focus on velocity (Part 1)

When you think of Big Data, you may imagine the billions of rows and petabytes of data many companies are struggling to manage and process on a regular basis. You may also think about the challenges of handling diverse unstructured data such as audio, video, image and text-based files coming from an ever-increasing number of sources. In terms of the three V’s of Big Data, the volume and variety aspects of Big Data receive the lion’s share of attention. However, you should consider taking a closer look at the velocity dimension of Big Data—it may have a bigger impact on your business than you think.

In terms of velocity and Big Data, it’s easy to fixate on the increased speed in which data is pouring into most organizations today, especially from “firehose” data sources such as social media. However, velocity also underscores the need to process the data quickly and, most importantly, use it at a faster rate than ever before. These velocity-related challenges are typically viewed as technical ones, but there’s often more to it than just technology. People, process and cultural limitations can hold your company back from a speed and agility perspective—no matter how fast you collect and process data.

Data is said to age like wine—meaning the longer it’s kept, the more insights you’ll be able to glean from it. While this may be true for some forms of data, this analogy doesn’t apply to all situations. Many types of data have a limited shelf-life where their value can erode with time—in some cases, very quickly. For example, in retail it’s better to know which products are out-of-stock in terms of seconds or minutes rather than days or weeks. The more quickly a retailer can restock its products, the faster it can return to generating product sales.

Tweet this

Using real-time alerting, Walmart was able to identify a particular Halloween novelty cookie was popular in most of its stores—except two locations where it wasn’t selling at all. A quick investigation at those two locations revealed a simple stocking oversight meant the cookies weren’t yet on the store shelves. If Walmart discovered this stocking problem after Halloween, the value of this insight would have already vanished. Data velocity doesn’t just apply to the retail industry—it can apply to many diverse business models and functions.

High-velocity decision making

If your organization is still grappling with how to become more data-driven, the thought of operating at an even faster pace with data may be disconcerting and intimidating—especially when it comes to decision making. Traditionally, business decision makers have been accustomed to waiting days, weeks or even months to have ample information before they can make a high-quality decision based on past business performance. For fast-paced organizations like Amazon, the traditional approach to decision-making is far too slow.

Tweet this

In CEO Jeff Bezos’ recent letter to Amazon shareholders, he emphasized how “speed matters in business” and focused on the importance of “high-velocity decision making.” Bezos suggested, “Most decisions should probably be made with somewhere around 70% of the information you wish you had. If you wait for 90%, in most cases, you’re probably being slow.” Nobody wants to make bad decisions; however, constantly waiting for near-perfect information can lead to atrophy and missed opportunities. For Bezos, it is critical that companies are good at rapidly recognizing and correcting bad decisions. “If you’re good at course correcting, being wrong may be less costly than you think, whereas being slow is going to be expensive for sure.”

**This article was originally published on Forbes.com on June 28, 2017.