A growing big data acceleration market is bridging the disconnect between applications and CPUs, adding ASICs, FPGAs, and GPUs for improved performance.
Analytics Needs More Than CPUs
At the risk of stating the obvious, speed matters. Much of technology innovation is around acceleration, making things faster. Bigstream’s business is certainly built around this principle.
Application providers have speed and performance at the top of their priority list, and each version finds new ways to execute more efficiently and gain speed. Hardware development is also largely about speed and has produced amazing performance gains over time. These two industries develop and innovate in their own ways, abstracting away the complexity of outside parts.
Despite the huge gains these two siloed innovation paths have brought, they have opened the door for an additional market to develop to accelerate the applications with greater context awareness of the underlying hardware and computing.
This market is yet to be formally defined, and there will be many opinions on what to include, and even what to label the market. Regardless, we see a range of companies—from hyperscale cloud providers to small disruptors—bridging this same gap which we argue comprises a big data acceleration market. The need is consistent, though companies address it in varied ways, including:
- Developing dedicated application specific integrated circuits (ASICs) for artificial intelligence (AI) models
- Redeploying graphics processing units (GPUs) for machine learning applications
- Programming field programmable gate arrays (FPGAs) for data warehouses
In this blog article, we identify a few diverse examples, yet make the case that big data acceleration is a single market with the potential to deliver enormous value.
Platform Abstraction Enables Centers of Innovation: The Virtualization Model
One way to think about this developing big data acceleration market is with an analogy to server virtualization. In that market dynamic, applications and operating systems had their own innovation cycle separate from server innovation, though each needed to consider and accommodate the other. Advances on a given operating system needed to acknowledge the constraints of the available servers of the day. When VMware popularized virtualization, it enabled the further abstraction of the two, empowering greater innovation in the two other industries.
VMware and virtualization ultimately made for more effective use of both the applications and operating systems, as well as the underlying servers. In the same way, this big data acceleration layer helps the applications and the underlying hardware to continue advancing, while enabling the applications to get the most out of the available hardware.
Business Imperatives Drive Technology Innovation
For big data, speed for its own sake is not the goal. More and more, big data applications are mission critical, and the next order of magnitude of speed gains can transform what is possible. These improvements yield faster time to value, a fuller look at available data and enriched analytics, as well as the efficiency to reduce total cost of ownership. Let's look at some examples.
Big Data Acceleration Examples
Prominent big data platforms include data lakes, data warehouses, transactional databases, NoSQL databases, and more. Specific examples include:
- Apache Spark
- Apache Hadoop
- Amazon Redshift
- Azure Synapse
These platforms add performance through distributed computing across many nodes, but they still default to the general-purpose CPU as the computing infrastructure. The more businesses depend on these platforms to generate value, the more it is clear that the general-purpose CPU is an area ripe for improvement, and in many cases the weak link. More effective approaches include CPU-optimizing software and adding hardware acceleration like GPUs, FPGAs, and ASICs.
AI and ASICs
One area ripe for acceleration is ML and AI. The rapid advances in AI/ML come in large part due to advances in computing and memory and the growing ability to manage enormous data sets. More data helps produce more useful AI/ML applications which, in turn, generates the need for more data. This feedback loop ultimately requires vast computing requirements.
Here, both GPUs and ASICs have been popular approaches. Very large organizations have been able to dedicate resources to develop their own ASICs such as Google’s tensor processing units (TPUs) and Facebook’s ASICs for AI. It is worth developing these custom devices with such specific AI demand.
Spark Machine Learning and GPUs
Where the CPU is general-purpose, the ASIC is the other end of the spectrum—a chip designed and built for a specific, typically narrow, usage. Even for large companies like Google or Facebook, developing an ASIC is a significant time and financial commitment. More accessible is the GPU, which is closer to a CPU and can more easily be added into existing workflows.
While the GPU was created (and named) for its specialized ability to process graphics better than a CPU, this architecture has been tapped for broader use cases including deep learning model training. The “general purpose” GPU (GPGPU) is well-suited for highly parallelized workloads and has been a significant contributor to deep learning’s rapid adoption. With that in mind, NVIDIA, the leading GPU provider, has developed its RAPIDS software libraries to connect data science applications with GPUs. Specifically in 2020, it released the RAPIDS Accelerator for Apache Spark.
Spark is used in computationally intensive steps, including:
- Extract, transform, and load (ETL) operations
- Batch analytics
- Feature engineering for ML preparation
- Training for those ML models
GPUs have an advantage over CPUs in ML model training because they process matrix multiplication faster, parallelizing across multiple threads.
Amazon AQUA: The Data Warehouse and FPGAs
Amazon recently announced AQUA, a project that brings acceleration to its Redshift data warehouse via FPGAs. The data warehouse space is fiercely competitive and evolving fast, with legacy providers like Teradata trying to keep up with the rapid cloud adoption led by Redshift, Snowflake, and others. These data warehouse providers largely innovate in their own space and are less focused on the underlying compute infrastructure.
AWS covers both the data warehouse and much of the infrastructure, so it is addressing acceleration itself. With AQUA, AWS has identified a path to acceleration beyond the application itself, moving beyond the traditional CPU that underlies the data warehouse.
There is a growing list of big data acceleration companies and applications they support. At a certain large scale, ASICs make sense, and the accessibility of GPUs make them a tempting tool to deploy for many computational demands.
At Bigstream, we believe that the FPGA is an ideal data analytics acceleration tool because it strikes the right balance between the easier deployability of GPUs and the power of ASICs. Once initial programming is automated by acceleration software, FPGAs can be deployed to a wider range of operations than an ASIC—and with better performance, cost, and power profiles than a GPU.
For example, Bigstream has been able to accelerate the 100 TPC-DS (decision support) benchmark queries by an average of 6x with FPGAs, with 97 of the queries at least doubling speed, whereas GPUs have a more limited impact. While GPUs do well with ML Training, FPGAs provide more overall acceleration opportunity for Spark workloads. Bigstream has thus focused on FPGAs for some of the most processor-demanding parts of Spark.
FPGAs are gaining popularity among third-party acceleration software companies. Reniac uses FPGAs to accelerate Apache Cassandra, a popular distributed NoSQL database system. Like RAPIDS and TPUs, Mipsology addresses neural networks, but with software to let users tap into FPGAs. TigerGraph uses FPGAs to accelerate its graph database solution.
The Market Opportunity: Accelerate Value and Rationalize Spending Growth
What kind of value can this big data acceleration segment provide? While Marc Andreessen’s “software is eating the world” proclamation remains true today, every software application still runs on a chip and, with few exceptions, both cost money. The early days of on-prem big data were a mix of mostly free open-source software and commodity servers. As the market has matured, customers have spent large sums on both software and hardware.
One way to get a sense of the mix of big data hardware and software expenses today is to look at pricing for managed Spark services like Amazon EMR. The EC2 compute costs are typically 4 or more times the EMR software costs. Put another way, software is just 20 percent of an EMR user’s Spark budget. For third-party software like Databricks or Cloudera, software comprises a larger share, but still typically less than the computing costs.
The big data acceleration industry’s value is very quantifiable here. Existing acceleration technologies commonly yield 3x, 10x, or sometimes over 30x performance gains. But even a modest, across-the-board 20 percent acceleration would deliver huge value and TCO reduction based on multibillion dollar big data computing expenditures.
In its recent “Big Ideas 2021" publication, ARK predicts that “accelerators, such as GPUs, TPUs, and FPGAs” will become a $41 billion industry in the next ten years, even surpassing CPUs. Big data analytics and AI are among the demanding computing tasks driving this shift, and they clearly highlight why companies are building the software to bridge applications to this advanced hardware.
Bigstream: Spark, FPGAs, SmartSSDs, and Software with a View to the Future
Bigstream is excited to be one of the early drivers of this big data acceleration space, and excited to see the growth in applications and interest. Today our available products accelerate Apache Spark with FPGAs, SmartSSDs, and multicore CPUs with software. While that reflects a decision of where we can maximize our initial impact, our innovation reflects a wider view of the space. As a bridging technology, much of our focus is on the two sides of that bridge—Spark and processing.
While we don’t aim to cover every acceleration possibility, we look forward to bringing accelerators to other big data applications and other hardware platforms, partnering with the wider ecosystem and pursuing our own innovation.