The advent of cloud computing and generative AI (genAI) has significantly enhanced the capabilities of data-driven businesses, offering powerful analytics and deep business insights. Cloud services are crucial in providing the infrastructure and tools needed for the development and deployment of genAI technologies. The accessibility of pre-trained models and software packages through the cloud has accelerated the integration of genAI into data analytics workflows. However, this progress has also led to increased data volumes and rising cloud infrastructure costs.
A recent 2024 State of Big Data Analytics report by SQream, a GPU-based big data platform, underscores the financial burden that cloud analytics costs impose on data-driven enterprises. The survey, which included 300 senior data management professionals from US companies, revealed that 71 percent of respondents frequently face unexpectedly high cloud analytics charges. Specifically, 5 percent of companies experience cloud “bill shock” on a monthly basis, 25 percent every two months, and 41 percent quarterly.
Furthermore, despite having substantial budgets, an alarming 98 percent of companies encountered failures in machine learning (ML) projects in 2023 due to escalating cloud costs.
Bill shocks often arise when data workflows exceed the capabilities of existing cloud query engines, either in complexity or size. As enterprise AI and data analytics tech stacks increasingly rely on scalable cloud platforms to handle large datasets and sophisticated algorithms, the associated cloud costs continue to rise.
“As data and analytics evolve, companies are compelled to limit dataset sizes and reduce complexity to control expenses, which adversely affects the quality of their insights,” said Deborah Leff, Chief Revenue Officer at SQream, in an interview with Observer. “The high cost of cloud-based experimentation often deters the initiation of AI/ML projects, and poor data preparation and insufficient data cleansing further contribute to project failures.”
The Cloud-AI Paradox Strains Enterprise Analytics Budgets
The rising costs associated with cloud-based data and AI technologies have become a major hurdle for enterprises. As cloud cost inflation is expected to continue in 2024, companies are feeling the pressure to implement cost-cutting measures more aggressively than ever. US government economic data and vendor research reveal a consistent pattern of increasing cloud expenses. According to the Bureau of Labor Statistics’ Producer Price Index (PPI) for 2024, data processing and related services, which includes cloud computing, have experienced a 3.7 percent year-over-year increase.
The financial strain is affecting data queries and project volumes. Nearly half of the enterprises (48 percent) in a recent SQream study reported scaling back the complexity of queries to manage analytics costs, particularly those related to cloud resources and compute loads. Additionally, 46 percent are restricting AI-powered projects due to cost concerns.
The impact of cost pressures extends beyond just vendor pricing. Deborah Leff, Chief Revenue Officer at SQream, noted that many businesses fail to thoroughly evaluate which in-house IT assets might benefit from cloud migration.
“Cost is a major driver of project failures, particularly during the experimentation phase. It’s not that machine learning architectures are inherently flawed, but rather that management often decides to halt investment when expenses become untenable. The time to value is crucial, and the costs of experimenting can soar due to the scale and complexity of modern data,” Leff explained.
In terms of data preparation, a third of the companies surveyed (33 percent) use between 5 to 10 different solutions or platforms, complicating the process. Multiple users employing various tools in parallel can create challenges in identifying bottlenecks and analyzing processes effectively.
“The data center ecosystem, reliant on outdated technology, needs a significant overhaul. Sticking with obsolete methods is not a viable solution. Companies should explore innovative approaches to prevent cost and data limitations from constraining their analytics capabilities,” Leff advised. “While tools like NVIDIA Rapids offer value, they require specialized developer skills, highlighting the need for more accessible solutions. Companies must challenge the status quo and seek better alternatives to overcome current constraints.”
Embracing New Methods to Mitigate Cloud Costs
As companies contend with market disruptions driven by generative AI and the surge in large language models (LLMs), the dramatic increase in data volume and complexity underscores the need for machine learning technologies to stay competitive. However, constraining data queries to manage cloud costs often results in shallow insights and premature project cancellations. According to a recent study, 92 percent of companies are actively working to “rightsize” their cloud analytics spending to better align with their budgets.
Deborah Leff, Chief Revenue Officer at SQream, highlighted that while GPU acceleration might seem costly, it can significantly lower overall expenses while enhancing processing speed. GPU acceleration offers cloud benefits with appropriately scaled parallel processing resources and flexible pay-as-you-go pricing, allowing enterprises to rent GPU resources as needed and automatically scale on demand.
Leff cited the example of NCBA, a major online bank with up to 60 million daily users, which initially faced a 37-hour delay in updating marketing models with daily click data. Despite efforts to optimize queries and invest in expensive hardware, this lag prevented strategic use of data. Transitioning to GPU acceleration cut their data pipeline cycle time to just seven hours, enabling daily model updates and more agile data use.
“Companies need to adopt a proactive mindset and push the boundaries of what’s achievable,” Leff advised. “The rapid evolution of generative AI suggests that current data strategies may soon become inadequate. The next two years are likely to bring dramatic changes in the IT sector.”
Leff concluded, “To thrive in an era of growing data and increasingly complex queries, businesses must move beyond outdated limitations. Embracing innovative methods like GPU acceleration can unlock substantial value, and those who adapt swiftly will benefit the most.”
