AI Is Starting to Look a Lot Like the Early Days of Cloud – and the Real Race Is Operational

Over the past two years, most of the noise around AI has focused on the model race – whose model is bigger, faster or scoring better on benchmarks. But as AI moves from pilots into the core of products and workflows, a familiar pattern from the early days of cloud is re‑emerging: systems are more programmable than ever, but they are also much harder to run. And that means we now know where the most important competition in AI is shifting: from who has the “best” model to who can operate AI reliably, efficiently, and safely at scale.

When looking at real‑world telemetry from thousands of production systems, a clear picture starts to form. Nearly 1 in 20 AI requests fails once applications reach scale, and a majority of those failures now stem from capacity limits such as rate limits, quotas and concurrency caps, rather than from model bugs or poor accuracy. That is a very different story from the benchmark charts most teams used to obsess over. The amount of data sent per request is also climbing. Across many production estates, median users have more than doubled their token usage, while heavy users have seen volumes grow several‑fold. That growth is both a symptom of more ambitious AI use cases and a direct driver of cost and IT infrastructure stress.

Across Asia‑Pacific, and especially in ASEAN, we’re currently seeing structural pressures: AI adoption is accelerating, but operational maturity is uneven. Singapore is further along on governance and observability, driven in part by regulatory expectations and a more mature cloud landscape. Meanwhile, markets such as Indonesia, Malaysia and Thailand are moving very fast on deployment, often pushing AI into customer‑facing services while operational practices catch up. As organizations across these markets roll out multi‑model and agent‑based architectures, they are running into reliability issues, limited visibility and inconsistent model performance.

With the evolution of AI resembling the early days of cloud, the good news is that we can predict, at least a little, where things are headed. Now, the question AI leaders should be asking is this: which disciplines distinguish the teams that will cope best with this complexity? In my view, there are four that teams working with AI need to adopt to see sustainable success: establish visibility and attribution, enforce control and guardrails, optimize GPU utilization before scaling supply, and manage cost and optimization.

You cannot operate what you cannot see, and AI is no exception. Teams need to see how GPU hours and tokens map to specific applications, teams and use cases, so they can connect that usage to latency, error rates and user impact. Without guardrails, AI systems will consume as much capacity as you give them. Practical controls include rate limits and budget caps, along with safeguards on agent behavior to stop unbounded retries, loops and poorly bounded workflows from exhausting shared resources. Most teams reach for more GPUs when what they really have is a utilization problem. GPU instances already account for a significant share of compute costs, and that proportion only grows as organizations push deeper into training and inference at scale. What we learned during the early days of cloud is that in these instances, overprovisioning becomes the safest default – but then spend balloons even when there is stranded capacity.