Part 1: Why Small is the New Big
I found it heartening to see not just one but multiple sessions focused on Small AI at the 2026 WB/IMF Spring Meetings in Washington DC last week. For years, the AI narrative was dominated by Big AI with mammoth models like GPT or Gemini or Claude or Grok, boasting trillions of parameters and requiring the energy of a small city to run.
But for a farmer in Malawi or a community clinic in a rural Appalachian town, Big AI is often a Big Barrier.
The digital divide used to be about who has an internet connection and who has the capacity to use smart devices. Now, AI, instead of bridging that divide, is widening it by adding a new dimension of barrier: who can afford the compute to process intelligence.
The contrast between the two paradigms is stark. Big AI is high-cost, cloud-reliant, and power-hungry. In contrast, Small AI, typically defined as models with under 10 billion parameters (often just 100 million), is designed for the edge.
During the Spring Meetings, experts and industry leaders emphasized that while connectivity has improved, the usability gap of digital remains. Big AI models require expensive API fees and high-speed data transfers that are cost-prohibitive for local governments and general masses in LMICs.
Small AI, however, ushers in a new paradigm of possibilities for digital inclusion.
The Economics of Intelligence
Training a GPT-4-class model is estimated at $100-200 million. By comparison, SLMs can often be trained and fine-tuned for a few thousand dollars. The contrast is even more striking with inference cost, which is the cost of getting an answer from an AI. Recent research indicates that Small AI models can reduce inference costs by 10x to 200x compared to frontier LLMs. Because these models can run on local CPUs or basic mobile chips rather than $30,000 H100 GPUs, the device requirement drops from a server room to a $100 smartphone. Cost per query drops from 9 cents to 0.04 cents in many cases.
| Feature | Big AI | Small AI |
| Inference Cost | High (API fees/GPU clusters) | Low (can run on local CPUs) |
| Connectivity | Requires stable high-speed internet | Functional offline or on edge devices |
| Energy | Massive data centers | Mobile-phone friendly |
The Quality of Intelligence
Research shows that task-specific models can achieve 95%+ accuracy in narrow domains, often surpassing generalist LLMs that get distracted by their own vast training sets.
Small AI models achieve high scores (e.g., 85 on the MMLU benchmark) by using data that has been filtered to remove low-quality internet noise. This results in a model that doesn’t just know more, but knows better quality information about specific subjects.
In sectors like finance and law, specialized SLMs have been reported to achieve 35% fewer critical errors than general-purpose LLMs. For tasks like processing complex medical records or legal contracts, specialized Small AI models can reach 99% accuracy, whereas general models frequently hallucinate or miss niche terminology that wasn’t a priority in their massive training sets.
Data representing LDCs and LMICs and marginalized populations in developed countries most likely fall into the low priority category as well. This means LLMs will very confidently provide wrong answers when applied to these populations.
Small AI models have a significantly higher probability of doing much better.
The Important Question
So, if the cost of intelligence is much lower and the quality of intelligence is higher (and sometimes much higher) with Small AI, why are we not adopting it in droves? What do you think?
Anir Chowdhury
#WBGMeetings #DigitalTransformation #AIGovernance #DigitalInclusion
Leave a Reply