Open Source Models: "Small" AIs Challenging Gemma on TPU v5e
·2 min read·Beginner
“
Imagine having the intelligence of a giant in a pocket-sized format. A new analysis tested open source AI models to see who can keep up with the big players.
In 30 seconds
01Qwen3-4B matches Gemma 4 E2B's accuracy on math problems, though it's less efficient in output.
02
→
💡
What this means for you
This news means you don't need a supercomputer or an unlimited budget for powerful AI in certain tasks. Artificial intelligence is increasingly becoming "accessible to everyone," even with modest hardware.
Imagine buying gold for your treasure chest, and the supplier tells you they won't sell it anymore because you might do something bad with it. That's pretty much what Jensen Huang, Nvidia's boss, said about AI chips.
·1 min·Beginner
SmolLM3-3B and Granite 3.3 2B equal Gemma in 'tool calling' capabilities, vital for AI agents.
03This study shows smaller, open AI models can compete with larger ones on modest hardware.
0101
Can the Small Ones Beat the Giants?
The short answer is yes, at least in certain areas. Recent research compared fourteen open-source AI models, created by various developers, against Google's Gemma 4 E2B. The goal was to see which of these "featherweights" could stand up to a more established model, all running on a single TPU v5e chip, a specialized processor for AI.
This wasn't just any race; it was a real-world test. The models were run one after another on a single pinned system image, using vLLM to optimize performance. This allowed for an apples-to-apples comparison, or rather, chip-to-chip, to see who was the fastest and most accurate in assigned tasks. The tests focused on mathematical problems (GSM8K) and the ability to use external tools (BFCL).
0202
Who Was the Biggest Surprise?
Among the many, Qwen3-4B proved to be a real surprise. This model matched Gemma 4 E2B's accuracy on mathematical problems, which is no small feat for a smaller model. Sure, it produced 43% less text in the same amount of time, but the precision was identical. A bit like a sprinter who finishes first, but with less apparent effort.
📬 Enjoying this article?
Get the best AI news every week, straight to your inbox.
But that's not all. Two other models, SmolLM3-3B and Granite 3.3 2B, excelled in a different area: tool usage. They matched Gemma in their ability to handle "tool calls" (BFCL), which is crucial for AI agents that need to interact with the outside world. Imagine an AI that needs to book a flight or check the weather: the ability to use tools is everything. Who would think of asking an AI to calculate travel costs without knowing how it interacts with booking sites?
0303
Does Gemma Still Have an Ace Up Its Sleeve?
Absolutely. Despite the strong performance of the smaller models, Gemma maintained its advantage in a crucial area: handling long prompts. That is, when the AI needs to process very extensive texts, like an essay or a long document, Gemma proved to still be a step ahead. This means that for tasks requiring significant "memory" and contextualization capabilities, Google's model remains king.
These tests were conducted by George Deen on dev.to and published on May 30, 2024. The research involved fourteen Apache-2.0 and MIT licensed open-source models, all capable of running on a single TPU v5e chip. It's proof that the AI race isn't just about who has the biggest model, but also about who can optimize resources best. And for us, end-users, that's excellent news.
Voice translation isn't sci-fi anymore, it's a booming reality taking off fast. DeepL is at the forefront, and its CEO has clear ideas about tackling the future.
While everyone else rushes to unveil the next big AI, OpenAI is hitting the brakes. Safety first, or so they say, keeping their next-generation model on standby.
NUOVO
