Smarter AI Without Bigger Models: Recursive Composability with Tiny Models

Intro

You’ve probably heard the mantra: “Bigger model, better results.” But what if you could get smarter, more reliable outputs without scaling up your parameter count? That’s the bet behind a new project from Samsung SAIL Montreal called TinyRecursiveModels.

It’s a departure from the “just add more GPUs” mindset. Instead of cramming everything into a single large transformer, this repo explores how tiny models can be composed recursively to produce surprisingly robust reasoning. It’s not about achieving GPT-4 level results on one pass — it’s about making small models work harder, not bigger.

What It Does

The core idea is recursive self-improvement using small language models. You take a tiny model (think 100M-300M parameters), run it iteratively: each step feeds its own output back into the model, allowing it to refine or build on its previous response.

The key difference from simple “chain-of-thought” prompting? The recursion is baked into the architecture and training process. The model learns to not just generate a single response, but to generate and then reason about its own generation, then generate again. It’s like giving a small model a notepad and asking it to think out loud, then check its work, then try again.

The GitHub repo provides:

• Training scripts for recursive fine-tuning on small models
• Evaluation benchmarks that show this recursively composed output outperforms the same model used in a single forward pass
• A lightweight framework for experimenting with recursion depth, temperature, and context window

Why It’s Cool

This isn’t “just prompt engineering.” The recursive behavior is trained into the model, not added as a user instruction. That means the model learns how to leverage its own internal state across multiple steps — something that feels a bit like iterative reasoning but without needing a massive parameter budget.

A few genuinely clever bits:

• Composability: You can stack a few tiny models together and get emergent performance that matches or beats a single model 2-3x its size on specific tasks (like symbolic reasoning, arithmetic, or simple code fixups).
• No extra hardware: These models run on a single GPU easily. You’re not trading cost for performance in the usual way.
• Transparency: Because the recursion loops are explicit, you can actually inspect what the model “thinks” at each step. That’s a huge win for debugging and trust.

The biggest real-world use case? Edge or on-device AI. If you’re shipping a model to a phone, a Raspberry Pi, or a web browser, you can’t afford a 7B parameter monster. TinyRecursiveModels gives you a path to better results without blowing your memory budget.

How to Try It

Clone the repo, set up a Python environment with PyTorch and Hugging Face Transformers, then run the included notebook. Here’s the quickstart:

git clone https://github.com/SamsungSAILMontreal/TinyRecursiveModels.git
cd TinyRecursiveModels
pip install -r requirements.txt

Then run the demo:

python run_recursive.py --model "microsoft/phi-1_5" --depth 3 --prompt "If 2x + 3 = 11, what is x?"

You’ll see the model output for each recursion step and a final answer. There’s also a train_recursive.py script if you want to fine-tune your own small model on a custom dataset.

Full details are in the README, but the TL;DR is: pick a small base model, choose a recursion depth (2–5 works best), and see how the output improves with each loop.

Final Thoughts

This project feels like a smart counterpoint to the “scaling is all you need” trend. It acknowledges that bigger isn’t always feasible or practical, and shows that smarter usage of small models can go a long way.

Will it replace foundation models? No. But for specific tasks like step-by-step reasoning, code debugging, or constrained generation, recursive composition offers a tangible win — without the cloud bill. If you’re building for environments where every megabyte counts, this is worth a few hours of experimentation.

— @githubprojects