Olam Labs
Two-person startup testing AI models by having humans unknowingly play games against them.
Olam Labs has an idea that academic frameworks already do for free and that funded rivals are now buying up, with no ranked product yet to show it collects data those rivals cannot.
Olam Labs uses games to test how well AI systems work, which is a smarter way to evaluate AI than standard tests. The problem is that universities, other startups like Scale AI and Surge AI, and even OpenAI have all launched similar game-based testing tools since 2023, so the basic idea is no longer unique. Olam's only real advantage is data from real people playing against AI, but that data collection is still in early stages with no public results to show. The one thing that could change everything is if they launch a public leaderboard that becomes the standard place where AI companies want their models ranked, which would give them real stickiness before bigger competitors fully copy them.
Olam Labs publicly launches a ranked leaderboard or benchmark from Social Arena. Would show the product has moved past alpha and is gathering usable data at scale.
Mercor, Scale AI, or Surge AI announce a game based behavioral eval feature within a year. Would show funded incumbents are closing the gap Olam relies on for defensibility.
- Team size: 2 people, San Francisco
- Mercor valued at $10B after $350M raise
- AI2 shipped olmo-eval workbench
- Core game-based evaluation method is open academic art, not proprietary, easily replicated.
- Funded incumbents (Mercor, Scale, OpenAI, AI2) are actively acquiring or building the same niche.
- Product remains unranked 'Alpha' with no leaderboard, benchmark, or disclosed customers as of July 2026.
- No disclosed funding round beyond standard YC investment, limiting runway to compete.
- 2-person team faces execution risk against 15+ employee peers like HUD.
| Rival | Direction | Evidence |
|---|---|---|
| Mercor | losing to | Acquired sim-eval startup Deeptune, July 2026, directly entering Olam's niche |
| Scale AI | losing to | 49% Meta-owned, operates human-panel data at industrial scale vs Olam's 2-person team |
| OpenAI (Promptfoo) | losing to | OpenAI acquired eval tool Promptfoo March 2026, absorbing eval-tooling niche |
| AI2 (olmo-eval) | contesting | Shipped open eval workbench olmo-eval June 2026, free alternative to proprietary eval startups |
| clembench (academic) | contesting | Open dialogue-game LLM eval framework active since 2023, non-proprietary precedent |
| Surge AI | losing to | Reportedly $1B+ revenue building realistic RL environments for frontier model evaluation |
- Is Olam Labs profitable?
- No revenue or profitability data exists for Olam Labs as of July 2026. The company is a 2-person, pre-seed YC S26 startup with no disclosed customers, and its products are still labeled 'Alpha' with no ranked leaderboard released yet.
- Has Olam Labs had layoffs?
- No layoffs are reported. Olam Labs is a newly founded 2026 startup with a 2-person team based in San Francisco, part of Y Combinator's Summer 2026 batch, and has no hiring history to reduce as of July 2026.
- Could AI replace Olam Labs' own business?
- Olam Labs' core method, game-based LLM evaluation, is already open academic art via clembench (since 2023), GameEval, GVGAI-LLM, and V-MAGE, meaning frontier labs and open frameworks could replicate its orchestration approach without needing the startup itself.
- Who is Olam Labs' biggest competitor?
- Mercor, valued at $10B in October 2025, acquired simulation-eval startup Deeptune in July 2026, directly entering Olam Labs' niche. Scale AI (49% Meta-owned) and Surge AI (reportedly $1B+ revenue) also operate at incumbent scale in adjacent data-eval markets.
- Who owns Olam Labs?
- Olam Labs is privately held with no independent parent company; it is backed by Y Combinator as part of the Summer 2026 (S26) batch, based in San Francisco. No funding round or valuation beyond YC's standard batch investment has been publicly disclosed as of July 2026.
- Does Olam Labs have a working product yet?
- Yes, called Social Arena: Catan-style, Risk-style, poker-style, and Codenames-style games where humans unknowingly play against AI agents. As of July 2026 all games are labeled 'Alpha,' still collecting data before any leaderboard or benchmark release.
- 2026-07-22YC directory lists Olam Labs at 2 employees under Reinforcement Learning category
- 2026-07Mercor acquires simulation-eval startup Deeptune, entering Olam Labs' niche
- 2026-07Olam Labs confirmed as part of Y Combinator Summer 2026 (S26) batch, San Francisco
- 2026-06-16YC P26 batch breakdown: 137 of 196 companies building LLM agents
- 2026-06-12AI2 ships olmo-eval, an open evaluation workbench for LLM development
- 2026-03OpenAI acquires Promptfoo, consolidating AI eval tooling market
- 2025-10Mercor reaches $10B valuation after $350M Series C round
- 2025-06-19Meta takes ~49% stake in Scale AI via $14.3B investment
- Olam Labs: Evaluating models for behavior and performance through simulated games
- Olam Labs, Social Arena
- Jobs at Olam Labs
- Dialogue Game-Based Evaluation using clembench (2025-07-01)
- olmo-eval: An evaluation workbench for the model development loop (2026-06-12)
- Meta's $14.3B stake triggers Scale AI customer exodus, could be a windfall for rivals like Mercor (2025-06-19)
- Mercor -> History (aggregated wiki source)
- AI Agent Eval Frameworks 2026 (2026-05-21)
- Reinforcement Learning Startups funded by Y Combinator (YC) 2026 (2026-07-22)
- YC P26 Batch: The Full Breakdown (2026-06-16)
- Olam Labs
- Our Re-organisation