Compares

Gemma 4 on Your Machine: How Google’s New Open Weights Stack Up (Model Showdown)

Will Schenk April 4, 2026

Gemma 4 on Your Machine: How Google’s New Open Weights Stack Up (Model Showdown)

We benchmarked Gemma 4 (e2b, default, 26B MoE, 31B dense) through Ollama against 50+ hosted and local models on reasoning, knowledge, instruction, coding, and TezLab MCP tool use—same Umwelten harness as our other showdowns. Here’s where the new line shines, where frontier models still pull ahead, and how the biggest Gemma handles real EV data tools.

Read more →

Same Weights, Different Results

Will Schenk March 24, 2026

Same Weights, Different Results

We ran the same Nemotron model on four providers and got wildly different results. MCP tool use ranged from 1/6 to 6/6. Speed varied 16x. The weights are identical. The results are not.

Read more →

Can LLMs Use Real-World Tools? Mercury-2, ELO, and the Umwelten Setup

Will Schenk March 15, 2026

Can LLMs Use Real-World Tools? Mercury-2, ELO, and the Umwelten Setup

We ran 39 models against real Rivian driving data via MCP tools. Inception's Mercury-2 delivered a perfect 15/15 in 8.6 seconds. Here's the standout model, the ELO narrative rankings, and how the same Umwelten setup powers both chat and evals.

Read more →

The Car Wash Test: Learning from Model Evals

Will Schenk March 1, 2026

The Car Wash Test: Learning from Model Evals

We asked 131 AI models a simple question — should I walk or drive to the car wash? 76% got it wrong. Simple gotcha questions reveal more about model reasoning than any benchmark leaderboard.

Read more →

June 2025 Coding Agent Report

Will Schenk June 15, 2025

June 2025 Coding Agent Report

A comprehensive analysis of 15 leading AI coding agents in 2025. We break down the strengths, weaknesses, and surprises from top tools, with clear winners for pros, tinkerers, and casual users alike.

Read more →

Moral Vibe Check

Will Schenk March 24, 2025

Moral Vibe Check

Technical correctness and meaningful insight: well-formatted, detailed AI responses can mask a fundamental lack of understanding—a "raving lunatic" hidden behind impressive form. Maybe P-doom is less about malice and more of making us intellectually poorer by substituting form for substance, facts for understanding, and technical accuracy for wisdom.

Read more →

Agentic YOLO with Warp, Cursor, and Claude

Will Schenk March 7, 2025

Agentic YOLO with Warp, Cursor, and Claude

What happens when you let AI help you think through and build your ideas, with minimal supervision and maximum trust? What does it mean to be a programmer? Are we closer or further from thought-stuff?

Read more →

Slicing up a design from figma

Will Schenk November 27, 2024

Slicing up a design from figma

In this hands-on comparison, three coding tools - Cursor, Aider, and v0 - are put to the test as they attempt to replicate a design from Figma into functional HTML and CSS code, revealing their strengths, weaknesses, and quirks.

Read more →

Subscribe to our newsletter

Powered by Buttondown.

Ready to ship production AI?

Whether you need a quick Vibe Check or a full Habitat built on Habitat OS, we'd love to hear what you're working on.

Talk to a principal See Our Work