Validation & Experimentation

What is an MVP?

A Minimum Viable Product is the smallest version of a solution that lets you test your riskiest hypothesis with real users — not a stripped-down "v1" of the final product, but a learning vehicle.

MVP is Not the Same as "Minimum Viable Feature"

An MVP can be far smaller than a working feature: it can be a landing page, a concierge/manual process behind the scenes, or a clickable prototype — whatever is cheapest to build that still produces real evidence.

Common MVP Types

• Landing page MVP — test demand before building, via sign-ups or pre-orders.
• Concierge MVP — manually deliver the "service" behind the scenes to a few users before automating.
• Wizard of Oz MVP — users believe they're using an automated product; humans operate it behind the curtain.
• Single-feature MVP — build only the one core feature, skip everything else.

Choosing the Right MVP

Match the MVP type to the assumption you're testing: a landing page tests desirability cheaply; a concierge MVP additionally tests usability and process feasibility with real interactions.

Designing a Good Experiment

Every experiment should map back to a specific hypothesis (Module 9) and produce a clear, unambiguous result.

Key Components

• Hypothesis — what you believe and the threshold for success.
• Method — the smallest, fastest way to test it (interview, prototype, MVP, A/B test).
• Sample — who you'll test with, and is it representative of the target segment?
• Success metric and threshold — defined before running the test.
• Duration — long enough for valid data, short enough to learn fast.

Avoiding Bad Experiment Design

• Don't change the success criteria after seeing results (this invites confirmation bias).
• Don't test with internal employees or atypical users when validating real customer assumptions.
• Make sure the experiment can actually fail — if every possible result would be spun as "success," it's not a real test.

The Build-Measure-Learn Loop

Lean Startup's core loop: build the smallest test, measure against your defined metric, learn, then decide to pivot, persevere, or kill the idea — repeating quickly rather than in one large cycle.

What is A/B Testing?

A randomized controlled experiment where users are split between a control (A) and a variant (B) to measure the causal effect of a specific change on a metric.

When A/B Testing Works Best

• You have enough traffic/users to reach statistical significance in a reasonable time.
• You're testing a fairly contained, well-defined change (not a totally different product direction).
• You can isolate one variable at a time for a clean read on causality.

Key Concepts

• Statistical significance — confidence the observed difference isn't due to random chance.
• Sample size — too small, and you can't trust the result either way.
• Novelty effect — short-term lifts that fade once users get used to a change; consider running long enough to rule this out.

When NOT to A/B Test

For low-traffic products, completely new concepts, or qualitative usability questions, A/B testing is the wrong tool — use interviews, usability testing, or smaller MVPs instead.

A Toolbox of Validation Techniques

• Fake door / smoke test — show a button/feature that doesn't exist yet to measure click-through interest.
• Prototype testing — put a clickable mockup in front of users and observe task completion.
• Pre-sales / letters of intent — ask for a real commitment (payment, signed intent) as the strongest desirability signal.
• Diary studies — track real behavior/feelings over time for slower-forming habits.
• Dogfooding — use your own product internally to catch obvious usability issues cheaply.

Matching Technique to Risk

Pick the technique based on which risk (value, usability, feasibility, viability) is most uncertain and costly if wrong — don't default to the same method for every situation.

Validating Without Building

The cheapest, fastest validations (fake doors, concierge MVPs, prototype tests) should always be tried before committing engineering time — protect build capacity for ideas that have already cleared a desirability bar.