Shipulski On Design

Posts Tagged ‘Sam-I-Am’

If you want to learn, to really learn, experiment.

But I’m not talking about elaborate experiments; I’m talking about crude ones. Not simple ones, crude ones.

We were taught the best experiments maximize learning, but that’s dead wrong. The best experiments are fast, and the best way to be fast is to minimize the investment.

In the name of speed, don’t maximize learning, minimize the investment.

Let’s get right to it. One of the best tricks to minimize investment is to minimize learning – learning per experiment, that is. Define learning narrowly, design the minimum experiment, and run the trial. Learning per trial is low, but learning per month skyrockets because the number of trials per month skyrockets. But it gets better. There’s an interesting learning exponential at work. The first trial informs the second which shapes the third. But instead of three units of learning, it’s cubic. And minimizing learning doesn’t just half the time to run a trial, it reduces it by 100 or more. It’s earning to the hundredth power.

Another way to minimize investment is to minimize resolution. Don’t think nanometers, think thumbs up, thumbs down. Design the trial so the coarsest measuring stick gives an immediate and unambiguous response. There’s no investment in expensive measurement gear and no time invested in interpretation of results. Think sledgehammer to the forehead.

A third way to minimize investment is to evaluate relative differences. The best example is the simple, yet powerful, A-B test . Run two configurations, decide which is better, and run quickly in the direction of goodness.  No need to fret about how much better, just sprint toward it.  The same goes for trial 1 versus trial 2 comparisons. Here’s the tricky algorithm: If trial 2 is better, do more of that.  And the good news applies here too –  the learning exponential is still in play. Better to the hundredth power, in record time.

I don’t care what norms you have to bend or what rules you have to break. If you do one thing, run more trials.

But don’t take my word for it. Dr. Seuss had it right:

And I would run them in a boat!
And I would run them with a goat…
And I will run them in the rain.
And in the dark. And on a train.
And in a car. And in a tree.
They are so good so good you see!