How Business Experimentation Leads to Business Innovation Pt. 2

There could be too much experimentation if you’re not bringing rigor to the experimentation itself and if you’re deriving too many false positives and too many false negatives. We initially want to define a solid hypothesis. We then want to structure an effective experiment that controls as many variables as possible to ensure the experiment’s integrity. Suppose we don’t follow that process and experiment at too grand a level. In that case, we introduce too much noise and too many uncontrolled variables, resulting in outcomes we can’t trust because we don’t know if it’s a true positive, false positive, true negative, or false negative. Running too many experiments without standardization and rigor in the process can lead to experimentation failure.

Now, I will say that many other individuals believe that you can’t run enough experiments, no matter how many you do. In that case, I would say that you need:

• An effective governance process.
• Well-structured experiments.
• The ability to learn insights from your experiments.
• The ability to put those insights into action.

If all you’re doing is running experiments and not using lessons you’ve learned and key insights to drive services or products, or drive better hypotheses and better experiments, all you’re doing is exercising a process for no value.

There is a level of subjectivity inherent in business experimentation; we can’t avoid that at Gannett Fleming because we are a consulting services firm and must be agile. Suppose all we’re doing is experimenting based on rigorous logic and things we or others have already proven. In that case, as professional businesspeople, we are losing an opportunity to bring new ideas to the table that we do want to test and validate. There is still a role for instinct and intuition to play in this, and that’s to help us ask the right questions and then quickly derive a logical hypothesis to test. The key to selecting effective business experiments is to focus on those hypotheses under review from individuals or groups that are proposed in combination with a well-structured experiment, and an objective that is business-value driven.

We need to take failure into context. If you are experimenting at scale, the overwhelming majority of your experiments will likely fail. For me, I will accept failure all the time if we’re:

• Failing small and failing quickly.
• Failing with well-structured experiments.
• Putting into action the lessons we learned from those failures.

Innovative companies don’t want to experiment for the sake of experimentation. In the private sector, we realize business value based on three pillars:

• Drive top-line revenue.
• Increase bottom-line profits.
• Mitigate business risk.

I think experimentation is effective and provides value when you can ultimately map it to one or more of those three business outcomes.

You need to evaluate the professional risk inherent in that action. There are so many changes and experiments that any individual within a firm can run at a very small scale that help accrue data to validate what they think is a logical outcome of a change. This gives you data to defend why you believe a change should occur.

When you’re considering testing, start small using a sample size within your control to determine how you mitigate your own personal and professional risk. You may run an experiment using a sample size that gets you an outcome you can then take to your manager to scale up from there. In situations where you have a logical assumption and are confident in your hypothesis, but may experience managerial blockers, I recommend being “politely subversive” because you need to experiment in small ways to come up with the data that either validates or invalidates your assumptions.

There definitely is, and there is certainly a journey to go through. The internet generally provides a lot of material for maturing your understanding, bringing more rigor into experimentation, and offering opportunities to discover toolsets that help automate experimentation.

I would also point to YouTube and simply search “experimentation, “scientific method,” and “business experiments.” There are plenty of YouTube videos that are short and excellent in their accessibility to help you understand things like:

• What is an experiment?
• How do you derive a proper hypothesis?
• How do you structure a proper experiment?
• How do you understand and discern or interpret results?

Once you have a basic understanding of experimentation, you can dig deeper into statistics, standard deviations, and better controlled and uncontrolled variables management.

Lastly, a go-to book I’ve read that is just fantastic is Experimentation Works: The Surprising Power of Business Experiments by Stefan Thomke. It’s probably the most accessible, practical book I have found on the subject, and you can also check out a presentation he gave on the topic of business experimentation if you prefer this format.

Advocate for starting small and starting fast. Think: What could be a business improvement? What could be a quick test? Break down the value proposition into as many small independent components as you can and then test small, experiment small, and realize small. This will help you start to connect those small dots into a bigger picture that allows you to scale up in terms of complexity in your experiments and how you may implement the results of those experiments.

Take your innovative ideas and business experimentation to a new level – apply to our open positions and join our talent community today!