Hypothesis Driven Development with example

My previous blog-post was an introduction to HDD (Hypothesis driven development). In this blog post I will take it a step further and give you an example how you can use it and how easy it can be and how it can change the way you work entirely to a new level.
You can read it [here].

Remember:
HDD is an experimental driven process, it’s main goal is to find out if a certain idea generates ROI and happy users. To get fast result you will also need fast support to evaluate your experiment, if you want to evaluate the idea in production I prefer a smooth decoupled architecture and a mature CI & CD organization. It’s not a must to, but it will really decrease development time and cycle time.
Sadly, I have no maturity model written in English yet. You can read one I did in Swedish on my LinkedIn posts [here]

Let’s start the story.

Team:  A cross-functional team with talent people regarding:
Design, UX, Coding, Testing and so on. And also including some market people and product owner.

At a meeting with the customer and team a team member (TM) got this cool idea regarding an e-commerce platform.

– What if the stock value changes in real time? Wouldn’t that be cool? Maybe it increases the conversion rate? The users might be stressed that this nice shoe will get out of stock?

A business person (BP) responds:
– What will it cost? Can you estimate?

– I don’t know, and I don’t. And I don’t know if my hypothesis even works. Can’t we skip this estimating and cost discussion and find out and then see if it’s worth implementing at all?  (TM)

– How? (BP)

– Let’s just do some A/B testing and fake a simulation of stock value changes and monitor if the user will buy the product more often or faster than today? If we get an 2% increased conversion rate it really worth implementing isn’t it?  (TM)

– Yes, you got a points there, this Sounds interesting. (BP)

Instead of creating User Stories the team now write Hypothesis stories.

Real-Time stock value changes
We Believe That
real-time stock values on the product page
Will Result in improved customer engagement and conversion
We Will Know We Have Succeeded When we see a 5% increased speed and about 2% conversion increase from the time the stock change in real-time till they press the add to cart button.

The meeting generate lots of hypothesis stories and the product owner can now prioritize them based on the ROI for the result outcome.

– A 2% increased conversion rate is really nice. Today we got like 5000$ each day, 2% will give us about 100$ more. In a month it’s around 3000$. If the experiment works and generate this value; You can spend around 30h of development time and we still get more money than not implement it at all. (BM)

– Yes you can say that, but let’s not convert it to time yet, let’s see if it’s worth implementing at all. Because if we spend around 10-30 hour for this feature and it will not increase conversion at all; we just wasted lots of money for developing a crappy feature. Let’s be smart and just spend time creating features that generate values in short and long runs instead. Assumptions is not a good thing. (TM)

– Ok, we can give it a try, because we have done so many bad assumptions we even can’t monitor at all. (BP)

The Product owner decided to activate the experiment and tells the team to do the fastest possible implementation to get information to see if this hypothesis will generate values at all.

The team gathered as usual at a stand-up and activated the work of the hypothesis.

– So what is the fastest possible solution to get rapid feedback regarding this? (TM)
– We can ask the customer?
– Nah, the customer might say, cool feature or – Do you want to stress us even more?
– Ok, let’s implement a test in our code and monitor the outcome?
– Sure, what’s the fastest way?
– This is part of the UI so let’s do it simple with some JavaScript. The only thing we need to understand is if it works. So there is no need to make it perfect. We do not need to add back-end functionality to get this information either. I can type a JavaScript that just randomly decrease the stock-values on some products. And then monitor with our application-insight code if they will press the button faster or not. It doesn’t matter if the stock value is a faked value for a while… If the value is 5 or 4 doesn’t matter that much.
– Nice, cool this will take us around 15 minutes or so? And thanks to our CI  & CD approach the feature will be out within 20-30 min or so and we can then start monitor it. If the result is good, then we implement it all the way with tests and quality in mind.
– Cool, this is nice, if the PO asked us to estimate it and add management work around it and do all the other stuff we do for a user story it would take longer cycle time than 30 minutes for just get started on this one. We aren’t wasting anything here, just getting knowledge and information if the idea is worth implementing or not. We will save around 15 minutes and also get the information if it’s even worth implementing. It’s much better than assume it does by implement it and hope it will generate values.

The team agreed.

Liza loved the idea regarding NoPSD so instead of spend time on working with PSD files or other tools for design she just opened the SASS file (css manipulation) and added three new classes with three different color. Green for normal stock values, Orange that indicate near out of stock and red that indicate very soon out of stock.

.stock-Normal
{
     color: #38893d;
}.stock-Warning

    color: #fe8f01;
}
.stock-Critical

    color: #be0000;
}

Thanks to nice SASS structures it just took her 2 minutes. When she pushed the code to the source control it automatically generated a new css file that got released into production, this because the team had nice CI & CD approach. (Why not release it? It will not hurt anything though nothing using the classes at the moment anyway.)

The front-end was built on AngularJs and the stock area was created as a web component and rest of the system is based on ASP.Net MVC and Web Api. Meanwhile Liza added the css John added a feature toggle in the code and took the html from the web component template and added it to the new feature for the hypothesis.

The feature toggle framework they use uses one configuration file to indicate if the feature is on and off.
They simple add it as an appSetting in web.config or appSettings.json (if using ASP .Net 1.0 Core).

“IRLStock” : “true”;

The team uses a framework that uses a C# class as a feature toggle so they easy can remove the class and get compilation error where the feature existed, so they do not forget to remove the code later on.

public class IRLStock {}

The code with razor syntax (ugly code? I know but much faster than create a new web-component to get the result from this experiment, it’s important to not waste unnecessary time for a more complicated solution, then you missed the whole point. This code will be deleted after the experiment anyway. For experiments it’s just waste to add sugar to it.):

@if(Feature<IRLStock).Is().Enabled)
{  
     < script >
… a script that just gets a random number between 2-6 seconds…
… this number just trigger a interval eg.
setInterval(changeStock, randomNumber);..
..
and set a new class to the element…
getElementById(“stockDemo”)…

< /script >

< div class=”stock-number” id=”stockDemo”>< /div >
< div class=”stock-text”> In Stock< /div >
}
else
{
<stock-indicator></stock-indicator>
 //Old code that works. If feature is turned off the ordinary code will run. An easy way to turn on off features, you can even turn on feature in dev mode but disable it in production mode…

}

John started the page to see if the script worked as intended. He didn’t create any tests since the code will be deleted as soon the experiment is done.

It worked. So he pushed the code, the CI & CD took care of everything and now the experiment was live.
It took him only 15 minutes.

And the result:

 

Two weeks later:
The PO asked the team how the experiment went. The team showed the PO the monitored data they have gathered. It didn’t show any direct speed increase only 0.5% faster than average. But the conversion rate was higher than before. 1.8%. So the hypothesis with the goal of 5% increased speed and 2% conversion rate did not work. But it still generates 1.8% conversion rate. The PO was not fully convinced if the feature was that good so he wanted to test the experiment one more week.

One week later:
Still the same result 1.8%.

– Ok the experimental code just gave us about 2400$ extra income. That’s nice, lets implement it. Is it possible to get it done within a week or two? (BP)

– Sure, we already have the infrastructure with SignalR since before so we just need to add a new REST API in our Asp .Net Web Api module for the product page. We already have code for retrieving Stock   values in our validation of the cart so thanks to our SOLID approach with SRP (Single responsibility principle) we already have a repository method to retrieve the actual stock value for an articleid.
The style we added can be reused, no need to change that; we just need to add the event handling in our Angular web-component for the stock indicator. It will not take that long either though we only need to act on the event triggered from SignalR and add the new stock value to our view model. Add some code that switch the class based on the stock value. I think with tests we will have this implemented up and running within few days or in a week. Especially thanks to our CI & CD approach and our architecture. (TM)

– Nice. Let’s create a user story and add it to the backlog (BP)

Summary
Sounds too easy right? Like a Utopia? I know, but it’s not. With good agile organization mindset, with well-educated team, with good CI & CD, DevOps and architecture it will be easier than before. As humans we mostly do things too complicated, and thanks to a lack of slack in organizations you don’t have time over for the team to be more innovated and creative in learning.We need time to invent things easier and better than last time. Continuous improvement is so important so don’t waste that benefit.

 

Who is Johan Normén?

Johan Normén is 37 years old, work as a speaker, mentor, team leader, agile coach, and senior .net developer at Softhouse in Gothenburg Sweden. He has over 18 years business experienced and worked in many different projects and roles. Was one of the creators of Swenug (Sweden .Net User Group) with over 3000 members all over the country. He started the computer era as game designer at the age of 12 with his Amiga and team. He has been nominated as the top 10 developers in Sweden in the Swedish version of Computer Sweden 2015

Twitter: @johannormen

 

 

Advertisements

3 thoughts on “Hypothesis Driven Development with example

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s