A mature maturity model for continuous delivery including culture and DevOps

Do you love the idea of a model that rules them all? I bet you have, we all have 🙂

“Devops has huge business benefits: Statistics show that organizations that practice devops outperform the SMP500 over a 3 year period, high-performing IT organizations have 50 percent higher market cap growth, and so on.”

I have ben fan of models in my whole life (kind of, since I remember anyway). The idea with a model it that it’s not a process. It just helps you understand things. While a process is how you shall navigate. I’m not fan of processes. I love to create them though, for others. It might sound rude but I’m a creative person that love to be on the top of the wave. Processes can’t make me think outside of the box or live outside it so I just creates them for others that don’t have time or interest to be on the top of the wave all the time. My daily activity in my head loves the ideas of innovation, efficiency – how can we make it easier and still with high quality? How can we do it in such a way that we do not need to do a lot of bad work when we want to change something? These are the questions I always ask myself every day in all areas. I can not stand the feeling to stand still and be smug. I am a thinker who loves changes. I’m curious person.

The last 10 years or so I have always been interested in different working models/processes. Therefore ALM (Application Lifetime Management) became an interesting area for me. Agile processes, gamification and so on.

We have this constant goal and vision to work towards something better. It shall be easy to make adjustments to the code and the architecture. It shall be easy to delete code that is not used anymore and, last but not least it should be fun and innovative approach to work with software development and deliver a smile on the customer’s lips every time. Faster deliveries and keep up with the competition has never been more important than now. Before, it was easier to become complacent and just sell on. Today we cant work this way if we want to keep up.

drastolen

Today we talk about agile delivery, continuous delivery. We’re talking about spending less time on code problems with smart and effective continuous integrations. We talk about the importance to measure value via monitor behaviour etc. We’re talking also about saving time in testing, operations and so on in order to keep up. To help and make managers and buyers less busy, several companies worked to develop various maturity models. With the main goal to be able to run faster delivery and early take part of the digitalisation and its digital transformation taking place today.

I wrote myself a Swedish post on my LinkedIn about five maturity levels for the  transformation regarding better continues delivery benefit.

Level 1 – A few develop are the organisations super heroes (Ad hoc solutions)
Level 2 – Time-Boxed releases
Level 3 – Regular deliveries
Level 4 – Release Command
Level 5 – Hypothesis-driven development

I recently wanted to find a model that explains all this in a more technical level. To help companies to identify where you are in the area of DevOps, test, code, culture, processes and so on. I have read four different maturity models from four different companies. But I didn’t like them. Most of them feel like they were created based on other companies’ problems and not based on a general level. A colleague of mine gave me a link to a model that actually is a great addition to the five levels I previously presented.

The model comes from the company Praqma. Not only that it is presented nicely, it is also spot on based on my thought and experiences in that field. The expert step is a great addition to #NoEstimate, Lean Startups with Hypothesis-Driven-development/Design and Pain-Driven-Development/design as the level 5 above.

cd-maturity

You can read more about it here
http://code-maturity.praqma.com/

If you are interested how you can work or how a system need to be designed and what transformation you need to do to reach the expert level, don’t hesitate to contact me or read my blog and forthcoming posts.

Who is Johan Normén?

Johan Normén works as a speaker, mentor, team leader, agile coach, and senior .net developer at Softhouse in Gothenburg Sweden. He has over 18 years business experienced and worked in many different projects and roles. Was one of the creators of Swenug (Sweden .Net User Group) with over 3000 members all over the country. He started the computer era as game designer at the age of 12 with his Amiga and team. He has been nominated as the top 10 developers in Sweden in the Swedish version of Computer Sweden 2015 and member of IDG Expert Network Team

Twitter: @johannormen

Hypothesis Driven Development with example

My previous blog-post was an introduction to HDD (Hypothesis driven development). In this blog post I will take it a step further and give you an example how you can use it and how easy it can be and how it can change the way you work entirely to a new level.
You can read it [here].

Remember:
HDD is an experimental driven process, it’s main goal is to find out if a certain idea generates ROI and happy users. To get fast result you will also need fast support to evaluate your experiment, if you want to evaluate the idea in production I prefer a smooth decoupled architecture and a mature CI & CD organization. It’s not a must to, but it will really decrease development time and cycle time.
Sadly, I have no maturity model written in English yet. You can read one I did in Swedish on my LinkedIn posts [here]

Let’s start the story.

Team:  A cross-functional team with talent people regarding:
Design, UX, Coding, Testing and so on. And also including some market people and product owner.

At a meeting with the customer and team a team member (TM) got this cool idea regarding an e-commerce platform.

– What if the stock value changes in real time? Wouldn’t that be cool? Maybe it increases the conversion rate? The users might be stressed that this nice shoe will get out of stock?

A business person (BP) responds:
– What will it cost? Can you estimate?

– I don’t know, and I don’t. And I don’t know if my hypothesis even works. Can’t we skip this estimating and cost discussion and find out and then see if it’s worth implementing at all?  (TM)

– How? (BP)

– Let’s just do some A/B testing and fake a simulation of stock value changes and monitor if the user will buy the product more often or faster than today? If we get an 2% increased conversion rate it really worth implementing isn’t it?  (TM)

– Yes, you got a points there, this Sounds interesting. (BP)

Instead of creating User Stories the team now write Hypothesis stories.

Real-Time stock value changes
We Believe That
real-time stock values on the product page
Will Result in improved customer engagement and conversion
We Will Know We Have Succeeded When we see a 5% increased speed and about 2% conversion increase from the time the stock change in real-time till they press the add to cart button.

The meeting generate lots of hypothesis stories and the product owner can now prioritize them based on the ROI for the result outcome.

– A 2% increased conversion rate is really nice. Today we got like 5000$ each day, 2% will give us about 100$ more. In a month it’s around 3000$. If the experiment works and generate this value; You can spend around 30h of development time and we still get more money than not implement it at all. (BM)

– Yes you can say that, but let’s not convert it to time yet, let’s see if it’s worth implementing at all. Because if we spend around 10-30 hour for this feature and it will not increase conversion at all; we just wasted lots of money for developing a crappy feature. Let’s be smart and just spend time creating features that generate values in short and long runs instead. Assumptions is not a good thing. (TM)

– Ok, we can give it a try, because we have done so many bad assumptions we even can’t monitor at all. (BP)

The Product owner decided to activate the experiment and tells the team to do the fastest possible implementation to get information to see if this hypothesis will generate values at all.

The team gathered as usual at a stand-up and activated the work of the hypothesis.

– So what is the fastest possible solution to get rapid feedback regarding this? (TM)
– We can ask the customer?
– Nah, the customer might say, cool feature or – Do you want to stress us even more?
– Ok, let’s implement a test in our code and monitor the outcome?
– Sure, what’s the fastest way?
– This is part of the UI so let’s do it simple with some JavaScript. The only thing we need to understand is if it works. So there is no need to make it perfect. We do not need to add back-end functionality to get this information either. I can type a JavaScript that just randomly decrease the stock-values on some products. And then monitor with our application-insight code if they will press the button faster or not. It doesn’t matter if the stock value is a faked value for a while… If the value is 5 or 4 doesn’t matter that much.
– Nice, cool this will take us around 15 minutes or so? And thanks to our CI  & CD approach the feature will be out within 20-30 min or so and we can then start monitor it. If the result is good, then we implement it all the way with tests and quality in mind.
– Cool, this is nice, if the PO asked us to estimate it and add management work around it and do all the other stuff we do for a user story it would take longer cycle time than 30 minutes for just get started on this one. We aren’t wasting anything here, just getting knowledge and information if the idea is worth implementing or not. We will save around 15 minutes and also get the information if it’s even worth implementing. It’s much better than assume it does by implement it and hope it will generate values.

The team agreed.

Liza loved the idea regarding NoPSD so instead of spend time on working with PSD files or other tools for design she just opened the SASS file (css manipulation) and added three new classes with three different color. Green for normal stock values, Orange that indicate near out of stock and red that indicate very soon out of stock.

.stock-Normal
{
     color: #38893d;
}.stock-Warning

    color: #fe8f01;
}
.stock-Critical

    color: #be0000;
}

Thanks to nice SASS structures it just took her 2 minutes. When she pushed the code to the source control it automatically generated a new css file that got released into production, this because the team had nice CI & CD approach. (Why not release it? It will not hurt anything though nothing using the classes at the moment anyway.)

The front-end was built on AngularJs and the stock area was created as a web component and rest of the system is based on ASP.Net MVC and Web Api. Meanwhile Liza added the css John added a feature toggle in the code and took the html from the web component template and added it to the new feature for the hypothesis.

The feature toggle framework they use uses one configuration file to indicate if the feature is on and off.
They simple add it as an appSetting in web.config or appSettings.json (if using ASP .Net 1.0 Core).

“IRLStock” : “true”;

The team uses a framework that uses a C# class as a feature toggle so they easy can remove the class and get compilation error where the feature existed, so they do not forget to remove the code later on.

public class IRLStock {}

The code with razor syntax (ugly code? I know but much faster than create a new web-component to get the result from this experiment, it’s important to not waste unnecessary time for a more complicated solution, then you missed the whole point. This code will be deleted after the experiment anyway. For experiments it’s just waste to add sugar to it.):

@if(Feature<IRLStock).Is().Enabled)
{  
     < script >
… a script that just gets a random number between 2-6 seconds…
… this number just trigger a interval eg.
setInterval(changeStock, randomNumber);..
..
and set a new class to the element…
getElementById(“stockDemo”)…

< /script >

< div class=”stock-number” id=”stockDemo”>< /div >
< div class=”stock-text”> In Stock< /div >
}
else
{
<stock-indicator></stock-indicator>
 //Old code that works. If feature is turned off the ordinary code will run. An easy way to turn on off features, you can even turn on feature in dev mode but disable it in production mode…

}

John started the page to see if the script worked as intended. He didn’t create any tests since the code will be deleted as soon the experiment is done.

It worked. So he pushed the code, the CI & CD took care of everything and now the experiment was live.
It took him only 15 minutes.

And the result:

 

Two weeks later:
The PO asked the team how the experiment went. The team showed the PO the monitored data they have gathered. It didn’t show any direct speed increase only 0.5% faster than average. But the conversion rate was higher than before. 1.8%. So the hypothesis with the goal of 5% increased speed and 2% conversion rate did not work. But it still generates 1.8% conversion rate. The PO was not fully convinced if the feature was that good so he wanted to test the experiment one more week.

One week later:
Still the same result 1.8%.

– Ok the experimental code just gave us about 2400$ extra income. That’s nice, lets implement it. Is it possible to get it done within a week or two? (BP)

– Sure, we already have the infrastructure with SignalR since before so we just need to add a new REST API in our Asp .Net Web Api module for the product page. We already have code for retrieving Stock   values in our validation of the cart so thanks to our SOLID approach with SRP (Single responsibility principle) we already have a repository method to retrieve the actual stock value for an articleid.
The style we added can be reused, no need to change that; we just need to add the event handling in our Angular web-component for the stock indicator. It will not take that long either though we only need to act on the event triggered from SignalR and add the new stock value to our view model. Add some code that switch the class based on the stock value. I think with tests we will have this implemented up and running within few days or in a week. Especially thanks to our CI & CD approach and our architecture. (TM)

– Nice. Let’s create a user story and add it to the backlog (BP)

Summary
Sounds too easy right? Like a Utopia? I know, but it’s not. With good agile organization mindset, with well-educated team, with good CI & CD, DevOps and architecture it will be easier than before. As humans we mostly do things too complicated, and thanks to a lack of slack in organizations you don’t have time over for the team to be more innovated and creative in learning.We need time to invent things easier and better than last time. Continuous improvement is so important so don’t waste that benefit.

 

Who is Johan Normén?

Johan Normén is 37 years old, work as a speaker, mentor, team leader, agile coach, and senior .net developer at Softhouse in Gothenburg Sweden. He has over 18 years business experienced and worked in many different projects and roles. Was one of the creators of Swenug (Sweden .Net User Group) with over 3000 members all over the country. He started the computer era as game designer at the age of 12 with his Amiga and team. He has been nominated as the top 10 developers in Sweden in the Swedish version of Computer Sweden 2015

Twitter: @johannormen

 

 

Why Microservices and continuous deployment (delivery) is win / win

Microservices is nothing new but has become a standard in many systems during the last years. There have been trends in architectures some engineers use them as a silver-bullet. They used the Architectures even if not required by system nor requirements. DNA, 3-teir, SOA, DDD, CQRS, Onion architecture and so on. Is Microservices just a trend that will destroy systems because people will use it as a silver bullet? Maybe! The future will tell. Let’s investigate why microservices will be a good benefit for the continuous delivery approach.

 

As most of you know by now, is that continuous delivery (CD) is an approach to deliver functionality in a smooth and almost automatic throughput. CD is an extension to Continuous Integration (CI) that are the configuration that tells you something will happen as soon you check in your code in to your source control. It can run tests, if it fails it will not publish the system or do something else that’s important four you and your business. I use CI to tell the build server to build my system, run some test and send it to production with continuous deployment.

The different between Continuous Deployment and Delivery is just that deployment automatically deploy your code when delivery just give you a package that you can manually deploy

softhouse_CD_loop2_new3

As always it is requiring discipline in your mindset as well in your code. It will be forced since you release everything in to production. So why is microservices a win/win with this approach?

The opposite to microservices architecture is the monolithic systems, it’s a system with a server where all your business capabilities are grouped into one big product. With continuous deployment and delivery, we just need to release lots of code and run lots of tests, this can be tricky for the benefit of a smooth delivery. Maybe you need to migrate your database, change many contracts and increase a big and complex domain model every time you will push some new feature to your CI/CD configuration. You can save you lot of problem with feature toggling but you still need to release the whole system. This is when the micoservice architecture will save you from lots of headache. Micorservices are like small systems with their own architecture, components and tools needed for your business capabilities. I often explain them as CRUD services for different business areas like a product service, invoice service, order service, authentication service and so on. I also like to refer to them as a SOLID principle architecture on an architectural level rather than just on the code level.

Microservices shall have single responsibility (S) for its business area, it shall be designed open for extension (O), like version controlled REST APIs. A new version extends its functions. With backward compatibility you will have a subtypes of the API (L) and so on.

scaling_agile_model1scaling_agile_model1

Microservices allows you to deploy specific services instead of the whole application when minor changes is applied. You simply deploy that service and only that service code. If you use containers like Windows Containers or Docker you have even better CD benefits though you don’t need to replace any files on the already running service, you just deploy a new container and can easy activate that one as soon you feel comfortable doing so. With other tools like Azure service fabrics you will get even more nice supporting tools for crashes, rollbacks for versions if one deploys fails etc…

Microservcies also benefit from less merge problems despite the fact that it is simple to use different microservices within your team. Discipline implementation of feature toggle minimize the feature branches so you really go Lean with the whole architecture and spend less time on configuration management.
Summary:
– Less code within the codebase

– Less problem with merges if using feature toggling, version handling instead of     branches

– Less problem with deploys though you now deploy the whole system, just a   small part of it

– You will have benefits regarding scaling each heavily loaded service instead of scale up the whole system

And lots more… But remember, this is not another silver bullet, it’s just a bullet of many

Who is Johan Normén?
Johan Normén is 37 years old, work as a speaker, mentor, team leader, agile coach, and senior .net developer at Softhouse in Gothenburg Sweden. He has over 18 years business experienced and worked in many different projects and roles. Was one of the creators of Swenug (Sweden .Net User Group) with over 3000 members all over the country. He started the computer era as game designer at the age of 12 with his Amiga and team. He has been nominated as the top 10 developers in Sweden in the Swedish version of Computer Sweden 2015

Twitter: @Johannormen