A/B testing is jargon for a randomized experiment with two variants, A and B, which are the control and treatment in the controlled experiment. It is a form of statistical hypothesis testing with two variants leading to the technical term, two-sample hypothesis testing, used in the field of statistics.
It’s really hard to change or add something and know if it’s a success or a failure before we have test it on real users. In Lean UX it’s important to do this test process as cost effective as possible by decrease unnecessary waste without skipping important work. The best feedback is when you let people use it and see if they like it or not. You can test wireframes, design sketches but that’s not the same as test It for real in real action. Many users will see this cool new thing as a sketch and think; that looks really cool! but is it really that cool when it’s implemented and live in action? Not all the time, maybe some times but we can’t be really sure before we can test it. In Lean UX A/B testing is one approach that can help you get early and rapid feedback if the idea is great or just a new pain for the user. The core idea of Lean UX is to aim small and hopefully shoot big. In other words, do as little as possible and test it before you spend too much time on your ideas.
Let’s see how ASP .Net MVC can make this by just add very little code to make A/B testing fun and easy. 🙂
I love to use the creativity to think; – how can I add something simple that works and also gives me many other possibilities? It’s some kind of evil obsession I have and that’s also what I love most about software design. In this case I will show you how you can add some code to make your page not only A/B test friendly you can use this idea for other things like themes, change layout bases on personalisation and you can even use it to give your user some luxury in e-commerce sites and so on.
In the software, there are thousands of ways to solve the A / B testing. What I found very useful that also decrease waste from more complex solution is to copy the view you want to test to a new location and add changes to that new file, and then let ASP .Net MVC show just that new view based on some business rules. I also want less code, and it shall be easy to remove my test-view without too much work. Therefore, I want to use the view folder and it’s files and not manipulate controllers if not needed. It’s more waste to add new controller if you can use the the you already have when you want to test another layout. My approach doesn’t stop you from A/B test controller as well if you want to add that feature in the future. Remember I’m fan of Agile and Lean approach and principles like KISS (Keep it simple stupid) and YAGNI (you aren’t gonna need it). That’s why I keep things as simple as possible and expand it if more advanced featured is needed later on.
This is my goal:
Let’s say you want to test your home view, you can then easily add a new folder inside the view folder for example “B Testing” and add your home folder and its view in there. And then add some changes to it… DONE! Cool right? In less of a second you just made a A/B tested home view. Sell it for 5h of work 😉
To make this possible I want to let my view engine look for views in special folders but fallback to the original if there is no files in this special folder. ASP .Net MVC do this by default. Like this setup:
I call the variable theme because I can use it for other purpose than just A/B testing too.
Lets create this
Go to your ASP .net MVC project and add a new class that implements the interface IViewEngine from System.Web.Mvc. And implement the view engine code.
Then add the folder locations within a method. I call mine CreateThemeRazorViewEngine
Then I create a RazorViewEngine and configured the Partials, View and Master locations to the new folder structure of mine. In this case ASP .Net MVC razor will act as normal but start looking for files in my theme folders if they exist.
I love the factory and provider pattern so I will use those. As factory I will use DI/IOC framework, I like Autofac but you can use whatever you like the idea is the same.
I start by creating one IViewThemeManager interface with one method GetTheme. This interface is the one we use on all the ViewManager providers that will handle all the business rules based on why you want ASP .Net MVC call for another view or partial.
And then I implementation my factory method to the ViewEngine with the IOC and some error handling.
If there is no IViewThemeManager provider configured it will give me null and return the default Razor view engine. That’s why I use the _fallbackViewEngine. If there is an IViewThemeManager provider registered I ask for the Theme which in this case is the folder name I want the view engine to use.
After that we need to tell ASP .Net MVC what Engine it shall use so we need to add some code for the methods implemented from IViewEngine
In this case I call my CreateViewEngine that will return either the default engine or mine.
Now go to Global.asax and clear all View Engines and add this new one in Application start.
We can now run the project and everything will work as normal. This because the resolver can’t find any IViewThemeManager provider yet and fall back to default.
Now it’s time to add the A/B testing business rules. In this case I want to know if the “Learn More >>” button gets more clicks with another text. So I want some user to get “Learn more >>” and others “Want to learn more?”
First I copy the index file in my home folder to a new home folder under “B-Testing” and just change the text on the button nothing more.
Then I create my IViewThemeManager provider. My ABTestingViewThemeManager.
After that I will add some simple rule. In this case I just send them to my “B-Testing” view folder if the date time second contains number 2. (Just for the test, ok? Real A/B testing might need more complex logic and maybe even som settings, depends on how important the function is, remember don’t make it to complex if not needed. KISS 🙂 )
After that I configure my interface and provider with autofac.
And when I run the page and if the DateTime.Now.Second contains number 2 it will use my B-Testing view instead.
I also added the view rules for partials so you can just test a partial as well if you want to, you don’t need a full view, you can use the default one that just calls for your partials.
Just copy your A/B partial to “B-Testing/<your controller folder>” and this code will take care of that too.
And if you use partials to render templates for example angular you can even test unique directives and web components if you want to. Just add the angular template in your partial and copy just that partial and its folder structure to the “B-Testing” folder or what name you prefer.
You can also add a provider that will give some user a whole new view, layout or webcomponents layout if they are lets say “pro users”. Just add a pro user folder and create a new IViewThemeManager provider that check if the user is of the role pro or something like that.
Happy play around.
Who is Johan Normén?
Johan Normén is 37 years old, work as a speaker, mentor, team leader, agile coach, and senior .net developer at Softhouse in Gothenburg Sweden. He has over 18 years business experienced and worked in many different projects and roles. Was one of the creators of Swenug (Sweden .Net User Group) with over 3000 members all over the country. He started the computer era as game designer at the age of 12 with his Amiga and team. He has been nominated as the top 10 developers in Sweden in the Swedish version of Computer Sweden 2015