Talk: A/B testing DIY
Speakers directory
Speaker:
Marcin Bazydlo
Talk description
Title:
A/B testing DIY
Short synopsis:
Last year we really wanted to get data driven in all our decisions. Unfortunately after reviewing of online offering in that area we concluded that we cannot afford of any off-the shelf solutions due to the amount of traffic we were processing. So we set off on the journey to build our own A/B testing infrastructure It took us six months from first design to first experiment. Decision to make iterative development allowed us to get some early benefits of extended monitoring and feature toggling. Already extensively depending on Azure PaaS helped us to limit our infrastructure effert and drive costs and complexity down. In this talk I will explain architecture of an a/b testing subsystem and how it can be delivered incrementally. I will mention technical and social challenges we had to face and how we overcome them.
Max size: 500 chars
Long synopsis (optional):
It’s usually not good idea to build something on your own unless you must. Last year we really wanted to get data driven in all our decisions. Unfortunately after reviewing of online offering in that area we concluded that we cannot afford of any off-the shelf solutions due to the amount of traffic we were processing. So we set off on the journey to build our own A/B testing infrastructure. It took us six months, with ups and downs, with wider company supporting our efforts and stopping us to a halt. Decision to make iterative development allowed us to get some early benefits of extended monitoring and feature toggling. Already extensively depending on Azure PaaS helped us to limit our infrastructure effert and drive costs and complexity down. On countless meetings we described what is our goal and how we will get there sneaking in one feature into every sprint. Just to get surprised ‘we didn’t know we can do that’ from design team when we came back to them with first early results, angry search team when we claimed that their new fancy microservice doesn’t do better than old hacky implementation and mad account managers when we changed behaviour of the application for their clients without notice. We learned plenty in the process and had quite a fun. In this talk I will explain architecture of an a/b testing subsystem and how it can be delivered incrementally. I will mention technical and social challenges we had to face and how we overcome them.
Max size: 5000 chars
Tags:
Speaker directory:
Listed in directory
Not listed
Speakers directory