Purpose Of Experimentation
Experimentation allows us to learn about any new topic of interest more quickly avoiding a dive deep into details.
API Supplier creating a 5-10 minute quickstart introduction describing their purpose provides a good example for experimentation. Developers can evaluate through experimentation if the API is an overall good fit or not before building an actual prototype. If Supplier A shows that their API is a good fit within 5 minutes, while another Supplier B has a rich feature set but their API requires more steps to experiment (i.e. a complicate account creation, or payload format), it is very likely one would choose the API of Supplier A.
Experiments tap into our imagination and creativity. The more we experiment the stronger our imagination becomes. The problem we ought to solve as developers are getting more and more verbose and complex. AI algorithms, datastructures and distributed computing to mention just a few examples. In order to come up with new ways or challenging the existing one, great imagination is key.
Last but not least Experiements are a favourable way to bring junior developers up to speed. They learn how to estimate and work in a scientific way. Eventually they also experience to fail fast.
As software developer we test new territories all the time, otherwise copy and paste existing solutions would be sufficient.
My Five Pillars Of A Great Experiment
As Disclaimer upfront: Don’t expect any magic in the five pillars of experimentation. However, on many occasions in my career I have seen projects failing where theses very basics have not been applied. Projects fail because to many complexity was added in the beginning leading to stress and demotivation on the team; eventually the entire project fails.
Goal
Our motivation seems greater if we work torwards a particular goal. Accomplish something provides a pleasant feeling in return. A goal should be realistic and reachable instead of being a moon shot, otherwise it inverts all the positive effects. Repetition over perfection is key.
Scope
The scope often measured in days, weeks or months ensures that a goal is iteratively revisited. There is no one fits all size, however it should be short enough to achieve a fast-feedback for a future iteration.
Be Resourceful
Experiments should be easy and quick to setup and not-expensive to run. That refers to any type of resource. A given budget or time spent on the execution of the experiment. As lower the cost as more energy and resources are available to improve on subsequent experiments. Developers tend to bring in automation very early. However if building the automation is not absolutely mandatory to evalute the experiment, it can be postponed to a later time.
Don’t Make It A Perfection
Iteration over perfection. One experiment should not stand alone but rather be part of an iterative sequence of experiments. Lessons learned on failed experiments help to improve the next one.
Record Metrics and Results Along The Way
(Success) metrics can be taken during and/or after the experiment. If feasible they get recorded in an auditable and machine readable way (shared document with standardized structure like architecture decision records, git, …). Some could be even recorded automatically(don’t let a human do a machines job). Documenting results of experiments can be a great way to learn what context has lead to a certain architecture. Knowing only about final results leads often to confusion and misunderstandings (best practices and technologies change fast) especially for new team members.