Bring Your Own Tools…But First, Think Twice

Software teams have more liberty than ever. They have access to a wide variety of tools, and many of them are free. There is an abundance of open-source building blocks available for creating applications. And best of all, many employers encourage teams to set up their own development systems, methodologies, and team members to bring their own tools.

The freedom to use the tools, methods, and technologies you like most and know best is likely to contribute positively to your team’s productivity—up to the point where you hit the limits of the self-containment of the team.

Avoid A Testing Bottleneck

My friend, Jan, was a leader of a very competent software team. They really understood how to be agile. And they had a quality problem. They managed to accelerate their release cycle so well that testing had become a bottleneck. And they cured the bottleneck by skipping testing. Jan, of course, understood what needed to be done. He downloaded Robot Framework, used it to build a custom test automation framework for the team, and plugged it into their continuous integration pipeline. Gradually, he added more and more tests and extended the framework with a large number of high-level keywords to make test scripting easy.

Meanwhile, the other teams—and there was a wealth of them—were experiencing similar challenges and resolved them in their own ways, with Selenium or some tool of their choice.

Teams Need An Automation Framework For All 

Pessimists say that every solution contains a seed of a new problem. Jan soon found out that he’s become a full-time test automation engineer. Sure, his libraries were easy to use—for him, and his test architecture was easy to understand—for him.  In every other team, there was a colleague in the same situation. One of them changed jobs. The developer who took over decided to rewrite everything as she didn’t like the way the automation had been built and found it hard to understand. As many teams began reporting these challenges to their managers, the obvious question arose: why is there no consistency across teams?

Something had to be done. The solution was obvious: let’s establish a centralized test automation team that will serve the development teams by providing one automation framework for all. In practice, the team was Jan and another developer. This is when the struggle really began.

Share Methodology, Not Just Tools

Tests broke frequently. So did the framework. The development teams had a hard time changing their practices to match the tooling, and the tools team had a hard time accommodating all those different needs. The complexity of the framework kept growing. The testing bottleneck had been transformed to a test automation tooling bottleneck.

Shared internal tooling is a beautiful idea, but it’s hard to make it work. To succeed, the tool has to be managed like a real product, and it has to be tested well. Paradoxically, in-house test automation frameworks are often buggy. But there is an even bigger challenge. To benefit from shared tooling on a large scale, you need a shared methodology, too.

The story of Jan is a typical one—and it is not over yet. They are now contemplating their next move: return to autonomous teams and accept the cost, invest more in the shared tooling and establish consistent methodology, or select a commercial tool and focus on the methodology only. None of these choices are wrong, but each of them is going to have different consequences.