Experimentation before implementation

At a conference last month I spoke to an attendee about how they were using AI in their workplace. It had been ‘rolled out’ they said, but nobody a) knew how to use it properly; or b) what the company was looking to achieve by doing so.

It’s a modern workplace malaise that’s not unique to AI.

Organisations think that by simply implementing a new tool (or process), that this will tick the box marked ‘innovation’ on their strategy. Or that by buying a licence it means that they will somehow immediately reap the benefits. Yet this has never been true.

The only way to ascertain whether we’re ever likely to achieve benefits from something in the way that it’s been sold to us, is to test it first. 

Create a small control group. Ensure that the people testing it are fully trained on what to do and understand why they are doing it. Then run through a series of repeatable steps to assess its ease of use, effectiveness and to actively seek the flaws that may inhibit wider use, undermine value realisation and/or create cynicism loops.

Throughout this trial, everything should be documented, so that at the end of the set period, you can assess whether - or not - it’s a solution that will produce the benefits that you expect. 

If the answer is yes - and it has to be categoric given the time, money and resources it will cost to roll it out - then it should be meticulously planned so that the control group conditions can be replicated.

If the answer is no, then you can satisfy yourself that you did everything you could to ascertain whether it would be something worthwhile to invest in, but ultimately it would never deliver the benefits promised.

Just because a sales person (or tech doom-mongerer!) says that you should invest in a solution doesn’t mean that you should. There should never be any implementation before experimentation.