Last year I attended a conference on diversity and what struck me most is that, finally, the approach is starting to shift away from random, box-ticking initiatives to a systematic, increasingly sophisticated, evidence-based journey. The rising use of data in thinking about diversity is weeding out which strategies work and which don’t, such as, for example, much of the current unconscious bias training. To fix biases in the workplace we must not be afraid to challenge conventional wisdom and to focus on what is effective in achieving results rather than on what looks good on paper.

The FT Women at the Top conference was an excellent opportunity for me to understand how the thinking on diversity is evolving. It is no secret that progress in the area remains slow on many metrics and the latest statistics continue to reveal a depressing picture.

While there was still a lot of focus on the well-known facts and initiatives, what impressed me was that at long last the approach to diversity is becoming more evidence-based. This helps differentiate between things that are effective in changing attitudes and increasing diversity, and things that simply have no impact, or indeed lead to even worse outcomes. This is not only challenging the conventional wisdom on tackling diversity issues, but it’s also highlighting where failures lie and how organisations can improve.

Unconscious bias training is one area where an evidence-based approach is providing useful insights. The training method is one of the most pervasive tools across industries to promote diversity - but it’s also largely ineffective. There’s scant evidence of its success in changing behaviours and, counterintuitively, it could reinforce stereotypes in some cases. For example, the latest data suggests unconscious bias training is, overall, negative for women of colour.

Former NBA pro and and psychologist John Amaechi gave a talk on the evidence around unconscious bias training.

How to measure change

Part of the problem is that measuring the effect of diversity programmes is tricky. How do you quantify attitudes, opportunities and workplace cultures systematically and over time? Employee satisfaction surveys, staff retention metrics, and demographic data throughout the employee life cycle (recruitment, promotion, access to training, etc.) are all useful in theory, but might not provide objective assessments in practice. In most companies, therefore, ‘success’ ends up being anecdotal. Research based on good quality data and sound evaluation methods is necessary here to put the anecdotes in perspective.

Research evidence helps evolve our thinking around diversity and develop more informed approaches. For example, data shows that diverse teams only perform better than non-diverse teams when they are led by competent managers. If management is bad, then diverse teams may not perform any better. As well as creating diverse teams, we need to train managers to build environments in which teams can thrive. Another example is mentorship. Findings show that it’s most effective when the mentor and mentee are diverse relative to one another. In practice, most people are matched with mentors of similar background (think of your ‘formal’ mentors over the years!).

Boardroom disconnect

More evidence is also emerging on the increasing disconnect between leadership and people who are led in organisations. Many CEOs, for example, believe they have the right procedures and initiatives in place. But, for issues such as sexual harassment, this is often not the case. Younger employees tend to feel they lack the power to bring matters to light. Reassessing and overhauling existing policies is required in many organisations to eradicate harassment - this is one area where more research is needed, but, unfortunately, progress is slow.

The increasing use of evidence in designing diversity policies and training is testing our pre-conceptions about what’s effective. While this may lead to fine-tuning some current strategies, we should not be afraid to challenge conventional wisdom and to rethink our approach on a large scale - across firms and industries. There’s still a long way to go to creating workplaces which fully harness the benefits of diversity, but data and sound analysis are crucial tools in that challenge.

But, for all this to work, (most) organisations must start by admitting they are not a true meritocracy because if they are, then their current workforce composition is exactly as it should be! Once this fallacy is eradicated from the minds of top leaders - and from recruitment websites - things might start to shift.

• Kriti Sharma, VP Artificial Intelligence at Sage, was part of a panel discussion on AI and inclusive work cultures at the FT Women at the Top conference attended by Anna Stupnytska.