Measuring change programmes' effectiveness
Jenny Roper, March 01, 2018
Tosca Bruno-van Vijfeijken and Richard Eastmond talk measuring change programme effectiveness
When HR magazine caught up with Richard Eastmond, senior director of people and organisational services at Amnesty International, back in 2013, he was in the middle of a transformation programme that would faze a more faint-hearted HRD.
Through a global transition plan the structure of the organisation was being decentralised, with staff in its London headquarters redistributed to new international regional hubs. The overarching aim was to improve the human rights impact the organisation delivered through recalibrating a model that, in line with many other large NGOs, had traditionally centred around the global North dispensing human rights support to the global South. Now the emphasis is on regional teams tackling human rights issues they’re expert in as locals.
This represented the most significant shift in the organisation’s then-52-year history. Winning staff trust and belief in the need for, and positive potential of, change was a key challenge. Fast-forward four years and the organisation’s made impressive progress in bringing people along with the change, and crucially improving Amnesty’s impact tackling human rights abuses globally.
But ensuring successful change at such an organisation can never be about setting it in motion and then sitting back, says Eastmond. Crucial is evaluating what has and hasn’t worked and where the organisation needs to go next.
For this Amnesty enlisted Maxwell Syracuse University, in New York, who it’s partnered with in various ways over the last eight years. The university analysed the change project two years in, in 2015, and again this year – a process all change projects could but often don’t benefit from, says Eastmond.
Tosca Bruno-van Vijfeijken, director of the Transnational NGO Initiative at the university and leader of the team evaluating the Amnesty project, explains the methodology used. “We insisted on full access to internal documents; including confidential ones,” she reports.
“We used a combination of surveys on an anonymous basis, questions on the efficacy of the change in the staff engagement survey… then we did about 50 interviews with individual employees, and a survey with external peers and partners. Finally we attempted to talk to a selection of campaign targets.”
So what did Syracuse find, why is such a review important, and what key change principles did it shed light on? To find out HR magazine caught up with Eastmond and Bruno-van Vijfeijken...
Tosca Bruno-van Vijfeijken: We work with quite a few INGOs [international non-governmental organisations]. Our role has been to help them analyse: what has been the efficacy of their change management?
Richard Eastmond: Culturally, not just at Amnesty but at most charities, because we’re dealing with donors there’s a piece on evaluating why we did what we did, and did we spend money correctly? We have a whole team called our strategy and evaluation programme; a big part of what they do is institutionalising learning and evaluation. So it’s natural that when we undergo the largest single investment Amnesty has made in its 56-year history, we had to put in the mid-term and final assessment. The commercial world does change projects and then just moves on; whether they failed miserably or succeeded spectacularly. So I think very few organisations do this in any structured way.
TB: The final assessment was not just about the change management and its effectiveness but also the effect on human rights – so impact on the ground. That’s an important compositional dimension that made it even more challenging… We tried to be as rigorous as we could and used multiple methodologies. But there will always be a subjective element. We primarily relied on staff perceptions. We tried to assess those impact perceptions as carefully as possible to triangulate different methodologies and we introduced some external interviewing of peers and partners. But rigour is always a relative term.
RE: It’s taking a practical perspective of: ‘you could critique it on this basis, but as a route to holding the mirror up it’s a good compromise, given the resource we have available’.
TB: It’s important to realise the change process had a large amount of contention involved – both with the direction of travel under the global transition plan and how it was implemented from a change leadership point of view. This was both a politically sensitive and high emotion change process – therefore our assessment of it was too. That was palpable at times.
RE: One of the key frameworks Tosca used was Bosman’s four frames. You should look at change from a political, symbolic, HR and structural perspective. We discovered we had a lot of the HR bits sorted, we had the structural bits mainly sorted, we thought we had the political bits and stakeholder management sorted. But there’s a huge amount in the emotional space [we didn’t have sorted], which was where the challenge came.
TB: That emotional space can often get neglected. The symbolic frame is: what kind of meaning do people find in the organisation and the work they do? It’s how they constantly look at leaders as being on stage – so their narrative and stories and behaviours really matter. There are always negative and positive resistors. There are people who say: ‘I really don’t want this change…’ Then there are the positive resistors who say: ‘I agree with the direction of travel but not how the change management team is going about it’. I think getting those two confused is quite easy.
RE: I think that’s the differentiator between the commercial and the INGO worlds. Because in the commercial world there’s this profit prerogative; people march in, there’s little consultation, somebody’s come up with a cunning plan and the business is delivering it. Whereas in the INGO world we look at participative approaches. So we have to get everybody intellectually on board to have a fair wind of delivering.
TB: Another thing I think was important was the difference between leading and cheerleading. This relates to negative and positive resistors. We understood very well why the change team at the top had to be clear saying: this is going to happen, this is non-negotiable.
But you don’t want to go into cheerleading mode where, when people voice their concerns, you’re constantly advocating.
RE: The thing we had most difficulty accepting as a senior leadership team, in terms of the outputs of the Syracuse work, was this cheerleading/advocacy piece. We get the concept intellectually and see its relevance. But we find it difficult to see from a practical point of view how we could have done it any differently, given the huge countervailing forces we were dealing with.
TB: If you see culture as a set of unspoken belief systems and assumptions then the global transition programme was a shock to the system. It was a bigger cultural shock than I think some of us understood. One thing I would want to watch for is the tenure of new people. Are they sticking around because they feel there is a lot of potential but also because there aren’t constraints from a cultural perspective?
RE: Yes, certainly as a measure of success of the stickiness of a culture, that’s a core thing absolutely.
TB: The programme has obviously brought more diversity in terms of more types of staff… But one of the things from the literature we think about next is that diversity is more than getting people who have different profiles or social identities. It’s also a question of to what extent do you allow those people to change the very definition of what the work is, the ways of working, and eventually the culture of the organisation? I’ll be looking for: has Amnesty changed not only the formal ways of working but also the informal? I emphasise that because I think the old Amnesty was very formal because of the heavy influence of the legal profession on the organisation.
RE: We were a very legalistic, written, meetings-driven culture. Equally we’re conscious we’re a very Anglo-centric organisation, so the sort of leadership traits that work tend to be very Western-biased. That’s something we’re trying to get to grips with. So not bringing someone new in from Asia and just thinking because they’ve worked internationally they’re going to be able to operate in our environment.
We’re doing well though. We’ve got far more managers from regions in which they’re working. We’ve pushed ourselves on the advocacy/cheerleading piece. We’ve pushed ourselves on the four frames.
There’s a lot of nitty-gritty stuff that underpinned our change programme that we’re now tweaking, changing, and adapting under the bonnet. So there’s always more to do.
If you’re an HR director and would like to be featured in conversation with an academic email email@example.com to express interest