You’re a busy HR director. The phone rings and it’s an experienced consultant, who enthusiastically tells you about an exciting new idea to improve employee performance. At a meeting he is more forthcoming: it has no agreed definition; there is no evidence it can be reliably measured; it’s almost identical to ideas that have been around for 50 years; and there is currently no quality research showing that if you implement this idea it will increase performance.
Employee engagement shares precisely the same characteristics as the idea presented by this consultant, but curiously it hasn’t put off the HR community. Far from it. But are these and other problems with employee engagement really so bad? I believe they are. What’s even more worrying is that many HR professionals don’t seem to know or care about them.
No meaningful definition
There are dozens of definitions of engagement. Some focus on behaviour, or attitudes, some on feelings and others on the work environment. Many definitions use combinations of these or characterise it as a situation in which one of these things, such as attitudes, affects another, such as behaviour. When it comes to interpreting engagement it seems that anything goes.
If confronted by this analysis HR practitioners often say “but it doesn’t matter what you call it”. I don’t get this. Firstly, there is no “it” as engagement does not mean one thing but many different things. Secondly, it seems to me that a concern for precision and the clarity of thinking that it brings is the hallmark of skilled practitioners in any field. So surely it should matter what you call things.
This definitional mess must not be ignored or trivialised but should profoundly trouble us all. Without a clear and agreed definition of engagement we literally do not know or understand what we’re talking about or doing.
No valid measures
Producing valid, reliable and hence usable measures of any phenomena depends on several things. There needs to be clarity, not confusion, about definition and the phenomenon needs to be different from those already known. Engagement is definitely not such a thing. As a consequence there is almost no evidence that any measures of engagement are valid or reliable.
Predictive validity is important for almost any measure. If you measure something at one point in time does it predict something important at a later point in time? I do not know of any publicly available, good quality evidence showing any measure of engagement predicting any measure of performance.
Of course, it may be that in the future such evidence becomes available. However, given the similarity between engagement and job satisfaction it would be surprising if engagement predicted performance any more strongly than job satisfaction (which is only sometimes and not very much).
It’s nothing new or different
If we don’t want to be taken for a ride, it’s essential that we scrutinise any idea that claims to be new and different. Is it worth taking seriously? Or is it just a re-branding of old ideas? We already know that definitions of engagement are almost identical to existing ideas such as job satisfaction or commitment. And, as discussed, there is also some evidence that engagement and job satisfaction are highly correlated.
Is engagement old hat? Or is it really something new and different? There are two simple possibilities.
- No. Engagement is not a new and different idea. It should therefore be dropped immediately because using a new term to describe existing concepts is confusing, unhelpful and does not add any value.
- Yes. Engagement is a new and different idea. If so, then proponents of engagement need to define it in a way that shows precisely how it is new and different, and gather good quality evidence to show that measures of engagement are indeed measuring something new and different.
At the present time there is no evidence that clearly demonstrates that engagement is a new or different idea.
No decent evidence
For any question we’re trying to answer we look around for information or evidence. When we find it we don’t always take it at face value, but think about how reliable or trustworthy it is. Not all evidence is created equal. It’s vital that we consider how good each piece of evidence is so we can pay more attention to the better quality evidence and less attention to evidence of a lower quality.
Suppose you’re going on holiday in a few months to a place you’ve never been before. You want to know what temperatures to expect. You first ask a colleague who went there a few years ago. You then look up the temperature recorded there yesterday. Next you find a website showing monthly and daily temperatures averaged from records gathered over the past 20 years. Which source of evidence is going to be most reliable? It’s fairly obvious.
The same is true for the questions we ask about engagement. The two most important or fundamental points we need to consider are:
1. Do increases in engagement actually cause any increases in performance?
2. Do engagement interventions increase engagement levels and subsequently increase performance?
In other words, does engagement do anything and, if so, can organisations do anything about engagement? If we can’t confidently answer “yes” to both these questions then employee engagement is (or may turn out to be) a fairly worthless idea. So, what types of evidence allow us to be confident about its effectiveness?
At the Center for Evidence-Based Management we consider which types of evidence might provide reliable and less reliable answers for different types of management questions. Evidence-based management is about acting on the best available evidence, which means first identifying the reliability of each bit. In the case of our two fundamental engagement questions there is a very clear hierarchy.
- Expert opinions, anecdotes, success stories: This is the very lowest quality evidence, yet almost all the evidence we have about engagement fits into this category. Such evidence can be interesting but it is not trustworthy. What is important is the evidence itself not the opinions of people who often have vested interests in it.
- Commercial non peer-reviewed consultancy research reports: There is also a very large quantity of this type of evidence. In relation to our questions this too is low-quality evidence as it is likely to be biased and has not usually been subject to external scrutiny. This is similar to the evidence published by drug companies about the efficacy of the drugs they sell.
- Cross-sectional studies: Though more trustworthy, particularly if peer-reviewed, cross-sectional studies don’t tell us about cause and effect and therefore results from such studies are not relevant to our two questions.
- Longitudinal studies: Probably the best quality type of evidence as these could, in principle, untangle cause and effect and answer the questions. At the present time there are no published longitudinal studies of employee engagement.
So, does engagement do anything and, if so, can organisations do anything about engagement? I don’t think we can answer this question as all we seem to have is a lot of poor quality evidence and no good quality evidence.
Overhyped claims
It should come as no surprise that claims routinely made about employee engagement are at best sexed-up exaggerations, and at worst plain wrong. Such over- and mis-claiming can be found in many places – particularly in popular management and consultancy writing. Here is one example taken from the Engage for Success website.
“Despite there being some debate about the precise meaning of employee engagement there are three things we know about it: it is measurable; it can be correlated with performance; and it varies from poor to great…”
The first claim, that engagement is measurable, is correct in that anything can be measured. But that doesn’t matter. What is important is whether those measures are reliable, valid and therefore of practical value. As discussed above, there is no clear evidence that measures of engagement are reliable.
The second claim is that engagement is correlated with performance. Perhaps. But so what? From a practical perspective knowing about correlations is of no value. What we need to know is whether engagement is an important cause of performance. I’m not too sure what the third claim means. But even if engagement scores do vary from low to high, whether that means engagement varies from poor to great depends on whether engagement scores predict anything important – which is unclear.
Engage for Success has gone beyond these rather vague claims by publishing a report called The Evidence. If you are seriously interested in the evidence for engagement I urge you to read this report.
I found that the research cited to support many of the reports’ most important claims did not actually provide direct or clear support for those claims. For example, claims about cause and effect used results from cross-sectional studies and many of those cited did not even measure or mention engagement but rather other concepts such as commitment and satisfaction. But don’t just take my word for it, take a look at the paper I produced, available on the Engage for Success website.
Such over- and mis-claiming found in the Engage for Success report is not at all unusual and can be found in much of the literature produced by engagement advocates. In general then, many of the claims made by proponents of employee engagement appear to be exaggerated and use supporting evidence that seems to be about something else.
So what to do?
Do we want to take employee engagement seriously, or not? That’s the simple decision we need to make. Taking it seriously means asking and trying to answer the sorts of questions I’ve addressed here. Not taking it seriously means accepting the definitional mess, being relaxed about whether it can be measured, claiming with little basis that it’s new and different, ignoring the fact that there is an absence of decent evidence, and not bothering to check whether claims about engagement have been sexed-up. So what’s your decision going to be?
Rob Briner is professor of organisational psychology at the University of Bath's School of Management. He is also one of HR magazine's Most Influential UK Thinkers.