The study, which looked at the state of management and leadership in the health sector, found particular problems at senior management levels, where the number of ineffective managers rose to 43%.
The CMI said ineffective leadership is down to a lack of investment in management and leadership development, and the promotion of clinicians into management roles without adequate training and support.
CMI chief executive Ann Francke said in the health sector strong management and leadership is critical.
"This report shows too many health and social care managers are ineffective, resulting in a lack of employee engagement, poor service and low patient satisfaction," said Francke.
"The way we train and recruit health sector managers needs to change."
The study, A Management and Leadership Health-Check, found investment in management skills and support is shown to increase organisational performance in the health and social care sector by 29% and people performance by 23%.
It also suggests a mismatch between the types of training provided and those that managers need. Managers said they are most likely to receive on-the-job training or be placed on an internal development programme, which they rated as least effective. Accredited learning and qualifications were rated most effective by managers polled.
More spending needed: Royles
NHS Employers Organisation chief executive Dean Royles said managers in the NHS are the most "under appreciated" staff in the workforce and called for more to be spent on the training and development of managers.
"This year has seen their numbers cut by thousands, a massive reorganisation, a pay freeze and constant media pressure following the review into Mid Staffordshire, with its implications and the efforts for cultural change that it heralded," said Royles.
"Yet they continue to cope with increasing demands on services and have delivered improvements in staff engagement and reduced sickness absence.
"It is an incredibly complex agenda they have to navigate and I agree that we need to spend more of our resources on their training and development, and to value them more for their incredible service."
