Sweeney was speaking to HR magazine after it was announced he would be leaving his post at the end of this month after six years at the helm of the employment relations advisory service.
"I have a passionate view that HR is undervalued because they're simply seen as a functionary of the chief executive - 'go there, do this'," Sweeney said. "They should be right at the coal face of policy division and policy implementation.
"The problem lies in our middle management structure, not because they're poor middle managers, but the fact that so much is heaped upon them.
"I remember the big change when HR moved everything down the line and that sounded great but that also meant all middle managers needed reasonable HR abilities. I don't think they have that and I don't think they ever wanted it."
Former TUC chief executive Brendan Barber will replace Sweeney. He will serve a three-year term.
Sweeney was awarded a CBE for services to employment relations in the New Year's honours list.
