Nicholson, who has held his position for seven years, was widely criticised after the public inquiry into the Mid Staffordshire NHS Foundation Trust in February, which found serious failings in public care.
He spent 10 years in charge of the local health authority in 2005 and 2006 at the height of the problems.
In his resignation letter to Professor Malcolm Grant, chair of NHS England, Nicholson said: "I have only ever had one ambition and that is to improve the quality of care for patients.
"My hope is that by being clear about my intentions now will give the organisation the opportunity to attract candidates of the very highest calibre so they can appoint someone who will be able to see this essential work through to its completion."
Nicholson, who has worked in the NHS for 35 years, will walk away with a pension of nearly £2 million when he retires, according to NHS figures published in March last year.
Health secretary Jeremy Hunt said: "Under Sir David Nicholson's leadership, NHS waiting times have fallen, infection rates reduced, and mixed-sex accommodation is at an all-time low.
"His job has often been incredibly complex and very difficult, and yet he has always had a reputation for staying calm and maintaining a relentless focus on what makes a difference on the NHS frontline."
The country's largest union Unite has welcomed Nicholson's retirement. Its national officer for health Rachel Maskell, said: "It is right that he steps down as chief executive of the NHS Commissioning Board.
"He should have resigned earlier this year over his role in failing to tackle the abuse of patients at Mid Staffordshire, highlighted by the Francis report.
"We believe that Sir David's job with its heavy responsibilities should not rest with a senior NHS bureaucrat - but with the secretary of state for health, Jeremy Hunt."
She added: "It is hoped that Sir David will not leave with a massive pay-off - as that would just be an insulting reward for failure."
