It said growing demand for emergency care is "unsustainable" and there are not sufficient numbers of staff to deal with it.
The report said emergency staffing levels are an area of considerable concern as only 17% of emergency departments are managing to provide 16-hour consultant coverage during the working week.
The situation is worse at weekends and consultant staffing levels are nowhere near meeting recommended best practice.
The Committee said NHS plans to tackle this problem "lack sufficient energy".
The report also said young doctors do not see emergency medicine as an attractive specialty, and there is a shortfall of senior and future consultants.
"A&E departments are in crisis, but this is not due to an increase in patients," said Stephen Dorrell, the former Conservative health secretary and chair of the Committee.
"The greater issue is the shortage of doctors prepared to do this kind of work when they can have a much easier life by working elsewhere in their hospital or as a GP.
"Only 17% of hospitals have the right number of consultants, leaving emergency care in the hands of junior doctors or locums - this needs to be sorted out as a matter of urgency."
The report recommends Health Education England and local education and training boards take steps to ensure that emergency medicine is both professionally and personally rewarding.
