This week more than 200 workers at Wigan's Hovis bakery began strike action in a row over contracts and pay.
The union claims full-time staff have been made redundant and replaced by agency workers on zero-hours contracts.
Hovis said limited use of agency labour was "commonplace".
Draper told HR magazine that the union had been given assurances the use of agency workers would be a "last resort" and only used when there are peaks and troughs in the business.
"For me it's just an excuse to use cheap labour, that's what it comes down to," said Draper.
"In April, 26 redundancies were made and these were replaced by agency workers on zero hours contracts."
Draper claims a number of agency workers have recently been offered permanent positions over those who have been served with redundancy notices.
"This sort of practice just allows for the continued casualisation of labour," he said.
Premier Foods response
After being contacted by HR magazine Premier Foods (which owns the brand) failed to respond to Draper's claims but said in a statement the use of agency labour to cover seasonal peaks, holiday and sickness absence is an "integral" part of the business.
"It is not, and has never been, our intention to replace full-time staff with agency labour at our Wigan bakery," the statement said.
"As a matter of principle, Premier Foods has moved away from the use of zero-hours contracts right across the business.
"We have recently written to the last remaining six employees with zero-hours contracts based at our Wigan site confirming that their roles will be transferred to full-time contracts."
Further strikes are planned for 11 September to 18 September, and 25 September to 2 October.
