Chancellor Angela Merkel's conservatives and the Social Democrats agreed last night to introduce the legislation, which will come into force in 2016.
From 2015, German companies will be made to publish individual targets to increase female representation on company boards.
Both parties are currently in talks to form a coalition early next year. The centre-left Social Democrats had pushed for a 40% target of women on boards from 2021, but Merkel's party had so far resisted fixed targets.
"This is an important signal to improve the career chances for women and for greater equality in the labour market," said Manuela Schwesig, who led the talks for the Social Democrats.
In October last year, EU Justice Commissioner Viviane Reding pushed to get all EU public companies to reserve 40% of board seats for women. However, the plans were watered down after proposals were met with strong rejections from countries, including Britain.
In Germany, women currently fill 17.4% of positions on the supervisory boards of the top 160 listed companies, and only 6.1% of management board positions.
Nicholas Robertson, head of employment at international law firm Mayer Brown, said the problem with setting quotas is the "one-size fits all approach" would have created a raft of legal challenges.
"This is an interesting development and, if it works, one that may well be followed by other countries," Robertson said.
"Employers of all sizes will just need to be conscious to balance the drive for greater representation of women against the risk of discrimination claims from unsuccessful male candidates."
