In addition, many owners aren't managing to take anywhere near their full annual leave allowance with more than half (55%) taking two weeks or fewer off work.
The report shows more than half of business owners (53%) make business calls and email while on holiday. The situation is so bad for owners of medium-sized firms, that more than a quarter (26%) say they miss out on family holiday moments.
One in 10 are so tied to the office that they are on the phone or emailing colleagues and clients every couple of hours during their time away. This perhaps explains why the survey also found that 16% of small business owners are frustrated by poor mobile phone network coverage at their holiday destination.
More than two fifths (42%) say they have to work extra hours just so they can actually go away, with one in 10 (12%) claiming to have to put in an additional 21 hours or more.
Bibby Financial Services' UK chief executive Edward Rimmer said: "The pressures on business owners are such that even at this point in the middle of holiday season, we can see just how many continue to work.
"I think it is indicative of just how committed UK business owners are to seeing their enterprises succeed, particularly during difficult conditions. "It could also be due to the fact that small and medium-sized businesses don't always have the support network required to completely switch off and as a result are missing out on quality time with the family as they continue to work through the best bits of a family break."
