Don't worry. They may not even contact your referees. I'm not convinced my PhD supervisors contacted mine - I hadn't spoken to them for 5 years but they told me at the interview they had contacted them...
They couldn't have got hold of my previous work referees either, since they had left the company.
If they do attempt to contact them, and they can't, I expect that they will ask you about it and give you the chance to explain or give another referee. Also, your referee may respond regardless of being off work, or may contact you and tell you they can't provide the reference.
So, my advice is to do nothing, or you could contact your referee yourself and ask them to notify you if they get asked for a reference?
Also, for my postdoc, they emailed my referees on the same day as tell me I got the interview, so maybe they have already contacted your referees...