Jamie Oliver has claimed that if it was not for workers from abroad “every one of my businesses would close tomorrow”.