The Adventure Bar Group co-founder leaves the listed late-night operator two years after it acquired his business