You didn’t do anything wrong by leaving. It wasn’t such a good idea to go to his place on a first date, but you are in charge of the chances you take. I’m surprised you got out of there without being molested.