Relationship question... what to do?

I'm 17.. he's 20. We've been together for about 2 months now. I'm being honest when I say this.. we've had no physical relation accept for kissing.. I'm not ready for that yet (too early in relationship for me) Last night I went out with my girls and just had a little party at her house. Yes there were drinks, but I honestly did not have any (i personally don't have the desire to drink) Today, he asked what I did last night.. and I told him. Then he automatically got defensive. He got paranoid and was worried that I did something wrong since there were drinks there. He told me he "loves me, but doesn't trust anyone" is that even possible? I know he has been hurt in the past.. but I'm not his ex girlfriends.. he chose to date them knowing they were not good girls (he told me that they got physical within 2 weeks of dating and that im the longest girl he has been with without having sex) he admitted to being a sex addict.. so that kind of puts a red flag up especially since I'm young. Do you think I should get out of this while I can? because i have a gut feeling that this is just gonna be a mess in the long run... any advice or opinions? please no harsh comments.. but be honest.