How do I get over my insecurity surrounding my boyfriend's attraction to others?

I have been with the same guy for over a year and a half. We have had some rocky points, but we are at a good point in our relationship now and I am very happy. He is a great guy who I care deeply about, and I know he feels the same about me. However, lately I have been getting upset over the silliest things. My boyfriend might make a comment about an actress on the cover of a magazine, and I'll freak out and think it has some reflection on me and that he doesn't see me as attractive, or that I'm not his ideal woman. For example, I know he's been to strip clubs before for bachelor parties, and it drives me absolutely nuts. It's been almost a year since the last time he went, and it still really bothers me. He tells me I'm pretty all of the time, and I know he is very much attracted to me, but I cannot get the idea out of my head that he is looking at other women and comparing me to them. I know I am a good-looking girl, and I know that he thinks I'm wonderful in many different ways, but I can't seem to fully embrace the fact that he can find many girls attractive, but will only want to be with me in the end. More than that, sometimes I feel as though I am exhausting him with my eruptions of jealously, insecurity, or whatever it is, and I'm not seeing the bigger picture. I just want to be free of thinking this way, because I really love this guy and I feel like I'm hindered by this crap that goes on in my brain. The rational part of me knows that I'm being ridiculous, but I can't seem to shake it.