"The Imperfect Enjoyment"

In "The Imperfect Enjoyment" John Wilmot describes a naked woman lying in bed clasped in this guys arms. He uses imagery of the ideal of sex very thoroughly: "Both equally inspired with eager fire,/ Melting with kindness flaming with desire/ With arms, legs, lips, close clinging to embrace/ She clips me to her breast and sucks me to her face..." (lines 3-6). However, the man in this poem is shamed that he had loved so many before this woman who he actually loves and makes him "impotent". He never mentions any women's names and perhaps that is because they do not matter to him. They were just past lovers or even one night stands?: "Thou treacherous, base deserter of my flame/ False to my passion, fatal to my flame/ Through what mistakes magic dost thou prove/ So true to lewdness, so untrue to love?/ What oyster-cinder-beggar-common whore/ Didst thou ever fail in all thy life before?/ When vice, disease, and scandal lead the way" (lines 46-52). I do not understand why now does he think about his past. Does he think that this woman is going to be like the ones before. These low class women he refers to as whores? Or why now does his past come into his mind? Is it possibly because like the rest this woman is too just a random woman? However, I still do not understand? If he is impotent does that mean that him and this lady he is lying in bed with did not have sex? Not that even matters. I do feel that this man thought he was used and taken advantage of: "Worst part of me, and henceforth hated most/ Through all the town a common fucking post" (lines 62 and 63). With this said, mabe he fell in love before and the women only used him for sex and pleasure? Perhaps. I find that literature has come along way considering at one time sex was forbidden to talk about and now it is out in the open, and not just that, but a man is expressing that he was possibly used and regrets his past sexual partners? I might have taken this out of context but this is how I understood it. Maybe I am completely sexually distraught myself? haha and perhaps this is nothing like I understood it.