"To His Coy Mistress"

Before I read the poem I thought a  woman would be the narrator and she would speak about her husband mistress. Since I had a pre-conceived notion it was the 'wife' speaking I beleived she spoke about their relationship. I understood later 'he' was talking about his mistress. It is confusing because he speaks of her in such a positive way at the beginning but later turns into negative things, like death, image detereoration, loss of lust ect. When I  read the second stanza it made me rethink who really was the narrator. A women would speak about her rival that way. Implying that his lust for the mistress is temporary while 'their love' is beyond physical look.