Well, if the nature documentary I saw is to be believed, they eventually get tired of the plant not acting quite right and go find a real female fly. (IIRC it’s some sort of orchid)

Which I guess makes evolutionary sense - you want to use the fly for pollination, but you can’t be so sexy it completely gives up on procreation, or you run out of pollinators!