It is well to be off with the old woman before you're on with the new.
George Bernard Shaw (2004). “The Philanderer”, p.53, 1st World Publishing

It is well to be off with the old woman before you're on with the new.