In the short term, it absolutely feels devastating to break a bond of friendship. In the long term, it is the best possible thing. You're actually doing something noble and good if you do it in the right way. You can leave them with, "I wish you the best, but I have to take care of myself." Or you don't have to wish them the best. It's okay if you don't. Maybe they don't deserve the best. That's not up to you to decide. You not wishing someone the best is not going to make anyone's life not the best.