Don't feel obligated to spend time with people who pull you off the path of your life purpose.