Don't be concerned with other people's impressions of you. They are dazzled and deluded by appearances. Stick with your purpose. This alone will strengthen your will and give your life coherence.
In order to please others, we loose our hold on our life's purpose.
There is only one thing for which God has sent me into the world, and that is to develop every kind of virtue or strength, and there is nothing in all the world that I cannot use for this purpose.