Without memory there are no worries.
We should worry about our own souls first and trust in God's plan for others.
That which you worry about most in life is seldom bad, and that likely applies to dying.
Worry not so much about other peoples morals but more about your own.
Perhaps the greatest difference among people is between those who never have to worry about money and those who do.