$1,000,000 in the bank isn't the fantasy. The fantasy is the lifestyle of complete freedom it supposedly allows.
To have an uncommon lifestyle, you need to develop the uncommon habit of making decisions, both for yourself and for others.
People will choose unhappiness over uncertainty.