We do not mind our not arriving anywhere nearly so much as...

We do not mind our not arriving anywhere nearly so much as our not having any company on the way.