I think that during the shoot, you should never be there, unless something goes really wrong and as producer, you're responsible. The sign you did your job right is if you are not there.
If you're doing a big spectacle film, you've got to be mindful of large masses. Even then, you've got to be responsible only to your storytelling.