Photographers don't need to be aggressive. Some are. Henry Benson is aggressive - but then he's from Fleet Street. If you can talk to people, you don't need to push people around.
I enjoy traveling and recording far-away places and people with my camera. But I also find it wonderfully rewarding to see what I can discover outside my own window. You only need to study the scene with the eyes of a photographer.