Amplifying on the things improperly attributed to him, Yogi said: "I never really said all those things I said