You can assert a fifth amendment privilege and not testify....

You can assert a fifth amendment privilege and not testify. But if you go in and swear to tell the truth, then you better do it. Otherwise, if a prosecutor finds that you have testified in a way that is factually incorrect and you had reason to know that it was factually incorrect, then you're guilty of perjury.