Use "on the other hand" when giving an opinion that's different from one that was just said. You can contradict another person's opinion, or you can contradict your own.

A: Flying there is going to cost a lot.

B: Yeah, but on the other hand, we can stay with my sister so we won't have to pay for a hotel.

I don't know what to do. On one hand, I've always wanted to be a writer. On the other hand, I have a great job now and it would be a shame to give that up.

When you use "on the other hand" in speech or writing, it makes you seem careful, intelligent, and thoughtful. It's not really the best choice when you want to disagree with someone angrily.