You start a sentence with "I must admit..." when you're telling people something that you feel just a little bit guilty or embarrassed about:

I must admit, I'm not very good at this sort of thing.

You can also say "I must admit..." when someone else impresses you more than you expected:

I must admit, you're quite a singer.

You don't use this for things that you feel really guilty about, like making a big mistake at work.

You can also use the phrase "I have to admit" in a similar way.