You use this phrase to sort of pretend to be shocked by something. When you're really shocked, you say things like "Oh my God!" and so on. But when you want to show other people that you're shocked or moved in a mild way, you can describe the situation like this:

How nice of you!

How stupid of me!

How horrible of him!

The basic meaning of this phrase is "What (someone) did is so (adjective)". So "How rude of me!" means "What I did was so rude!"