The phrase "___ doesn't concern me" means "I'm not worried about ___". Adding "really" and "too much" makes the phrase sound more casual.

You should know that there is also another meaning of the word "concern". If you tell someone:

This doesn't concern you.

...it doesn't mean that the person is not worried about something. It means that it's none of their business. It's not related to them. Usually when you use "me", you're talking about being worried, and when you use "you", "he", "she", etc. you're talking about being related to something.