People say this phrase when they're frustrated: they want something which seems simple and easy, but they can't get it. For example:

I just want to have one evening when we don't get in a big argument. Is that too much to ask?

I told you to take out the trash. Come on! Is that too much to ask?

When I call a meeting, I expect you all to show up on time. Is that too much to ask?

A person who asks this question sounds frustrated.