I'd love to, but I can't.
This is a polite and extremely common way to say "no" to an invitation. When someone invites you to do something, but you're busy at that time or don't want to do it, say this phrase and then give a reason:
I'd love to, but I can't. I'm completely broke this week.
I'd love to, but I can't. I have a karate class this evening.
Follow PhraseMix