This means to stop bothering someone. With friends and family, it may mean to stop asking questions, or to just give them some quiet time. You may ask a stranger to leave you alone if they are making you uncomfortable. 

This guy at the bar would not leave me alone.

Sometimes I just leave him alone ‘til he’s in a better mood.

We can also say “leave (someone) alone about (something).”

My mother won’t leave me alone about the Christmas party!

This probably means the person’s mother is asking a lot of questions.