"Going out with" someone usually means going to a restaurant or bar with them:

Use "go" when you and the listener are going to go to the place together.

Use "come" when you're already at the place, or when you already have plans to go to the place with other people and aren't sure whether the listener is going to come with you.

Another meaning of "going out with (someone)" is going on a date together.