"By the way..." is a phrase that you can use when you want to change the topic of conversation. You use it when you want to talk about something that's unrelated to what you were just saying.

For example, imagine that you were talking with a coworker about your vacation plans, but you suddenly remember something about her husband:

We're going to spend a day in Paris and then, I think, a day or two in Rome. Oh, by the way, how did Randy's interview go?