When someone stays with you during a difficult situation, they are "staying by your side". For example, you might stay by someone's side:

• during a surgery
• after an accident
• during childbirth

Usually "staying by someone's side" means staying with them physically. But sometimes we talk about staying by a person's side in emotionally difficult situations.

I can't thank you enough for how you stayed by my side throughout the entire divorce process.