Some problems are so difficult that it's best not to even try to talk about them or solve them. When you start trying to fix the problem, it causes many more problems. We call this situation "opening up a can of worms".

For example, imagine that you and your sister have been arguing with each other over who should take care of your parents in their old age. For the past few months, you haven't talked about it. Now you and your husband are on your way to eat dinner with your sister's family. You tell your husband:

Don't mention anything about my parents. I don't want to open up that can of worms.

You say this because you know that you and your sister will get into a long, angry argument if you start talking about that topic.

The phrase "a can of worms" is quite common and can be used in casual or formal situations.