Use the preposition "along" to describe walking next to a river. "Along" is also the word to use to talk about things that are next to a river in several locations:

There are some pretty yellow flowers growing along the river.

You can also use "down" or "up" to talk about walking next to a river. "Down" is the direction that the river is flowing in, and "up" is the direction that the water is coming from.

I walked up the river for about a mile, then turned around and walked back down.