"Ultimately" is a word that you use when you want to make a conclusion. Use it to summarize a decision or a final point that you want to make:

There are a lot of different options. Ultimately, though, it comes down to which one is cheapest.

You can also use "ultimately" to talk about something that happens eventually, after a long period of time:

The popular theory these days is that some dinosaurs ultimately evolved into birds.

You can see from the examples that "ultimately" can come at the beginning of a sentence, or it can come in the middle before a verb.