You describe an action as "only ___ for ___" when you want the time to sound short. This can be used to talk about the past, the future, ongoing actions, or imaginary actions:

I only slept 2 hours last night.

If you'd left by noon, you only would have had to wait for 20 minutes.

The word "for" is sometimes dropped in casual speech, as in the example at the top.