It can be exactly that, filler, or rather as I see it, a bad habit contracted somehow from listening to people having the same bad habit or maybe from people that use the word properly; it shouldn’t be inferred from this spurious usage that the word is empty, it’s not; it corresponds among other possibilities to “actually” or “as a matter of fact” in English when used correctly.

— You should buy some of those as soon as you can, they’re great !
Tu devrais en acheter aussitôt que possible, il sont excellents!
— I know ! As a matter of fact, I already did.
Je sais! En fait, j’en ai déjà achetés.