As a French native, both sentence feel natural and I would understand them.

Adding the “que” really emphasizes that it’s ONLY to “se tourner les pouces”.

You could also say :

• On n’a pas fait tout ce chemin juste pour se tourner les pouces.

Hope it helps !

(ne) pas que is the negative of que.
(ne) pas is just the negative.

Il n’en a qu’une – He only has one.
Il n’en a pas qu’une – He doesn’t only have one.

Don’t get confused by the ‘ne’. Que by itself is considered a negative. ne que does not mean ‘not only’. It means ‘only’. Just like ne pas doesn’t mean not not, it just means not.

In colloquial French, ne can be dropped. It’s just a dummy word that has no meaning.

In your example, the sentence with que just adds the meaning ‘only’. In answer to your question, no they are not the same.