N’avoir plus qu’à (prier) means The only thing left to do is (praying) / Have nothing left to do but to (pray).

Edit: your sentence would become something like : The only thing left for us to do is praying for this conflict not to grow into an actual war.

Edit2:

Ne pas avoir (de bonbons) : Not having (candies)

Ne plus avoir (de bonbons) : No longer having (candies)

N’avoir que (des bonbons): Having nothing but (candies).

Ne plus avoir que (des bonbons): Have nothing left but (candies)

If you’re not familiar with infinitive sentences in French, here are the conjugated ones:

Je n’ai pas de bonbons.

Je n’ai plus de bonbons.

Je n’ai que des bonbons.

Je n’ai plus que des bonbons.

This is the construction with nouns, but it’s the same thing with verbes, using à + VERBE. Note that the meaning of avoir à becomes the English of have to, that’s why I say it’s easy.

Ne pas avoir à (repasser): Not having to (iron).

Ne plus avoir à (repasser): No longer having to (iron).

[tricky one] N’avoir qu’à (repasser): Just/Only having to (iron).
/! This is used a lot when giving some piece of advice, as in “You just have to iron it / You should just iron it” –> “Tu n’as qu’à le repasser”. You can also speak about yourself in this fashion “Je n’ai qu’à le repasser”

N’avoir plus qu’à (repasser): Having nothing left to do but to (iron)