Pour means in order to, with the purpose of. Use that as a test.

Note that the syntactic environment is also different between the two cases. The pour version modifies a sentence; the à version modifies a noun phrase.

I could conceivably say

je vais au parc à jouer au foot

but that would imply the sentence structure

je vais au (parc à jouer au foot)

that is to say, it would assume there is a specific kind of park with the purpose of playing football (in), known as a parc à jouer au foot (this is not actually the case), and you’re saying it’s one of those parks you’re going to.