À is a preposition, it is used to introduce the prepositional phrase that serve to complement the sentence. And the rule says that after a preposition the verb must be in the infinitive, whether à or another preposition.

The choice of the preposition depends on what precedes the preposition and not on what comes after.

Prêt à frire.

If you use a verb after (être) prêt then this verb is preceded by à, this à is not part of the infinitive as such, it is part of the verb phrase.

We have a different preposition after different expressions. For example:

(J’ai) envie de partir.
(Je suis) heureux de rester.

Here it’s (avoir) envie de, être heureux de that is followed by the infinitive, we don’t have à but de.

Here is a related but not similar question: When would one use “à” before a verb?