I believe it comes down to the fact that the French present participle is a very poor equivalent to the English gerund.

In particular in English, constructions using a Non-finite verb as an object or a subject will frequently use a gerund (“walking is slower than running”). In French such constructions will never use a present participle (at least I can’t think of examples off the top of my head. I’m fairly sure the present participle cannot be used as if it were a noun), but normally use the infinitive (Marcher est moins rapide que courir) or a nominalisation (La marche est moins rapide que la course).

Regarding the part where you ask about de+infinitive, this is but the most common way to connect an infinitive serving as a verbal complement to its head. In many cases (as here), the conjugated verb expresses a grammatical aspect (here a cessative or terminative aspect). The other constructions are the direct verb+infinitive construction (aimer, faire and pouvoir work this way) and verb+à+infinitive (réussir, parvenir are examples of this one). A few verbs allow other prepositions in corner cases, like partir and suffire, which sometime construct an infinitive with pour.

As with the English catenative verbs, you ahve to know which verb uses which construction, although there is some variation (Grevisse’s Le Bon usage spends nearly 15 pages discussing it): continuer and commencer are often built with à instead of de in literary style, for example.

I don’t think there’s more to it than the somewhat arbitrary fact that the verb finir is constructed with the preposition de followed by an infinitive (a verb after the preposition de is always in the infinitive mood). Finir de mourir, achever de mourir, terminer de mourir (rare, terminer is normally only followed by a noun).

À mourant (note the accent on the preposition à, to distinguish it from il a from the verb avoir) is not valid French. The preposition à is also always followed by an infinitive: commencer à mourir, se préparer à mourir.

There is no grammatical analogue in French of the English (or Latin) gerund. To turn a verb into a noun, one usually uses the infinitive.
Mourir est inévitable = Dying (or: to die) is unavoidable.