The meanings are different:

Finir par means this is the last action you will take in a process of multiple actions you had to follow, or from multiple options that you had to choose from.

J’ai fini par te pardonner means “after coping with other feelings, in the end, I have forgiven you”.

You could extend the explanation that way:

Au début, j’étais tellement fâché que je ne voulais plus te parler, mais j’ai fini par te pardonner.

Finir de means the action has been fully completed.
J’ai fini de te pardonner sounds a bit weird as pardonner is usually not something that you decide to start and that should take you a while to complete, so as to mean “I started forgiving you some time ago, and now I have finished this action and you are forgiven”.

Another example with another verb:

J’ai fini de réparer ma voiture.

means you have completed the action of repairing your car

J’ai fini par réparer ma voiture.

means you had the option to have it repaired in a garage, or even to not repair it at all, but in the end you decided to repair it yourself.