Ça se comprend il me semble, mais la formulation est un peu bizarre car a priori on ne retourne pas une « charge ». Voici une autre possibilité (qui paraitra probablement un peu moins péremptoire).

Peut-être qu’il serait grand temps que tu te délestes de ces charges de revanche ?

I wonder if the intended meaning gets across to native French
speakers. Is there a better way to phrase the idea?

It does makes sense, though something along these lines would probably be more idiomatic –

Puisque ça fait X fois que tu me dis “à charge de revanche !”, le moment est peut-être venu pour un petit renvoi d’ascenseur !

(colloquial, implies that you have something to ask him/her in return right now)

or if you want to reuse charge de revanche in a slightly more fluent way

Il serait peut-être temps de t’acquitter de toutes ces charges de
revanche !

I really like the way that both answers given so far (with délester and acquitter) make the speaker sound (to me, at least) like s/he’s actually doing the other person a favor by offering to help lighten/relieve the heavy moral burden that the ever-growing backlog of promised, yet undelivered favors must certainly be causing!

Unfortunately I can’t find many examples of the following suggestion (which probably means that the structure is incorrect or at least not idiomatic), but with that cool notion/strategy of “appearing to be offering to help relieve a burden” in mind, maybe you could consider attempting to highlight further your playfulness by playing a bit off of the word “charges” with:

“En parlant des charges de revanche, peut-être qu’il serait grand
temps que tu t’en décharges un peu [sur moi]!”

See the following usage example from what appears to be a translation of a Hachette/Harlequin romance novel, via GoogleBooks:

“Tu portes un lourd fardeau, Dr. Riley. Il est peut-être temps que
tu t’en décharges un peu sur
quelqu’un.”

Cf : this similarly constructed example
(but using “par” instead of “sur”) from Volume 3 of the Sermons du pere [Louis] Bourdaloue, likewise via GoogleBooks:

Car vous ne ressentez pas le joug du baptême, ou parce que Dieu vous
donne des forces pour le porter, ou parce que vous vous en déchargez
par une lâche infidélité.

The problem is that “à charge de revanche” is an expression that uses an old meaning of “revanche” which is not used anymore today. You can’t use this colloquial expression outside of the canonical exclamation but for the sake of the joke it can be easily understood.

You can even distort it so it still looks like the expression but becomes syntactically correct.

Les charges s’accumulent, et la revanche n’est pas loin ! (the favours are accumulating, and the revenge is not far!)

It changes the meaning (as you now use the word “revanche” that means “revenge” in modern French but has the old meaning of “giving a favour back” which is used in the colloquial expression) but as it’s a joke you may like that.

Otherwise, if you want to remind him more seriously about the favours he owes you, you can say:

Il est temps de me rendre la pareille. (it’s time to give me a favour back.)

But it doesn’t sounds as much as a joke.