I am not sure how to explain the exact meaning of “par” here, but I could say that it is the same as saying:

pour garder ma contenance

I could find another expression where “par” is probably used the same way:

par inadvertance: Je l’ai bousculé par inadvertance.

But that is not quite the same… I have to say that even in Gide, this “par” sounds a bit funny to me. I would have naturally written something like:

Pour me donner bonne contenance

The reason this particular “par” sounds funny to me is probably that usually, “par” is used to denote a cause (“par amour de la patrie”, “par erreur”…), but here, it seems that contenance is the result (en prenant sa main, il garde sa contenance). It feels inverted between the cause and the effect. All three translations put “contenance” as a result, and that feels more natural to me, unless “contenance” has a meaning I missed (where it could be a cause).

The definition of contenance that applies here is:

Attitude extérieure exprimant une manière d’être voulue, précise, propre à une personne qui se surveille devant autrui.

and par contenance stands for par manière de contentance.

In other words, contenance reflects a desire to appear a certain way to others. As I understand it here, the narrator is acting out of an almost instinctive social impulse when he sees Alissa’s available hand and decides that it would be neat to hold her hand.

To me personally, the translations that make it sound like the countenance is a purpose, such as to keep myself in countenance and um mir eine gewisse Haltung zu geben, feel a bit far from the original meaning because I perceive par contenance to indicate more the reason for behaving a certain way than the goal. I feel that aus Schicklichkeit is a lot closer in terms of sentence construction. (Also this prose is obviously very carefully and beautifully written, it would have been easy for the author to express the idea of a goal explicitly if it had been their intent to do so.)

Here are a few other similar constructions of par + raison that will show what I mean.

Par respect pour ton père, écoute ce qu’il a à te dire. (Hear your dad out, out of respect.)

Par ignorance, j’ai causé du mal. (I’ve done wrong, out of ignorance.)

Elle répond à ses mails le week-end, par excès de conscience professionnelle. (She answers email on weekends, out of excessive professionalism.)

Even though um mir eine gewisse Haltung zu geben / to give myself a certain poise doesn’t translate the expression literally, I think it’s the best translation of the three given the lack of a strict equivalent for this meaning of contenance in English.

The expression for this in English is appearance.

Translation: for the sake of appearance or for appearance’s sake.

Appearance is the word for acting a certain way in front of other people.

Think of the British comedy: Keeping Up Appearances