In Latin, all cities and country names were feminine and not neutral as we might have expected.

The preposition used was in, e.g.:

In Gallia non solum in omnibus civitatibus atque in omnibus pagis partibusque, Caesar, The Gallic War.

In French, this in became en and is still used, without any article prepended to the country name:

In Gallia → En Gaule

Some country names started as adjectives:

Gallia belgica → Belgique

Other lost their i in French:

Francie → France (it. Francia)

Some country names appearing later were masculine for some reason, @vc-74 suggested the fact they were kingdoms might have played a role, e.g. Le (Royaume du) Portugal.

For masculine countries prepended with an article, the logic would have been to say en le + country name. However, the fact is en le is quite rare in French and has evolved in au.

Here is what says the TLFi about it:

Une servitude gramm. fait que, théoriquement, en ne peut pas s’employer devant les formes de l’art. déf. : le, la, les. En a. fr., l’art. masc. le − primitivement lo − se combinait par enclise avec la prép. en pour donner les formes el, en, ou, puis au par confusion avec la forme née de la combinaison de la prép. à et de l’art. masc. (cf. à, t. 1, p. 22; cf. aussi la forme arch. ès < en + les). Adrian Peake, bachelier ès arts (Cendrars, Bourlinguer, 1948, p. 135). Ainsi l’oppos. en/au représente en fr. mod. l’oppos. entre un nom sans art. et un nom déterminé par un art. En enfer, en paradis, au paradis; en été, au printemps; croire en Dieu, croire au Dieu de Jésus-Christ, croire au diable.

My translation:

A grammatical constraint means that, in theory, en cannot be used in front of the definite article forms : "le, la, les". In Old French, the masculine article le – originally lo – was combined by enclisis with the preposition en leading to the forms el, en, ou, then au by confusion with the form created by the combination of the preposition à and the masculine article.
…
Thus the opposition en/au represents in modern French the opposition between a name without an article and a name determined by an article. En enfer, en paradis, au paradis; en été, au printemps; croire en Dieu, croire au Dieu de Jésus-Christ, croire au diable.