The former is the common way or pronouncing that sound although slightly touching the upper teeth shouldn’t make that much of a difference.

I think both are correct. The first seems to describe using quite technical jargon, the second is using plain language and is simplifying the description of where your tongue should be.

When I pronounce “aller”, my tongue touches my alveolar ridge, but sometimes it catches my teeth a little too. The difference in sound is barely distinguishable.

Both are correct on the particular point you refer to. The explanation is different because targeted at different types of learners.

The video is meant for learners whose mother tongue is English and points at the differences between the pronunciation of the English letter L in which the tongue touches the alveolar ridge and the upper teeth¹ and the French pronunciation in which it is only the tip of the tongue that will touch lightly (this is the relevant word), which means that when speaking French you mustn’t let your tongue touch your upper teeth as much as you do when speaking English.

I expect the explanation you read about putting the tongue to the alveolar ridge is not specially meant for English speakers and it does not contradict the fact that the tip of the tongue can be at the front of the palate, thus touching slightly the area where the teeth meet the palate.

Where I disagree with that video is when it says that when pronouncing the French L the tongue stays down. That is neither relevant nor true. And later on the video you can see the lady not only curving her tongue up but sticking it out when saying final Ls ( which French people don’t normally do when saying that sound).

^{1 Not touching the upper teeth with the tip of their tongue when pronouncing the letter L is one of the various signs that will depict a French speaker of English.}