In France, the connotation is quite similar to the English: even though it’s supposed to be seen as “average”, it often has a “somewhat bad” meaning.

On a personal level, I often see a slight change of meaning depending on if you use “médiocre” or “la médiocrité”. It seems to me that “la médiocrité” (eg. “la médiocrité du gouvernement Macron”) is a stronger shade of bad than “médiocre” (eg. “le gouvernement Macron a obtenu des résultats médiocres”). But this is by personal experience, I wouldn’t be able to say if it’s a country-wide accepted nuance.

As for your example, although “médiocre” could be used to talk about cupcakes, it’s not a common occurrence, we would go for “pas très bons”, “pas super”, “moyens”, “bof”. I see “mediocre” generally used to talk about results, and not much else.

In English, the connotation is on the negative end of that spectrum: mediocrity is implicitly opposed to excellence.

Boileau serait bien d’accord : “Dans l’art dangereux de rimer et d’écrire Il n’est point de degré du médiocre au pire“

La zone concernée par le médiocre, part du milieu, du moyen pour s’étendre vers le pire.
C’est ainsi que l’on peut comprendre l’expression Plus que médiocre rapportée par Littré et dont le plus ne doit donc pas s’entendre méliorativement mais tout au contraire.