A “salle de sport” is a place where people of every walk of life go so as keep fit; they can do there a great variety of exercises, the most important being jogging on jogging machines. Nowadays, as these facilities were inspired by the american civilisation they are also called “salles de fitness“.
There is another meaning of “salle de sport“, the original one; the term describes building that can contain a considerable number of spectators and in which very often any of several ball games can be played (basket ball, handball, volleyball). Some of those salles can be used for numerous sports manifestations, as for instance The Madison Square Garden in New York, which in French you call “salle de sport omnisport” (tennis, ball sports, boxing, even concerts, …).

A “gymnase” is originally a building where specialised sportsmen go regularly for developping skills in the domain of gymnastics, gymnastics being the activity in which the body is trained to develop strength and at the same time agility.
However, this term, gymnase, is used also nowadays for omnisport buildings where you can play indoor ball games or even practice archery. They do not necessarily have facilities for a public of spectators and are meant as community facilities to provide the young with the means to participate in sports activities.

Addition due to user Laurent S.

In Belgium the term “hall omnisport” used instead of “gymnase” (France and Canada). https://fr.wikipedia.org/wiki/Gymnase_(sport)

A “salle des sports” is often called a “palais des sports” when the building is not part of a school complex. In a shool a “salle des sports” is also called a “gymnase” or “gym”, that latter term being a mere apocope of the former (réf.). Whereas the word “gymnase” is masculine, “gym” is feminine (réf.) (On va au gymnase ; on va à la gym.).

I would like to add something to the (very complete) answer from LPH:
In the french speaking part of Switzerland, “gymnase” can also be refering to high school (e.g.: “Le gymnase de Morges”, which translates to “Morges’s high school”).
We tend to use “salle de sport” or “salle de gym” when it comes to refering to the place where you do sports.
I’m not really sure where this comes from and often french speakers from France do not know what we’re talking about. You will likely not meet that word in its “high school” meaning but if you were to one day, you’ll know what it means:)