The use of the definite article is not optional. Sometimes both might be correct (I have no example right now though), but most of the time only one of the options (definite article/no definite article) is correct.

In this case, “je suis en les cours” is not correct because this isn’t the way we use “en” ; you can’t put “le”, “la”, “les” after it (except in some idioms).
“Je suis en cours”, “je suis en colère”, …

I don’t know if there is an easy rule to know if you have to put the definite article or not, or if you have to learn how to make a sentence with each preposition…

PS:
I disagree with Romain VALERI. “J’étudie le français parce qu’il est intéressant” isn’t french at all if nothing follows, while “Je répare la porte parce qu’elle est cassée” is the correct way to say this. “Je répare la porte parce que c’est cassé” isn’t french neither, except if you speak bad french…
I can’t comment so I answer here.

Réponse générale:

The use of an article is not optional when the noun it refers to is an object complement.
When it’s something else (like a time/location indication etc.), it depends.

Quelques exemples pour illustrer:

1. • Correct statement: “J’étudie le français”.

   • Inorrect statement: “J’étudie français”.

When we have an object complement (it answers to the “quoi” question: “J’étudie quoi? -> le français”) which is a common noun, it comes with an article. (excepted for some expressions like “j’ai faim” or “il fait froid”)

2. • Correct statement: “Je suis en cours”.

   • Inorrect statement: “Je suis en les cours”.

“en cours” is not an object complement, it’s a location indication (it answers to the “où” question: “Je suis où? -> en cours”. Therefore, it needs a closer look.

As Chugrothas says, we generally don’t use articles after the “en” preposition (excepted in some very special cases like “en l’occurence” or “en la personne de […]”, but this is some advanced stuff)

However, in this case, it’s possible to use articles after other prepositions, like “à”.
For instance, it is correct to say “Je suis à un cours de français”.

This is also why we say “Je suis en montagne” and “Je suis à la montagne”, both are correct statements (the meaning is not exactly the same though, as explained in the comments below).