It is not possible to communicate "kind of fluently" this way in French. It’s so widespread a rule that it affects a vast number of words, and in some cases it’s not only aesthetic, but it changes the meaning.

For example, if you have two possible subjects of which one is singulier and the other plural, the final letter (e.g. the /n/ or /t/ in viennent vs. vient) allows you to distinguish which one is meant:

— Il vient à la fête.

— Bon, j’ai hâte de le rencontrer. On me dit que c’est une personne très amusante.

— Ils viennent à la fête.

— Oh là là, il est en couple maintenant ?

Or if you have two groups of people, one feminine and one masculine, the final letter might allow you to distinguish which one is meant. In the comments jlliagre has constructed an example where you might use an adjective to refer to a group. Imagine four teams playing a match:

— Qui a gagné ?

— Les grands [the senior boys] ! / Les grandes [the senior girls]!

— Non, ce sont les petits [the junior boys] ! / les petites [the junior girls] !

Even the s (more accurately the /z/) at the end of suis could cause confusion. If it occurred before a consonant, the listener would be likely to perceive a short syllable starting with a vowel before the consonant, or perhaps assume they missed the /l/ of the direct object les, which would change the meaning.

So there’s plenty of opportunity for confusion. And in sentences where there’s none, it would still interfere with comprehension — like someone speaking Pig Latin is hard to understand because of all the extra "ay" sounds.