What is certain is that the oral will always applied this rule. You will say “t’es” and not “tu es”. When you write it, as a french person, I know 3 cases where you will contract :

• “je >> j’ + word begining with a vowel”
• “si il … >> s’il…”
• “cela est >> c’est”

I am French :

“Tu es” is the exact written and spoken version.

“T’es” is correct too, but more “spoken langage” if you understand what i mean.

I say “t’es”, i write “tu es”, but when i send a sms, i write “t’es”.

I know that when a word ends with a vowel and the next word starts with a vowel, then we would replace the first character by a ‘ character.

No, this is wrong. The only vowel that is elided in this manner is e, except that the feminine article la is elided like the corresponding masculine form le. An E with no accent at the end of a word either has an unstressed schwa sound or is silent (“e muet”). When the next word begins with a vowel, the E is never sounded, and some words are contracted, with the final E and the following space replaced by an apostrophe. Only certain “small words” are contracted in this way: articles, pronouns and conjunctions, plus a few compounds. I think this is the full list:

aujourd’hui [frozen idiom]
ce → c’
de → d’
je → j’
jusque → jusqu’
le, la → l’
lorsque → lorsqu’
me → m’
ne → n’
puisque → puisqu’
que → qu’
quelque → quelqu’
quoique → quoiqu’
se → s’
te → t’

The contraction is only made when the next word starts with a vowel sound (including semi-vowels). In words starting with h, the h may or may not count based on partly etymological, partly arbitrary rules (see How can we distinguish "H-muet" or "H-aspiré"?). Note that this is a spelling rule: the contraction is not optional and does not reflect a particular speech pattern.

Examples:

le matin, la soirée, l’aube, l’après-midi, l’œuf, le héros, l’héroïne, …
je viens, j’arrive, j’entre, j’ouvre, …
je le vois, je l’ouvre, …
je te vois, je t’attrape, …
quelque part, quelque chose, quelqu’un, …
Tu as eu une idée étrange. [none of these words can be contracted]

Apart from la, there is one other word not ending in -e where the final vowel is elided, but only in one specific case: the conjunction si is contracted when it is followed by the pronoun il or ils (but not when it is followed by other words, not even the feminine pronoun).

si + il → s’il
si + ils → s’ils
si + elle(s) = si elle(s)

In informal speech, some sounds are omitted. This can lead to additional forms of contractions, but these are not standard French and they never written except when transcribing informal speech. For example, “tu es” is normally pronounced [ty.ɛ], but in informal speech it is pronounced [tɛ] and this can be transcribed as “t’es”. Again, the contracted form this is not a correct way of writing it: it is sometimes used in highly informal contexts like emails between friends, but not in a newpaper article, a professional document, a school essay, etc.

Other personal pronouns are contracted in informal speech in different ways:
je → j’ even in front of a consonant (“je vois → [informal] j’vois, je suis → [informal] j’suis, chuis”)
ils → y (“ils sont → [informal] y sont”)

Well if you say “tu” to somebody it is necessarily informal…I am French and I always find it weird when I hear people say “tu es”, except in Nouvelle Vague movies perhaps!