You are right and should capitalize only words used as proper nouns, and in the given examples, only the first letter is capitalized.

Also don’t forget the accents, including on capitalized letters.

A leading uppercase is always mandatory when a word starts a sentence.

Papa, tu es là ?

Otherwise, papa, maman and the likes are noms communs so normally take no leading uppercase.

e.g.:

Quoi, maman, vous n’étiez pas sage ?

Charles Beaudelaire, Les Épaves, 1866.

However, you are also free to use what is called the majuscule de déférence.

Vous êtes là, Papa ?

Finally, the first letter is generally the only one in uppercase:

Grand-mère, tu es là ?

although there are a few cases of Grand-Mère and Grand-Père, either because it is a proper noun like a brand name, or because of the deference case.