Doudou can also be a term of endearment applied to a human being. In the sense of an adult loved one, according to the Petit Robert and the Trésor de la langue française, it’s only used in the Antilles. It can also be used by a parent to refer to their young child (the child is the parent’s comfort object?). I can’t find a dictionary that mentions this meaning, but it’s not the first time I’ve seen it, and it’s a logical derivation anyway: a euphonic doubling of doux (sweet).

The original song has “Under the sea, darling it’s better, down where it’s wetter”. Here doudou translates darling. There is no implication that doudou is an object.