“Du”, in this example, is a contraction of “de le”, meaning “of the”.

So the construct might have been “le nom de le père“, except that “de le” doesn’t sound very good, so it is incorrect. You need the contraction “le nom du père”.

“De la” sounds (or is believed to sound) better, so the construction for mother would be “de la mère”.