In this case "Ce" is correct, it means "cela / ceci" and it’s always singular.
You’re right, it’s a kind of catch all construction.
"Ce n’est pas mon livre" and "Ce ne sont pas mes livres" are both correct.

The subject pronoun ce and the demonstrative determiner ce/cette/ces aren’t the same word, even if they share a form, and don’t follow the same rules.

The subject pronoun is ultimately descended from a Latin neuter singular demonstrative pronoun (hoc), but by Old French it had become uninflecting (ço < ecce hoc).

In Old and Middle French, ce/ço wasn’t a subject pronoun however, but a demonstrative adverb. In a sentence such as "ce sont mes livres", the subject was "mes livres" and it was this noun phrase the verb agreed with. Likewise, "this is you" would have been said "ço estes vos", the syntactic equivalent of "ceci êtes-vous"/"This are you", with the verb agreeing with the subject pronoun vos.

Eventually, by late middle French, this demonstrative-verb-subject structure was reanalysed into the modern subject pronoun-copula-predicate of the modern language. This triggered a change in agreement of the verb so that "c’êtes-vous" and "ce suis-je" turned into "c’est vous" and "c’est moi". But the process wasn’t quite complete when the language was standardised: with third person plural objects, both "ce sont mes livres" (the old structure, with the verb agreeing with the object) and "c’est mes livres" (the new structure with the verb agreeing with ce) were in common use.

The language authorities favoured the first structure, and so in formal French you say "ce sont mes livres" to this day. In more casual registers, the new structure won out, and you say "c’est mes livres", with no apparent subject-verb disagreement.

But at no point had subject pronoun ce a plural form.

By contrast, the demonstrative determiner ce/cette/ces derives from a merger of the Old French demonstrative pronouns cest and cel, which had feminine and plural forms (ceste/cez and celle/cels) and inflected normally, a trait they’ve kept to this day.

"Ce" is correct in "Ce ne sont pas mes livres" because "ce" is a demonstrative pronoun (pronom démonstratif). However, "ces" is a demonstrative determinant / article (déterminant / article démonstratif) which has to agree with its subject, as in "ces livres sont les miens."