All of your sentences are valid and usual French.

The third sentence indeed negates the first one :

(1) Il n’y a qu’un livre: there is just one book (i.e. there are no more books than that one)

(3) Il n’y a pas qu’un livre: there is not just one book (i.e. there are more books than that one)

In the second sentence, plus isn’t there no negate anything but to add a piece of information:

(3) Il n’y a plus qu’un livre: there is just one book (and before, there was more)