In this case I would say the following even though it is not a very nice way to say it.

Ce n’est pas à elle qu’ elle veut parler.

Which litterally means

It is not to her she wants to talk to.

I see two ways. Indeed, one uses celui/celle à qui, like you mention :

Ce n’est pas – la personne à qui – elle veut parler.
This is not – the person to whom – she wants to talk to.

and :

Ce n’est pas – à lui/elle – qu‘elle – veut parler.
It is not – to this person – that she – wants to talk to.

Both mean approximately the same, with slight nuances.