Je ne veux pas confier ce travail à qui que ce soit n’est pas très idiomatique et serait probablement compris comme je ne veux confier ce travail à personne, ce qui n’est pas le sens attendu.

D’autre part, n’importe qui a un sens péjoratif que qui que ce soit n’a pas or ici, n’importe qui signifie quelqu’un qui n’est pas un expert.

"N’importe qui" here is used to refer to a person who is not well known as to their aptitude and skill as relates to a particular activity; it is someone that you can’t trust right away because you don’t know whether they are properly qualified. "Pas n’importe qui" could be translated in English by "not just anyone".

• Il ne veut pas que la peinture soit faite par n’importe qui, il veut que ce soit un peintre de métier qui fasse le travail.
  (He doesn’t want to let just anyone do the paint job, he wants it to be done by a professional painter.)

"Qui que ce soit" means everybody, no matter who they are; that is "anyone" in English. If you write "Je cherche un expert : je ne veux pas confier ce travail à qui que ce soit." then, what you are saying is that you are looking for an expert but that, all the same, you want nobody to do it; that is contradictory. As what you mean is "an expert and not just anyone" you have to use "n’importe qui" ("pas" is already in the sentence).