One not subtle but major difference is that the verb conger doesn’t exist…

I guess you are thinking about the expression prendre congé de … which is a formal way to say dire au revoir et quitter quelqu’un. e.g.:

Je dois partir. (I have to go)

Je dois vous quitter. (I have to leave you)

Je dois prendre congé de vous. (I have to take leave of you)

Unlike quitter, prendre congé is almost only used when referring to people leaving other people.

No natif speaker but the verb ‘conger‘ does not exist as far as I know.
You can find information about the word ‘congé‘ (I guess you confused the noun ‘congé’ with the possible past participle of the fictitious "verb" ‘conger‘ which as I said does not exist) below:

• https://en.wiktionary.org/wiki/cong%C3%A9#French;

• https://fr.wiktionary.org/wiki/cong%C3%A9;

• https://www.cnrtl.fr/definition/cong%C3%A9

Analyzing the usage of partir goes beyond the scope of a simple FSE’s answer. The only possible related construction that I can think of is

prendre congé : Faire les adieux qu’exige la politesse avant de partir.

See, for instance, https://fr.wiktionary.org/wiki/prendre_cong%C3%A9

There is also the related verb congédier.