« Avoir fini » → to be done, to be finished.

« Être fini » → to be done for, finished with, kaput!

That is, the fini you want is the past participle, not the adjective. (And if it’s the past participle, the auxiliary is just about always avoir unless it’s a Dr. Mrs. Vandertramp verb or is reflexive.)