Je n’ai pas le temps d’aller au restaurant. Je n’ai pas eu le temps d’aller au restaurant. (That particular evening, a particular date).

Je n’avais pas le temps d’aller au restaurant is possible as well but means that in that period in your life (and not only one particular date), you simply had no time to go.

If these two phrases were totally independent of each other (and without further context) it would be hard (if not impossible) to explain/understand why converting phrase #2 from the present to the past would AUTOMATICALLY and ALWAYS require the “imparfait,” because the passé composé (j’ai eu beaucoup de choses à faire) would be fine/correct, depending on the context (“Désolée pour le retard mais j’ai eu beaucoup de choses à faire depuis ce matin,” for example).

If, however, the two phrases were related, then the use of the “imparfait” (in phrase #2) is totally justified and easy to explain and (hopefully) easy to understand:

In addition to the uses for the French “imparfait” that you mention in your question (repeated actions, descriptions, state of mind or habitude [and a few others]), the “imparfait” is also used: IV. [to provide]… background information in conjunction with the passé composé.

This background information can take the form of “what was occurring (imparfait) WHEN (QUAND) another particular event/action occured (passé composé)” OR, as in your example, “what was occurring (imparfait) that caused (or prevented) another particular event/action to occur (or from occurring) (passé composé). That is to say that it expresses the “BECAUSE”/CAR (the reason, the excuse) as background info in this case.

Your example combined as one phrase would be:

Je n’ai pas eu le temps pour aller au restaurant CAR j’avais beaucoup de choses à faire.

Therefore, in order to decide if an action has been completed or not for the purposes of using the “imparfait” to provide background information when two past actions are discussed, you look only to see if the past background action was occurring (was still going on = on-going) WHEN the second past action occurred , OR to see if the second past action occurred (or didn’t occur) BECAUSE the past background action was occurring (was still going on = on-going) at that same time in the past. The fact that the past background action is now (presently) complete (has now been completed) is not important as long as it was on-going (still going on) when the second past action occurred (or didn’t occur).

(I do empathize with those who are confused with the “imparfait,” especially with “avoir”… perhaps the following “trick” might help:

(Many of us are taught, correctly in my opinion, that the notion of the “imparfait” is best captured by “was ____-ing”.
(With “avoir,” however, the notion of “was having” in the “imparfait” is often hard to capture in our minds, because, as in this case, “I WAS HAVING a lot of things to do” is awkward in English and we therefore often resort to the notion of “I HAD.” Unfortunately, this causes confusion and conflict because “I had” is what we are taught to be the proper translation of “avoir” in other, non-imperfect, past tenses.

(If you could instead try to understand phrase #2 as meaning “I was having a busy day/evening” [or any alternative where “was having” doesn’t sound so awkward] it might help you to recapture the accurate sense of “I was having…” and thus avoid the notion of “I had”, which, again, is a source of much confusion for non-Francophones when we encounter the French “imparfait.”)