What you explain in the first and second paragraphs is right.

However the syntax is wrong, you have to write something lke this:

• “On s’est bien amusées, nous deux. C’était une de ces conversations comme nous en avions déjà eu ensemble des milliers de fois.” or
• “On s’est bien amusées, nous deux. C’était une de ces conversations comme des milliers que nous avions déjà eu ensemble.”.

The three alternative formulations you come up with are entirely wrong.
“Comme si” will not do.
Given the construction in the main clause, there is no way that “que” could be a conjunction; it’s just not the type of constuction after which you find “que”; the only choice left is the relative object pronoun.

If you want to use the relative pronoun “que” you must take up in the relative clause an idea about the particular conversation and then introduce your comparison using “comme”; for instance you might say this among a thousand other possibilities;

• On s’est bien amusées, nous deux. C’était une de ces conversations que nous n’oublierons jamais, comme des milliers que nous avions déjà eu ensemble.

Other equivalent form (same meaning)

• On s’est bien amusées, nous deux. C’était une de ces conversations dans le genre de milliers que nous avions déjà eu ensemble.

1. Firstly, by “une de ces conversations”, you’re not talking about multiple conversations actually having taken place, right?

You are talking about one conversation which is similar to many other ones you already had. Comme means “like” here.

2. “It’s one of those conversations where you feel as if you’ve already had similar ones millions of times before”. Am I right in this regard?

It’s more than a feeling. You state you already had many conversations of this type.

3. Secondly, is “comme” the only possible conjunction in this sentence?

Telle que might have been used with the same meaning:

C’était une de ces conversations telle que nous en avions déjà eu des milliers ensemble.

4. Is “comme si” or “que” unusable?

Comme si would introduce a condition while there are none.

Que would be usable with a little change:

C’était l’une de ces conversations que nous avions déjà eu des milliers de fois ensemble.

but that would mean you already had the same conversation while the original meaning is just that you had similar conversations.