Both are correct.

• Au fait : what you are going to say relates to what you’ve been talking about.
• À propos : might indicate a switch to another subject.

In maybe a more familiar way, there’s also :

• “Au passage“
• “Tant qu’on y est” depending on the context. I think it refers mostly to actions like “Tant qu’on y est, on devrait laver la cuisine.” / “By the way, we should wash the kitchen” but I’m not quite sure of this one.

To add to the translations proposed by @zejam

• Au fait
• À propos (à ce propos)

I think some sentences could be translate with

• D’ailleurs

« Au passage, … » qui ressemble assez fortement à la version anglaise fonctionne plutôt bien à l’oral, et même parfois à l’écrit. En français, le sens littéral est plus souvent respecté qu’en anglais. Dans les cas où le « chemin » n’est pas suffisament matérialisé, plusieurs variantes existent :

• Au fait, (introduction d’un nouvel élément factuel lié au sujet)
• D’ailleurs, (introduction d’un sujet lié mais distinct)
• À propos, (introduction d’un sujet qui peut avoir une influence sur la discussion précédente)
• Tant qu’on y est, (quitte à parler de ça, autant pousser la discussion jusque là / jusqu’au bout…)
• Puisque tu le dis, Maintenant que tu le dis, (oral)(introduction d’un fait ou d’une idée en tenant compte du propos de l’interlocuteur).

There’s also tiens, which can precede about every answer mentionned yet:

Tiens, au fait, tu as pensé à acheter des galets ?

Tiens, à propos, as-tu pensé à acheter des galets ? (By the way, it’s grammatically better to say as-tu than tu as for a question, but that’s another question)

Tiens, en passant, as-tu pensé à acheter des galets ?

But it’s also worth noting that it’s also used on its own:

Tiens, tu y as pensé, cette fois ?

Although it’s quite informal (and somehow implies that’s you’re on a first-name basis) and I have no real clue why it’s used that way (worth yet another question) as you’re not actually asking the person to hold anything.

In Canadian French, we use en passant.

En passant, t’es tu rappelé de prendre…

I’m Belgian (Dutch part), and we have many dialects.

I’m from the the most westerly part, where the dialect uses a lot of French words.
We often use “à propos” to indicate what you are trying to achieve.

So even outside of French, “à propos” is used to say “by the way”.

None.

• Au fait, means: let’s get to the bottom of this. It is usually used as “by the way” but should not.

• À propos, means: This subject is related to this new idea I’m going to talk about.

I would use “Au passage” or “Pendant qu’on y est“.

I’d say à propos is slightly more formal than the other two. But all in all, there are totally interchangeable.

Yes, basically they are interchangable, it will depend only of your level of language :

• Au fait – Everyday language, casual register
• D’ailleurs – Everyday language too, less casual
• A propos – Used also in everyday language too but less casual thant the two others

By the way in English always introduces what is called an afterthought:
Here’s an example.

“We spent three weeks in the mountains camping, you know. It was so much fun. We hiked every day farther up the trails and also just hung out a lot. By the way, have you spoken to John recently?”

By the way is a subject shifter. The speaker is shifting to another idea that is completely unrelated to the previous subject he or she has been talking about.

Au fait and à propos can both be used like this. D’ailleurs cannot. It means besides.

Wikipedia: locution conjonctive: Se dit lorsqu’on vient à parler de quelque chose dont on se souvient subitement : introduit un changement de sujet, une idée secondaire ou une parenthèse dans un cheminement logique.

By the way in English always introduces what is called an afterthought:
Here’s an example.

“We spent three weeks in the mountains camping, you know. It was so much fun. We hiked every day farther up the trails and also just hung out a lot. By the way, have you spoken to John recently?”

By the way is a subject shifter. The speaker is shifting to another idea that is completely unrelated to the previous subject he or she has been talking about.

Au fait and à propos can both be used like this. D’ailleurs cannot. It means besides.

Wikipedia: locution conjonctive: Se dit lorsqu’on vient à parler de quelque chose dont on se souvient subitement : introduit un changement de sujet, une idée secondaire ou une parenthèse dans un cheminement logique.