It can be said that this form is not inappropriate for everybody as shows this title, possibly pert in the way of defying more or less accepted ways.

We say quite normally

« Ils vont en Afrique. », « En Afrique ? J’y vais aussi. ».

Why shouldn’t we say in the same manner « y voyager » which is used with the very same preposition (“en”).

For instance, the piece of advice given here is clear: there is no problem in using that form.

I should say that it doesn’t feel quite natural as it is must be rarely used; however as I think about it, I see that it does fit a pattern of assembly of words that is common (J’y travaille, j’y vis, j’y rencontre des gens, j’y séjourne, …); the comparison can be drawn with “j’y séjourne” and one sees that the difference is hardly appreciable as far as semantics goes. So, why it is recommanded not to use it is not explainable through grammar or semantics. I would think a possible reason is that the occasion of travelling in in given place having been rare in the past the form has never been used much; this can be verified in this ngram which shows that only since 1920 is the form used regularly enough. The reason could be the mere lack of a habit in hearing the particular combination of words of this form.

Personally I wouldn’t advise not using it. However, as noted in the second reference it must be used in the sense of “travelling in” and not “travelling to”.

j’y voyage
is correct. It means you are in the said country, e.g.:

Es-tu en France ?

Oui, j’y voyage.

But you can’t use j’y voyage when you are talking about a place you plan to go e.g.:

Vas-tu en France ?

Oui j’y voyage.

in this case you should use the verb "aller" in the present tense

Vas-tu en France ?

Oui j’y vais. OR Oui je vais y aller.