I would say

Il a toujours eu des goûts un peu particuliers.

You can say “très” instead of “un peu”, but then the sentence is emphatic, not euphemistic.

Another way to indicate that someone has eccentric tastes is

Il a toujours eu des goûts bien à lui.

(Here “bien” means “very”, not “good”: his tastes are very much his own.)

Your suggestion of “particulier” is a good one. You could also use “des goûts très originaux”.

If you want a typical expression (similar to “there is no accounting for tastes”), you could use :

Les goûts et les couleurs, ça ne se discute/dispute pas. 

or

Des goûts et les couleurs, on ne discute pas.

This means that his tastes are not like yours, but this is not something that you could argue about.

EDIT: as mentioned in the comments, you could also drop the second part if you are talking to native speakers.

Comme quoi, les goûts et les couleurs...