Origin

The original citation is from Delille – L Homme des champs.

Là, sur un tapis vert, un essaim étourdi
Pousse contre l’ivoire un ivoire arrondi ;
La blouse le reçoit.

Explanation

The Gradus explains the "tapis vert" (green carpet) is a billards (or pool or snooker) table.

In Littré and in leparisien.sensagen it is explained that some expensive billards balls can be in ivory or of color of ivory, and there, are called directly in the matter they are made from (or look like).

Moreover, @MakorDal added that "la blouse" is "the pocket on a pool table".

Answer

So, to conclude, this expression means to play billards/pool game.

Further research

The University of Toronto has one line about it :

Pousser contre l’ivoire un ivoire arrondi,

signifiera jouer au billard, si nous en croyons Delille ;