Intuitively, I understand the first one “à grandir” as emphasising some kind of general law or fatality (I almost hear “à force de grandir”), and the second one “grandissant” as describing something that is actually happening (we grow up surrounding by horses). One could also say “en grandissant”, which would convey a general law as well and would be very similar to “à grandir” but with less emphasis on fatality. The difference might be similar to that in English between “by growing up…” and “growing up…”.