Not really. In this case, "poids" just means "influence" (or more explicitly "interference/pressure"). It’s the exact same metaphor when you say in English that something "weights on" you (in nglish you might also talk about the burden of something).