No, it just means you never oversleep.

“Pas un gros dormeur” is a negation of “gros dormeur” which defines somebody who sleeps more than the average person.
If you’re not somebody who sleeps more than the average, you can be someone who sleeps normaly or who sleeps less than other people.

In everyday language, “pas un gros dormeur” just characterises the sleeping habit of a person that doesn’t need to sleep hours and hours. Someone who is used to sleep 6 hours a night for instance could tell you “Je ne suis pas un gros dormeur, quelques heures de sommeil me suffisent”. If someone replies by telling “Oh tu es insomniaque ?” most likely he/she is just wondering whether the fact of having little sleep wouldn’t explain some type of disorder (insomnia) as that could well be the case, but not trying to say anything else, I think.