Not that I know of in “internet slang”.

Older people would say (orally) “Oh, seigneur !“, “Oh, sainte !” or variants (such as “Oh, mon dieu !“).

On the internet, I’ve seen French people say “omg”, “my god”, “oh god”, but never “oh mon dieu” or “OMD” literally.

EDIT: I would like to add that this answer is from a French Canadian perspective. I know Gille’s answer is good, but you’ll most likely never hear “putain” outside France/Europe. Also, I think “Oh mon dieu!” is actually internationally closer to “oh my god!”

OMD ou une abréviation similaire n’existe pas à ma connaissance.

Pour exprimer la frustration, à peu près au même niveau de langage, on peut entendre “C’est pas possible / c’est pas vrai” ou simplement “Oh la la”. Agrémenté souvent d’un mot familier.

I’m pretty sure that the OP wanted the online slang translation, and if so, answers in correct french with actual words would sound quite off-topic, if I may. If not, feel free to correct me, Jonathan.

The English omg is mostly used because it has taken as a meme. Most of these key expressions of internet slang have no French counterparts (wtf, someone ?) and are used as such. And even these which do have a translation that was used enough to keep up, like mdr for lol, are often less used than the English original.

It must be noted however that the younger the community, the more people tend to use the French acronyms/expressions (for slang use, I mean. So, no, my dear mother does not say « OMG LULZ dude kudos you win the interwebs » to her friends).

(Has something to do with English being the language of the web roots, and probably also with some American dream never willing to die, I guess… ;-))

It must be also noted that from time to time it can even be heard in France, said in English, with a more or less Frenchified pronouciation. I don’t say it’s widely used, but not rare enough to raise an eyebrow in most informal contexts.

This kind of slang can rarely be translated one-for-one. The literal translation would definitely not do: “(oh) mon dieu” is possible as an expression of surprise or shock, but I don’t think “OMD” would be understood.

The closest equivalent I can think of is “putain”, which is a very generic swearword. It can mean “I’m surprised”, “I’m shocked”, “I’m impressed”, “That’s too bad”, “That’s bad”, and a number of other things. This is not a word that you’d use in polite company, and it’s often spelled “P***” in online media. It’s common on forums, at least. In SMS, it can be abbreviated as “put1” or “pt1”.

In fact, where an American would use a prude “Oh, my God”, a French would say “Putain” (excuse my French). Let’s say it is because we shall not misuse the name of God.

As this is short enough, we don’t really need an acronym. Sometime, we write “Put1”.

oh mon dieu is the best one to use online

I’ve been living in Montreal for 5 years and real-life conversation people insert the English “Oh my God” into sentences (although it’s pronounced more like “O-ma-gad!” and spoken very quickly. Of course if the emotions are more charged you would use a more vulgar expression. I’ve never heard a native Quebecois say “mon dieu” except to sound ironic. But as someone mentioned above, Quebec French slang is totally different from what you hear in France (the first time I went to France people looked at me as if I were from outer-space when I used certain phrases, which I didn’t know were Quebec/Montreal specific). Online, I see “OMG” the most.

Here in the French part of Switzerland and around here in France it’s sometimes used as is, OMG, but with a French pronunciation : "oh èm gé" instead of "oh em gee". It’s rarely used seriously and has lost any ties to religion. So it fits with your description of frustration.
Frustration wise we have swear words like "Putain !" ou "Merde !".

"Oh mon dieu" is more serious and more of a genuine, surprised reaction to something.

L’equivalent de OMG est “Oh mon Dieu”. Mais il n’est plus utilise’. “Oh putain” est completement different. “Oh putain” est utilise’ dans des circonstances similaires a OMG. Donc “Oh putain” est fonctionellement equivalent, mais pas semantiquement equivalent. La dichotomie entre “Oh putain” et “OMG” decrit une mentalite’ completement differente, l’abysse entre le paysan Normand du dix-neuvieme siècle et la France du vingt et unieme…

“Oh putain” conveys most of the meaning found in OMG but “la vie de ma mère” is the funniest approximation for me, especially followed by a MDR or a LOL.