Oui, les deux traductions sont valides.

“Depuis” n’est pas toujours utilisé en début de phrase. Par exemple: “J’étudie le français depuis 1983” (I study french since 1983)

Here, having a comma implies a stop in pronunciation :

J’ai mangé la tarte (optional stop), et depuis (stop), j’ai mal au ventre.

It implicitly says :

J’ai mangé la tarte. Et depuis [que j’ai mangé la tarte], j’ai mal au ventre.

I believe the stop is made to understand that we omit a part of the sentence. With the full stop after the first part of the sentence, depuis starts the sentence as you are used to.

As Chop suggested in comments, we also find this kind of structure :

J’ai mangé la tarte, et j’ai mal au ventre depuis [que j’ai mangé la tarte].

This structure is slightly more familar.

A famous example of that “depuis” standing truly alone:

Bien avant Arminius nous en avions pris l’habitude et, depuis, nous
l’avons conservée.

[ M. le général C. de Gaulle, Disc. et messages, 11 novembre 1942 ]

Note the position of the commas; I would therefore use:

J’ai mangé (toute) la tarte et, depuis, j’ai mal au ventre.

The comma placement is an invitation to explore grammar; many answers are possible. I choose to construe this in terms of clause analysis and, in that respect, considerations related to speech are minimal. The coordination conjunction “et” need not per se be followed by a comma; it may be in such cases as the “incise“, the “complément de phrase” or the “modalisateur“; but even then you may have suggestions that a shorter complement allows for more leeway in terms of not using it. Generally double commas are used to enclose or detach a function “unit” within the sentence. This contrasts with the use of the single comma as a coordinating element (generally see this). In Le Bon Usage (Grevisse et Goosse, ed. Boeck/Duculot), at §126 c), we are explained that « [l]e complément adverbial précédé […] d’une conjonction de coordination est généralement séparé de ces mots par une virgule : […] J’étais très contrarié, et, pour la première fois, la solitude à l’Amirauté m’apparut pesante. (GRACQ, Rivages des Syrtes, p.75). This is certainly not by any stretch an exhaustive analysis of such a rich material.

Depuis is quite interesting. Another answer discusses with interest the adverbial locution “depuis lors” that depuis can mean; this appears in the preposition part of the entry for the word. You may also have it as the adverb. A “complément de phrase” could be a clause constructed with depuis+que, along the lines of the “locution conjonctive” described in the Trésor entry (also refers to depuis le moment où, etc.). All sorts of ways to basically mark the posteriority of what follows, or subtle variations of causality. I wonder if qualifying one way or another has an impact on how one construes “depuis” here and the comma placement. I’m satisfied with the idea that depuis, in the middle of the sentence, whatever it refers to, is not the natural position of that word and should therefore be marked, and especially following the conjunction “et”. See this answer for further insightful considerations.

I’d be inclined to say depuis lors instead of using it alone; but I might be wrong, or maybe that’s old-fashioned or formal.

I think that if you did use depuis lors instead of just depuis alone, then you wouldn’t need the (stop) in the first quote of Random’s answer.

J’ai mangé la tarte (optional stop), et depuis (stop), j’ai mal au ventre.

J’ai mangé la tarte (optional stop), et depuis lors j’ai mal au ventre.

The best way you could use it as a standalone word would be,

J’ai mangé (toute) la tarte et, depuis, j’ai mal au ventre.

As said before. The reason is simple: the commas are used to do what we call a "proposition explicative" in french, therefore the sentence could be valid and have the same meaning without it.

J’ai mangé (toute) la tarte et j’ai mal au ventre.

"Depuis" is here used to make explicit a cause-and-effect principle: that’s its usage in every sentence.

If we want to push the analysis further, here is how you could construct this sentence to understand it:

First, you have two sentences:

J’ai mangé (toute) la tarte.

J’ai mal au ventre.

Then you use "et" as a preposition, to join the two sentences:

J’ai mangé (toute) la tarte et j’ai mal au ventre.

Note that you could use just a comma, instead of the "et".

This sentence is correct, but it does not imply any cause-and-effect relation between the fact that I ate the whole pie and that now my stomach aches. Here comes "depuis", and your sentence now has its full meaning.