La phrase est incorrecte. On doit dire:

J’ai l’impression de n’avoir survécu que pour devenir vieux et
décrépi.

On peut dire aussi:

J’ai l’impression d’avoir survécu uniquement pour devenir vieux et
décrépi.

In this context, que means the same thing as ne que.

French speakers often omit the word ne in negative sentences; this does not change the meaning. Here is some general information about this phenomenon: Omission of ne in the negative

This can be called incorrect, but it’s very common in the spoken language (Christensen 39, Lehti & Laippala “Results”). It is sometimes called “informal,” but that’s a pretty vague term. In speech, it isn’t markedly informal like the word “ain’t” in English. Educated speakers frequently use negations without ne: a better description might be that consistent use of ne with negations in spoken language is a feature of formal or prepared speech, such as a politician’s speech. It has been found that some students of French do not drop ne as frequently as native speakers since the students don’t have as firm a grasp on the situations where it is and isn’t appropriate, and they are often not taught to do so (Rehner & Mougeon).

In writing, the use of ne is considered standard and it is dropped much less often (generally only in markedly informal contexts such as chat).
However, transcriptions of quoted speech, such as dialogue in novels, may use negatives without ne. In a comment, you’ve said that this comes from a subtitle, so it would reflect how the speaker said it.

References:

• Christensen, Susan, “Analysis of the Standard French ne-drop Phenomenon“
• Lehti, Lotta and Veronika Laippala, “Style in French Politicians’ Blogs: Degree of Formality“
• Rehner, Katherine and Raymond Mougeon, “Variation in the Spoken French of Immersion Students: To ne or not to ne, That Is the Sociolinguistic Question“