There are many nouns in French that have an -s ending but are singular. Many of these nouns do not change when using singular or plural. Note that ‘temps‘ does not change in the singular (le temps) and the plural (les temps).
The French word ‘temps‘ comes from the Latin word ‘tempus‘, which also means ‘time‘ in English. This is the origin of the word. As words from other languages are used, they can be modified slightly such as this. ‘Tempus‘ in Latin, is singular however. It’s plural is ‘tempora‘. (Side note: Latin is also a romance language)
Other words in French with -s endings that are the same in singular or plural:
More information: http://fr.wikipedia.org/wiki/Temps#.C3.89tymologie