The French conditional (like the Italian, Spanish, Portuguese, Catalan, Occitan and other romance languages one) has its origin in spoken Latin (no surprise here).

It is built by appending to the infinitive the ending of the avoir auxiliary at the imperfect tense:

- Je   parler [av]ais    → Je parlerais
- Tu   parler [av]ais    → Tu parlerais 
- Il   parler [av]ait    → Il parlerait
- Nous parler [av]ions   → Nous parlerions
- Vous parler [av]iez    → Vous parleriez
- Ils  parler [av]aient  → Ils parleraient

In older French, the construction is the same but we see the avoir conjugation evolving:

- parler [av]eie    → parlereie   / [av]oie   → je parleroie
- parler [av]eies   → parlereies  / [av]oies  → tu parleroies
- parler [av]eiet   → parlereiet  / [av]oit   → il parleroit
- parler [av]iiens  → parleriiens / [av]ions  → nous parlerions
- parler [av]iiez   → parleriiez              → vous parleriiez
- parler [av]eient  → parlereient / [av]oient → ils parleroient

In late spoken Latin:

- *parabolare habebam    → *parabolarea
- *parabolare habebas    → *parabolareas
- *parabolare habebat    → *parabolareat
- *parabolare habebamus  
- *parabolare habebatis 
- *parabolare habebant  

In classical Latin:

- *habebam *loqui
- *habebas *loqui
…

The conditional has then been built more or less that way:

“j’avais parler” → je parler avais → je parlerais

The imperfect is telling it happened in the past but the association with the infinitive is locating the action in the future of that past.

References:

http://bescherelle.com/la-formation-du-conditionnel
http://asl.univ-montp3.fr/masterRECHERCHE/M1/j.bres/Conditionnel_coursM1A.pdf
http://www.trigofacile.com/jardins/chronica/actualite/0801-temps.htm