In order to be sure that it’s not due to a plugin or a custom I made, I have build a brand new site with a fresh WP+Frontier install. I made two posts. One with a picture without caption and another with the same picture with a caption.
The URL is : http://cloud.adsl2meg.fr