Cronus wanted to thwart a prophecy that was given to him, where his children would overthrow his rule. He attempted to do so by eating all of his children. Thus, Hestia was the first for Cronus to swallow.
When Zeus eventually forced Cronus to regurgitate, Hestia was the last one to exit their father's belly. In a way this makes Hestia both the eldest and the youngest of Olympian gods.
Hestia never married, though Apollo and Poseidon both tried to court her. She swore to retain her virginity forever instead. This made her immune to Aphrodite's powers, who otherwise had major influence in the Olympian affairs.