In "The Scarlet Letter," why has the prison door "never known a youthful era"?

In Chapter I of "The Scarlet Letter," Hawthorne as the narrator mentions that the prison was constructed as quickly as a cemetery was marked out in the new Puritan community:

The rust on the ponderous iron-work of its oaken door looked more antique that anything else in the New World. Like all that pertains to crime, it seemed never to have known a youthful era.

In other words, the prison door seems to have always been in the colony; in fact, it almost seems to have preceded the Puritans.  Hawthorne's striking mention of this "Prison Door" that looms over the community is extremely significant.  For, it indicates the iron rigidity of Puritanism, its unforgiving precepts, and its hardness and cruelty.  There is no innocence--no "youthful era"--in the Puritan community of New England. Sin has always been there; sin has always been punished, according to harsh Puritan law.  The grey men who stand outside with their "steeple-crowned hats," intermixed with the hooded women, both in "sad-coloured garments" indicated the austerity of their culture.  Only the wild rosebush in bloom by the prison suggests, as Hawthorne writes,

let us hope, to symbolise some sweet more blossom, that may be found along the track, or relieve the darkening close of a tale of human fraility and sorrow.

In his first chapter, Nathaniel Hawthorne sets the tone of his novel of the punishment of Hester Prynne and Arthur Dimmesdale, of the crushing of their passions and suppression of their spirits.