It been a place of Christian worship since medieval times, but after months of excavations, Italian archaeologists have found frescoes and other evidence which indicate that it was associated with St Peter as early as the 7th century.
Dr Patrizia Fortini, of Rome's department of archaeology for Rome, said: "It was converted from being a prison into a focus of cult-like worship of St Peter by the 7th century at the latest, maybe earlier.
"It was a very rapid transformation. We think that by the 8th century, it was being used as a church. It would have been wonderful to find a document with his [St Peter's] name on it, but of course that was always going to be extremely unlikely."
St Peter and St Paul are said to have been incarcerated in the jail by the Emperor Nero.
The two apostles are said to have caused an underground spring to miraculously rise up from the ground so that they could baptise their guards and their fellow prisoners.
Peter was then crucified, upside down, in AD64. He was buried on a low hill on which, 250 years later, the Emperor Constantine built the first Basilica of St Peter.
The hellish prison in which the founder of the Roman Church supposedly spent his final days consisted of two levels of cells, one on top of each other.
The lower cell could only be reached through a hole in its roof and was purportedly where the Romans imprisoned their most formidable enemies, including a Gaulish chieftain, Vercingetorix, who had fought against Caesar in 52BC.
Some prisoners starved to death and their bodies were tossed into the Cloaca Maxima, the city's main sewer.
In the 17th century a church – St Joseph of the Carpenters – was built over the Mamertine Prison and it still stands today, overlooking the ruins of the Roman Forum.
Its exterior bears the words "The Prison of the Apostolic Saints Peter and Paul" and a marble carving of the two bearded martyrs peering glumly through prison bars.
When Charles Dickens visited the site in the 19th century he described "the dread and gloom of the ponderous, obdurate old prison".
Hanging on the walls he found "rusty daggers, knives, pistols, clubs, divers instruments of violence and murder brought here fresh from use".
Historians have long believed the dungeon was built in the 5th century BC, under Servius Tullius, one of the kings of Rome before it became a republic.