The man, whom the BBC is not identifying, said the Right Reverend Stephen Conway did not do enough as Bishop of Ely when he was informed of John Smyth's abuse.
"He was the person in a position to stop John Smyth, bring him to justice, and he fundamentally failed," he said.
In a statement issued on Tuesday evening, Bishop Conway said he did all he could within his authority as a Church of England bishop, but that he was "deeply sorry" for not "rigorously pursuing" the matter with the Archbishop of Canterbury.
Justin Welby has now resigned as Archbishop after it emerged he did not follow up rigorously enough on reports of Smyth's abuse of more than 100 boys and young men.
The victim informed Bishop Conway's then diocese of the allegations against Smyth in 2013, according to a damning report by ex-social services chief Keith Makin.
In his statement, Bishop Conway said his immediate priority had been to provide support for victims and survivors in connection with the Diocese of Ely.
Bishop Conway said he made a detailed disclosure to Lambeth Palace, and contacted the diocese in South Africa to which Smith had fled.
'I had no authority'
"It was my understanding that this matter was reported to the police in Cambridgeshire and duly passed on to the police in Hampshire where the abuse had occurred," he said.
"I am clear that I did all within my authority as a bishop of the Church of England, bearing in mind that I had no authority over an entirely independent province on another continent.
"I acknowledge fully that my fault was in not rigorously pursuing Lambeth about that province-to-province communication, and for this I am deeply sorry."
The Makin Review said Bishop Conway was in a "potentially powerful position" to ensure relevant authorities in the UK and South Africa knew the full extent of the concerns about Smyth's abuse.
“We have demonstrated that this was far from the case, with serious abuse and crimes being covered up at the time," the review said.
“This complacency continues with comments from the Bishop of Ely and a lack of serious attention from Lambeth Palace and various police forces."
Smyth was a prominent barrister and a lay preacher - a member of the congregation who delivers sermons but is not ordained.
The Makin report described his "clearly sexually motivated, sadistic regime" of beatings during the 1970s and 1980s.
He singled out boys attending Christian camps and in sessions at leading public schools, including Winchester College, before taking them to his home and beating them with a cane in his shed.
Some of the victims had to wear adult nappies because of the bleeding they suffered.
Smyth was later able to travel to Zimbabwe and South Africa, where he is alleged to have continued his abuse.
He died in 2018.
The victim, who refuses to be referred to as a "survivor", said: "The Makin Review is quite clear about the culpability of Stephen Conway.
"I don't care about his prayers, I want his resignation."
Speaking before Mr Welby's stood down, he had this message for him: "Justin - you knew most of the victims. You have refused to meet us for seven years. You bloody coward."
Mr Welby had recently acknowledged that the review had identified how he had "personally failed to ensure it was energetically investigated" but had initially declined to resign.
In a statement, the Diocese of Ely said it was "committed to fully reviewing and implementing all recommendations" of the Makin Review "to ensure the safety and care of all individuals within our church communities".