Justin Welby stepped down on Tuesday in the wake of a damning report into a prolific child abuser associated with the Church.
The review found he "could and should" have reported John Smyth's abuse of boys and young men to police in 2013.
Bishop of Newcastle the Rt Rev Helen-Ann Hartley - the only bishop to publicly call for Mr Welby to resign - said an "old school" culture put the reputation of the Church "before the protection of the vulnerable".
Smyth, a prominent barrister who died in 2018, ran summer camps for young Christians.
A report by Keith Makin accused him of attacking up to 30 boys he had met at the camps during the 1970s and 1980s, in what was described as a "clearly sexually motivated, sadistic regime" of beatings.
Bishop Hartley said: "This resignation does not solve the Church’s profound failure over safeguarding and the ongoing trauma caused to victims and survivors of church related abuse, nor does it excuse others whose neglect of their duties is exposed by the Makin report."
She added she remained "deeply concerned" by a letter of 31 October which she received from Archbishop of Canterbury and the Archbishop of York, Stephen Cottrell, and "what its tone and content says about the realities of the culture in our Church that still exist".
Bishop Hartley previously said the letter indicated "a complete lack of awareness of how power dynamics operate in the life of the church".
Mr Welby had declined to comment and a spokesperson for Mr Cottrell said the letter was not intended to be coercive.
In her statement, Bishop Hartley said some progress in changing safeguarding culture had been achieved, but it was being "undermined by the arrogance of a few".
"We need to pause and pray for the victims we let down and commit to changing the culture of the old school: a culture that put the reputation of the Church before the protection of the vulnerable," she said.