John Smyth QC, who abused as many as 130 boys and young men at Christian summer camps, could have been brought to justice a decade ago, an independent review found.
Smyth - believed to be the most prolific serial abuser associated with the church - died in Cape Town in 2018 at the age of 75.
Mr Welby "could and should" have formally reported the abuse to authorities in 2013, the review said.
It found that "had that been done, on the balance of probabilities" Smyth could have been brought to justice "at a much earlier point" than a Hampshire Police investigation in early 2017.
Speaking to Channel 4 News on Thursday, Mr Welby said: "I have been giving [resigning] a lot of thought for actually quite a long time, there is nothing over the last 10 years that has been as horrible as dealing with numerous abuse cases.
"I have given [resigning] a lot of thought and have taken advice as recently as this morning from senior colleagues, and, no, I am not going to resign."
Asked if he considered resigning on Thursday morning, Mr Welby said "yes".
The archbishop also said he donated money to Smyth on missions in Zimbabwe.
He added: "Lots of people funded his mission out there. I think on two separate occasions I gave £40 or £50."
The report found that Smyth was able to move to Africa from England while "church officers knew of the abuse and failed to take the steps necessary to prevent further abuse occurring".
The barrister's actions were first identified in the 1980s, but the report concluded he was never fully exposed and was therefore able to continue his abuse.
Smyth, called an "appalling abuser" in the review, targeted boys and young men in the UK and Africa over five decades.
He is said to have subjected his victims to traumatic physical, sexual, psychological and spiritual attacks.
In a statement, Mr Welby said he was "deeply sorry that this abuse happened" and "sorry that concealment by many people who were fully aware of the abuse over many years meant that John Smyth was able to abuse overseas and died before he ever faced justice".
He added: "I had no idea or suspicion of this abuse before 2013.
"Nevertheless the review is clear that I personally failed to ensure that after disclosure in 2013 the awful tragedy was energetically investigated."