Rev Dr Richard Murray addressed an emergency meeting of the church's decision-making General Assembly.
More than 1,000 seats were set aside at the church's headquarters in Belfast city centre for the meeting, which was streamed live on the internet.
Presbyterian ministers from both sides of the Irish border were entitled to take part, as well as a senior representative, known as an elder, from each congregation.
Rev Murray said: "It's necessary that we gather together in this special General Assembly to grieve and lament for those who have been hurt and harmed - and for their families who have to pick up the pieces.
"There will be things these good people have to live with for the rest of their lives.
"And to them we say sorry, that you have been hurt or harmed, or both, by our failings."
During a lengthy debate there were some complaints about a lack of openness by the church's leadership.
Rev David Clawson, from West Kirk Presbyterian Church in Belfast, said: "I expect my Presbyterian colleagues who are meant to walk alongside me to give me as full a picture as possible. I do not expect what I feel are secret protocols."
He was referring to a protocol for the standing down of ministers, elders or senior leaders.
A commitment was given that the relevant policy would be circulated as soon as possible.
The Police Service of Northern Ireland (PSNI) announced last month it was looking into safeguarding concerns at the church.
The investigation followed an admission of "serious and significant failings in the central safeguarding functions of the Presbyterian Church in Ireland from the period 2009 to 2022".
The statement was made by the Rev Trevor Gribben, who was the head of the church, known as the moderator.
He stepped down at the end of last month saying that although he had not been directly responsible for the delivery of safeguarding, some of the failings happened when he had held senior positions within the church.
As well as the police investigation, the Charity Commission for Northern Ireland has launched an inquiry into the church.
Rev Murray, whose official title is moderator, spoke about the ongoing police investigation.
He said: "So many of us feel under suspicion and I know across our denomination many, many people meticulously observe proper procedures with regards to safeguarding but our denomination now has this cloud hanging over us.
"All of us have collective responsibility, though it may well be certain individuals have to carry greater responsibility than the rest."
Around 650 people attended the morning worship session with more expected for the afternoon debates and discussions.
Rev Murray told the meeting the police investigation was focusing on the church as a whole rather individuals.
He added: "As yet, we do not know the extent of our failings."
Those attending Thursday's meeting have been warned not to say anything which could compromise an ongoing police investigation.
'Can't buy credibility'
Rev Murray warned that it was going to cost "a lot of money" to address safeguarding issues within the church.
However, he said it would be money well spent.
He told the General Assembly: "People across the country in congregations and Presbyteries are reviewing and revising their procedures.
"We want to put into place structures and procedures that ensure we are never in this situation again as we seek to minimise risk.
"But let me warn you, we're going to have to pay for it – and it's going to cost a lot of money.
"Some things are more valuable than money.
"The credibility of PCI (Presbyterian Church in Ireland), you can't buy it."
He said senior members of the church needed to be allowed to "spend whatever it takes to get this right".
'Significant failings'
Rev Dr David Bruce, convenor of the church's general council, gave some more detail about what had gone wrong within the church.
He said the problems centred on the responsiveness of the church to requests for safeguarding help.
He said: "Often these responses were slow, and in some cases, completely absent.
"It was clear that emails had not been opened, or that phone calls had not been returned. This was not universal.
"Many requests for help were dealt with in a timely and professional way, but the overall picture was of significant failings, including in delivery and oversight."
What happened at Thursday's meeting?
Thursday's meeting of ministers, elders and members of the church gave all concerned a chance to debate recent events and chart a way forward.
The acting clerk of the General Assembly, the Rev Dr David Allen, said everyone attending the public meeting needed to be mindful that the "PSNI's criminal investigation - which we have welcomed, support, and will co-operate with fully - is ongoing and mustn't be compromised by anything that a member of assembly might say".
A senior church committee has recommended a review of "governance, culture and integrity issues" as well as safeguarding. The report and its proposals were discussed.
A number of proposals were outlined, including:
Creation of a new safeguarding department
An independent, external review of safeguarding
Review of the use of non-disclosure agreements
A task group to look at governance, culture and integrity
Speeches were time-limited.
Proposers and seconders of resolutions were allowed seven minutes with four minutes for other speakers.
Another special meeting of the General Assembly may be held early next year, on 17 February.