Brother Titus Keet, 76, who lives on Caldey Island, was arrested after Dyfed-Powys Police received a report on 18 September.
The island lies off the coast of Tenby, in Pembrokeshire, and is home to a group of Cistercian monks.
Caldey Island monastic community said they take the allegations very seriously.
In a statement, the community said they were co-operating fully with police and their priority was the wellbeing of those involved.
"If anyone has safeguarding concerns, then please bring those to the attention of the police or our safeguarding officer," they said.
As it is an ongoing investigation, the community said it would be inappropriate to comment further, but "asks your prayers for all involved, especially those who are most vulnerable".