He told the Historical Institutional Abuse Inquiry that St Joseph's childre's home in Termonbacca, Co Londonderry, was "run on starvation".
The man was handed over to a priest at St Joseph's by his mother when he was a child and lived at the home in the 1950s and 1960s.
He told the inquiry which is investigating abuse claims against children's residential institutions from 1922 to 1995 that responsibility for the younger boys was given over to the older boys by the nuns.
The witness described how the older boys would call out the names of children at night, before having them stripped and sexually abusing them "for their own entertainment".
He also said that he was battered with a brush while he slept and on many occasions pretended to be asleep to try and avoid the abuse.
The children's home and Nazareth House, also in Derry, were run by the Sisters of Nazareth.
The witness said he believed the nuns at the home must have heard the children's names being called, but he claimed that they never checked on what was happening.
As well as being abused by the older boys at night, he recalled how he was also attacked by a group of them in a laundry room.
He also alleged that one of the nuns at the home made him kneel for hours until the blood stopped flowing to his knees, referring to her as "evil and hateful".
The former resident told the inquiry he was constantly hungry, recalling how he fainted during Mass.
The inquiry has already heard from other former residents of the Sisters of Nazareth homes who told of lying in bed soaked in urine in an effort to discourage sexual predators, suffering public humiliation by being made to carry soiled sheets draped over them for wetting the bed, and being beaten for not working hard or fast enough.
Another said he did not have a childhood and never played football or enjoyed any other normal pursuits.