Dr John Sentamu made the symbolic protest gesture live on BBC One's Andrew Marr Show.
He said Mr Mugabe had "taken people's identity" and "cut it to pieces", prompting him to do the same.
Prime Minister Gordon Brown has boycotted an EU-Africa summit because of Mr Mugabe's presence.
Although Mr Mugabe is banned from the EU, African leaders demanded the Zimbabwean leader be invited to attend the event in Portugal.
Ahead of the summit, the African Union chairman, Ghanaian President John Kufuor, said it was right to invite Mr Mugabe.
"Africa is made up of 54 nations, sovereign states, and I don't think any of us has the right to exclude another," Mr Kufuor told the BBC. He added: "All the presidents of Africa were invited to this summit because we see it as a meeting of two continents, Europe and Africa.
"If we allow anyone - I wouldn't say even a nation, but an individual, whatever we think of him - to be a stumbling block then I say it's unfortunate."
The summit is being used to agree joint policy aims in areas such as security, development and good governance. Dr Sentamu - who has been a consistent critic of Mr Mugabe's regime - said the international community, especially South Africa, had to act to help Zimbabwe because people were starving.
Speaking as he used a pair of scissors to cut up his collar, he said: "As an Anglican this is what I wear to identify myself, that I'm a clergyman. "Do you know what Mugabe has done? He's taken people's identity and literally, if you don't mind, cut it to pieces. This is what he's actually done to a lot of - and in the end there's nothing.
"So, as far as I'm concerned, from now on I'm not going to wear a dog collar until Mugabe is gone."
The archbishop said power had gone to the Zimbabwean president's head and the leader did not seem to "realise the suffering of people".
German Chancellor Angela Merkel said Mr Mugabe's policies had "damaged Africa".
Dr Sentamu praised the German chancellor's criticism of Mr Mugabe.
Ahead of the Lisbon summit, key former Portuguese colonies in Africa, including the large and oil-rich nation of Angola, made it clear to Portugal, that they wanted Mr Mugabe to attend.
Some African leaders see Mr Mugabe as an historically important colleague and object to the idea former colonial powers, such as the UK, intervening in African politics.
'World community'
But Dr Sentamu hit out at African leaders for not taking a stand against him.
"It is African leaders who seem to say 'we are backing a revolutionary'. I'm sorry, that is a lot of nonsense. They ought to realise what he has actually done.
"It has become a scourge on the conscience of the whole of world. Why aren't we, as a world community, uniting against Mugabe?" he asked.
Zimbabwe's economy is currently enduring a severe economic downturn.
Only one in five of Zimbabwe's adult population has a job and basic items such as bread, sugar, petrol are often not available in local shops.
+++++++++++++++++++++++++++++++++++++++++++++++++++
No responsibility or liability shall attach itself to either myself or to the blogspot ‘Clerical Whispers’ for any or all of the articles placed here.
The placing of an article hereupon does not necessarily imply that I agree or accept the contents of the article as being necessarily factual in theology, dogma or otherwise.
Sotto Voce