Visiting Penola, the South Australian town where soon-to-be saint Mary MacKillop taught from the 1860s, Mr Fischer said he would not seek an extension to the job given to him by Kevin Rudd in 2008.
"I want to return home and to my family farm and slow down," said Mr Fischer, who was deputy prime minister in the Howard government from 1996 to 1999.
Mr Fischer was in Penola to check on local preparations for October's canonisation of Mary MacKillop and thank the region's winemakers for donating 117 cases of wine for celebrations in Rome.
He also presented an official Vatican flag to the Mary MacKillop Interpretive Centre.
Mr Fischer said if the Pope's reaction to Australian wine was an indicator, celebrations at the canonisation would be well attended.
He said the Pope's "poker face" was cracked by a gift of dessert wine from Mr Rudd last year.
"This great smile appeared across Pope Benedict's face as he looked down on these beautifully packaged bottles of Australian dessert wine," Mr Fischer said.
"I'm told within 48 hours they weren't to be found anywhere."
The wines were accompanied by a copy of Mr Rudd's apology to the Stolen Generations.
Asked about Mr Rudd's political demise and election-related matters, Mr Fischer stayed silent.
"I have zero comment on domestic political issues," he said.
Mr Fischer inspected the fallout from a weekend tornado that damaged a skylight at the Mary MacKillop Interpretive Centre and the roof of the old school house where MacKillop taught.
SIC: TAAUS