Unusually though, for a normally quiet Saturday morning in Ashburton, politics will boil over into a street march.

The Andrews government wants to develop 1.4 hectares of prime Ashburton land, and the plan has residents up in arms.

The site is the dilapidated Markham public housing estate where 56 units once stood.

"We are really not a militant bunch, but this has got everyone fired up," says one organiser, Susan Rayner, from the Ashburton Residents Action Group.

The government wants to shoehorn 62 new public housing apartments into the worst quarter of the site.

The remaining land - overlooking bucolic parkland, Gardiners Creek and a golf course - will be private development, with 188 apartments built in three towers rising to five levels.

Many of these apartments will feature stunning views of the creek, nearby sports fields and the golf course.

Planning powers will be taken off the local Boroondara Council, and handed to a state government agency.

Ms Rayner said the move to privatise most of the site to cover the cost of building public housing was abandoning what Labor was meant to stand for.

"I was born into a family who always voted Labor because the party used to have a reputation for giving the underdog a fair go," she said. "I'll never vote Labor again."

Public housing waiting lists stand at more than 33,000.

Susan Rayner outside the development site. Photo: Jesse Marlow

Ms Rayner said the government should be keeping the entire site for a big increase in public housing, not just replacing what was there on one corner and then selling the rest of the land to balance the books.

"We don't want the homeless being stuffed into dog boxes, while the rich are provided million-dollar views," she said. "And we don't want profit-driven, substandard apartments for the poor."

But Raoul Wainwright, from the Victorian Public Tenants Association, said there had been no public outcry over the delay to rebuilding the public housing.

"The old public housing ... was in atrocious condition," he said, and there had been a promise to rebuild it speedily.

He said it was more than a year since the 56 public housing units had been demolished, and every day construction was delayed meant "62 vulnerable Victorian families goes without a home".

He said replacing the lost public housing was what should be sparking outrage, not the impact of other projects on peoples' amenity. "We desperately need more public housing."

The government is understood to be considering placing a height limit on the land and undergoing a modest reworking of the proposal, in an attempt to pacify locals.

A spokesman for the Andrews government said the application was being assessed by the planning department.

"A final decision will strike a balance between the need to provide more social housing and the importance of protecting the amenity of the area," he said.