The government will not force local councils to hike rents to near the market rate for higher-earning council tenants in a U-turn on its controversial "pay-to-stay" policy.

A new set of earnings thresholds, meaning those who live in social housing would have to pay higher rent, was set to be compulsory, but councils will now be given discretion instead. They are already able to raise social rents for higher earners.

"Since the summer, the government has been reviewing this policy," said Gavin Barwell, the Conservative housing minister, in a written statement. "We have listened carefully to the views of tenants, local authorities and others and as a result, we have decided not to proceed with a compulsory approach. Local authorities and housing associations will continue to have local discretion.

"The government remains committed to delivering its objective of ensuring that social housing is occupied by those who need it most. But we need to do so in a way that supports those ordinary working-class families who can struggle to get by, and in a way which delivers real savings to the taxpayer. The policy as previously envisaged did not meet those aims."

Lord Porter, chairman of the Local Government Association, said the announcement "will be a huge relief to households across the country".

"Making pay-to-stay mandatory would have affected thousands of social housing tenants across the country, with the average affected households seeing their rents rise by £1,065 a year," Lord Porter said. "Councils would have needed to invest millions in new IT systems, hire new staff and write to over a million social housing tenants to try and understand household income and approve individual tenant bills."

John Healey, Labour's shadow housing minister, welcomed the news on pay-to-stay, but said now the rest of the Housing Act needs a "rethink". Among other controversial policies introduced by the 2016 Act are the abolition of lifetime tenancies for new council tenants and obliging local councils to sell their most valuable vacant homes.