Majority of Horsham councillors to leave after 2020