Toby Young has quit his new job in the Office for Students, claiming the backlash to his appointment had become a "distraction from its vital work".
There had been ongoing calls for the controversial right-wrong columnist to be removed from the post due his past comments and tweets which were considered offensive, including those aimed at women and working class students.
In the wake of the outcry, the associate editor of the Spectator was found to have deleted tens of thousands of tweets, including one describing how one woman he believed was Danny Boyle's wife had "huge knockers" and another in which he he joked about US cooking show host Padma Lakshmi, saying "I had my d**k up her a**e".
In 2012, during prime minister's questions, he also tweeted: "Serious cleavage behind @Ed_Miliband's head. Anyone know who it belongs to?"
Elsewhere, Young – who co-founder of the West London Free School – was also criticised for his comments regarding education. In a 1988 book The Oxford Myth, Young described working class students at Oxford as "stains" and also described how a teacher's job is "not that tough" during an 2013 interview.
Announcing his decision to step down in the Spectator, Young writes: "The caricature drawn of me in the last seven days, particularly on social media, has been unrecognisable to anyone who knows me.
"I am a passionate supporter of inclusion and helping the most disadvantaged, as I hope my track record of setting up and supporting new schools demonstrates. But some of the things I said before I got involved in education, when I was a journalistic provocateur, were either ill-judged or just plain wrong – and I unreservedly apologise."
MPs including Lib Dem leader Vince Cable, shadow secretary of state for work and pensions Debbie Abrahams and Labour's shadow education secretary Angela Rayner, were among those to urge the prime minister to sack Young due to his unsuitability for the role.
Speaking to the BBC's Andrew Marr Show, May said she was "not impressed" by his comments and would surely lose his job if he made them again in the future.
She added: "When he was appointed, I was not aware of these comments that he had made. Frankly I am not at all impressed by those comments.
"He is now in public office and as far as I am concerned, if he was to continue to use that kind of language and talk in that sort of way, he would no longer be in public office."