Public sector net borrowing reached an eight-year low in May 2015, falling to £10.13bn (€14.2bn, $16.05bn), a decrease of 18.31% compared to May 2014, the Office for National Statistics said on 19 June.
Borrowing went down 9.44% in the financial year to March 2015, compared to the year to March 2014, to £89.2bn, which is 4.9% of the GDP.
The main reason for the narrowing deficit was a 5.3% rise in income tax in May, up to £10.8bn.
In a statement, Howard Archer, chief economist at think tank IHS Global Insight, said the lower public borrowing was "very pleasing news for Chancellor George Osborne ahead of his 8 July budget as the public finances saw further appreciable year-on-year improvement in May".
He added: "Despite the improved performance so far this fiscal year, major questions remain ahead of the extra budget on 8 July over the new Conservative government's ability to meet its ambitious fiscal targets over the longer term."
Osborne has years of severe cuts in spending ahead and in the first two months of the 2015-16 fiscal year, the government has been running behind schedule.
Investment adviser Investec said: "It will be interesting to see whether he includes an additional £8bn per annum of resources for the NHS by 2020 in the spending 'envelope', as the Tories promised in the election campaign."
Of the government's expenses in May, investments, or capital expenditure, saw the biggest growth of 8.7% to £2bn.
After the Conservatives won the general election in May, George Osborne, who was reinstated as chancellor, announced major savings in government departments in order to cut the deficit. He will hold an extra summer budget statement on 8 July.