The UK's public sector net borrowing in the year ending in March 2015 was 11% less than it was the year before, the Office for National Statistics said.
In a report that revealed monthly borrowing in April was £2.5bn lower than the same month in 2014, the ONS said the public sector was slowly cutting the deficit and it announced the public deficit went down by 26.9% to £6.8bn ($10.6bn, €9.5bn).
The ONS reported a 3.7% rise in income tax and a slight jump in VAT income as well, which contributed to a decreased deficit. Another factor was the 4.2% decrease in public spending to April 2015, compared to April last year.
In total, central government income over the year was £613.1bn, while spending was £685.4bn. Income tax accounted for 27.7% of the total income and the government spent 58.5% of the total spending on central government departments.
Analysts at IHS Global Insights said that the results were a good step in the right direction of Osborne's plan to cut the deficit for the coming year to 4.0% of the GDP (£75.3bn) in 2015-16.
"Public Sector Net Borrowing excluding banks has come down from a peak... in 2009-10, although the decrease has been markedly less than the coalition government had been aiming for when it came to power five years ago," IHS said. "This is largely the consequence of the economy's recovery taking longer to kick in than had originally been expected."
In the first quarter of 2015, the UK saw a GDP growth of 0.3%, the slowest pace since the last quarter of 2012.
After being reappointed as chancellor of the exchequer, George Osborne urged for an early start to more spending cuts in a speech at the Confederation of British Industry.
"The deficit is less than half what it was, but at just under 5% it is still one of the highest in the developed world," Osborne said. "We've all learnt there's no shortcut to dealing with it – so we have to roll up our sleeves and get on with the hard work."
He urged Whitehall departments to cut spending in order for the government to invest in the NHS and still achieve its goal of removing the deficit by 2018-19.
"The chancellor has a tremendous opportunity at the next budget to transform his department into a modern finance ministry, with responsibility of enforcing fiscal discipline across Whitehall," Sumita Shah, public sector policy manager at the Institute of Chartered Accountants in England and Wales, said.
The chancellor will present an extra budget statement on July 2018.