A survey by the business lobby group, the Confederation of British Industry (CBI), showed that confidence in the UK financial services industry declined for the second consecutive quarter.
The CBI/PwC Financial Services Survey, covering 111 financial firms across the UK, found that 13% of firms were more optimistic and 29% were less optimistic in the June quarter, giving a balance of -16%.
While that represented an improvement on the -21% reading in the three months to March, it was the first time optimism saw a second consecutive quarterly decline in four years.
Survey respondents blamed stronger competition, rising uncertainty about demand and slowing profits for their continued pessimism.
The survey comes one week before the in-out referendum on the UK's European Union membership, with most in the financial sector in favour of staying in the single market.
The downbeat assessment from the financial services industry, which contributes around 10% to the UK's gross domestic product, will add to fears that a Brexit could plunge the economy into a prolonged slowdown.
"There's a mood of caution amongst financial services with the vote on our EU membership rapidly approaching and global economic waters still choppy," said CBI chief economist Rain Newton-Smith.
"When talking to financial services firms, it's clear that the low interest rate environment, increasing competition and regulatory pressure continue to weigh on profitability."
But he added that there were some positive signs in the sector, with business volumes expanding and employment levels remaining stable.
Andrew Kail, UK financial services chief at PwC, added: "The UK now stands at the crossroads of continued EU membership – the outcome will be keenly awaited by financial services firms.
"Financial services are vitally important for the UK economy – generating jobs, income, investment and exports."
Multiple opinion polls suggest that support for exiting the EU is mounting in the UK, with a YouGov poll for the Times published earlier this week showing 46% support for Vote Leave and only 39% backing Vote Remain.
The CBI has been vocal in its support for Remain campaign, saying that 80% of its members would vote to stay in the EU.