Snap Inc. IPO at the New York Stock Exchange Getty

After weeks of record highs, the Dow Jones Industrial Average fell 112.58 points, or 0.5%, on Thursday to keep it just above its latest milestone at 21,002.97. Aside from falling stocks, Thursday saw the market debut of Snap, owner of the messaging app SnapChat, which saw its share price soar 44%.

The S&P 500 fell by 0.59% or 14.04 points to close at 2,381.92. Nasdaq, the third main benchmark, also slid - by 0.73% or 42.81 points to end at 5.861.22.

Falling benchmarks have been unusual in the weeks since Donald Trump's electoral victory in the hopes that he plans to push through anti-regulation, business-friendly legislation during his presidency.

Snap's IPO was originally priced at $17 but hit highs of $26.05 during the day's trading before finishing at $24.48. Though the initial rally looked good, many were concerned that Snap was overvalued, with the company struggling to turn a profit. Investors will also not get voting rights, with the founders keeping control over operations.

Other big movers included Abercrombie & Fitch (ANF), which saw stocks rise 13.94% in the day. Having a less than stellar day though, was Caterpillar (CAT) which saw stocks fall 4.3% after investigations heated up over tax issues involving their Swiss subsidiaries.

Chief Investment Strategist at Rockefeller & Co., Jimmy Chang, told "It's not surprising to see the market take a breather today, but there's still a lot of optimism. We're in a period where people can still trade on concepts and visions and promises."