A strange extinct marine animal that has baffled palaeontologists for nearly two centuries has been finally assigned to the tree of life.

Hyoliths are invertebrate animals with cone-shaped shells that roamed ocean floors around the world more than 500 million years ago.

The shells were equipped with a trapdoor at the front and, in some species, a pair of tentacles for feeding.

The features of the animal did not obviously resemble any extant group, with scientists believing that they were closest to molluscs such as snails, clams and squid.

But a team of scientists led by a young palaeontologist working with the University of Toronto has concluded that the creatures are members of a group called lophophorata and are closely related to brachiopods.

The finding has now been published in the journal Nature.

Brachiopods have a soft body enclosed between upper and lower shells, as opposed to the left and right shell arrangement in molluscs.

Joseph Moysiuk, an undergraduate at the University of Toronto, made the discovery after studying newly discovered hyolith fossils from the Cambrian Burgess Shale in Canada.

"Our most important and surprising discovery is the hyolith feeding structure, which is a row of flexible tentacles extending away from the mouth, contained within the cavity between the lower conical shell and upper cap-like shell," he explained.

"Only one group of living animals – the brachiopods – has a comparable feeding structure enclosed by a pair of valves.

"This finding demonstrates that brachiopods, and not molluscs, are the closest surviving relatives of hyoliths.

"It suggests that these hyoliths fed on organic material suspended in water as living brachiopods do today, sweeping food into their mouths with their tentacles."

Hyoliths appear on the fossil record around 530 million years ago during the Cambrian Explosion, when life became more diverse and abundant in the world's oceans. They were first described in 1840.

"Resolving the debate over the hyoliths adds to our understanding of the Cambrian Explosion," said Martin Smith at Durham University, who co-authored the Nature paper.

"Our study reiterates the importance of soft tissue preservation from Burgess Shale-type deposits in illuminating the evolutionary history of creatures about which we still know very little."