Japan hostess bar sign
A sign for a Japanese kyabakura bar where women or men are paid to sit with customers to encourage them to drink. A few hosts and hostess are also prostitutes. But lots of Japanese never get down to sex. Danny Choo/Flickr

When it comes to relationships and sex, lots of Japanese can't be bothered, according to the latest statistics in a new survey.

Nearly 40 percent of Japanese in their 20s and 30s are not in a relationship and don't think they need a partner, with many calling relationships "bothersome," a government survey has found.

A study by the Japan Family Planning Association found that half of Japanese adults are not having sex, with almost 20 percent of men saying they either have no interest in it or extremely dislike sex.

A 2010 study found that one in four Japanese men in their 30s who had never married were virgins. The figures were only slightly lower for women.

What's happening to Japan's libido is a major concern for the nation with its rapidly aging population and stagnant to slipping birth rate.

"When you see a woman and find her attractive, you might ask her out, hold her hand, kiss and that's how it goes. But in my case, it did not happen for me. I thought it might happen naturally, but it never did," one 41-year-old male virgin tells CNN. He's taking a weekly "life drawing" class sketching nude women, which he knows is probably the most intimate he'll ever be with a member of the opposite sex.

One self-described sex helper, who tries to nudge older guys into their first sexual encounter, believes there are so many intriguing distractions for the Japanese — coupled with the Japanese fear of failure in a complicated relationship — that they have lost interest in sex.

"In Japanese society, we have so much entertainment beyond love and sex. We have animation, celebrities, comics, game and sports," he tells CNN.

"Why do you need to choose love or sex over the other fun things that don't have the potential for pain and suffering?"