Baidu's Ernie Bot makes significant strides, beats ChatGPT on key tests

Chinese technology company Baidu claims its recently unveiled Ernie Bot has managed to outperform OpenAI's AI bot ChatGPT on some metrics. To recap, Baidu revealed its ChatGPT rival during a live-streamed release event back in March.

SANS Institute's David Hoelzer raised questions about cybersecurity following the debut of the Ernie Bot. Interestingly, Baidu CEO Robin Li admitted that the Ernie bot isn't perfect during the launch event. However, the top executive claimed the bot has been trained on a set of 550 billion facts, according to a CNBC report.

Baidu's new AI makes significant progress

The AI Bot is based on the company's foundational AI model dubbed Ernie. To those unaware, a foundational model alludes to AI (artificial intelligence) that's trained on huge amounts of data. The latest version of Baidu's model is called Ernie 3.5. In its latest statement, the Chinese search engine revealed that Ernie 3.5 managed to beat ChatGPT hands down in a slew of benchmark tests.

It is worth noting that OpenAI's ChatGPT is based on the American AI company's GPT 3.5 model. According to a report by China Science Daily journal, Ernie 3.5 surpassed OpenAI's more advanced model, GPT 4, in Chinese language tests. It is no secret that tech giants are sparing no effort to enhance their AI tools owing to cutthroat competition.

For instance, Russia introduced its ChatGPT rival called GigaChat in May. Likewise, Alibaba announced its own ChatGPT-style AI model called Tongyi Qianwen to compete with Microsoft, Google, and OpenAI earlier this year. There's intense competition taking place between US-based and Chinese technology giants in the area of generative AI.

Ernie Bot undergoes various tests

The Chinese science journal cited a test which was based on standard admission and qualification exams. These exams are usually required to qualify as a lawyer or get into college. Apparently, Baidu's Ernie 3.5 beat ChatGPT and GPT 4 in Chinese. While it was ahead of ChatGPT in English as well, Ernie 3.5 couldn't beat GPT 4 in the language.

Another Chinese-focused test comprised more than 13,000 multiple-choice questions and covered over 50 different subjects, Ernie 3.5 managed to surpass both ChatGPT and GPT 4 in this evaluation as well.

The third test featured questions on a wide range of subjects including science and humanities. Much to Baidu's chagrin, Ernie 3.5 couldn't beat both ChatGPT and GPT 4 in this test, which was developed by a group of U.S. universities.

It is no secret that Baidu is leaving no stone unturned in a bid to be at the forefront of the AI space. The company recently opened up about its advancements. CTO of Baidu Haifeng Wang shed some light on the company's progress in the AI department.

"Just three months after the beta release of Ernie Bot, Baidu's large language model (LLM) built on Ernie 3.0, Ernie 3.5 has achieved broad enhancements in efficacy, functionality, and performance," Wang said in a statement. According to the executive, these improvements reflect in code generation, Q&A, creative writing, reasoning, inference performance, and training performance.

According to Baidu, one of the most notable features of its latest model is the ability to access plugins. These plugins are basically apps that you can access with the help of Ernie Bot. For example, the Ernie Bot uses a plugin to offer Baidu's search feature. To recap, ChatGPT started offering plugin support in March.

Ernie Bot v2.1.0, which was released on June 21, included a new ChatFile plugin. Moreover, it added improved capabilities in creative writing, as well as mathematical computations. These enhancements draw their power from Ernie 3.5. According to Dr. Wang, test users can access the service to get an idea of how Ernie 3.5 performs.

"Any applications involving language, text, or code can potentially utilise Ernie Bot," Dr. Wang explained.