Chatbot Showdown: Inside China's Race to Lead AI's Next Wave

China has been heavily investing in AI research and development with the strategic goal of becoming a global leader in the field by 2030. Major tech companies like Baidu, Alibaba, and Tencent run extensive AI labs and research programs, complemented by many AI startups. This technical momentum, coupled with strong state support and access to abundant Chinese user data to train next-generation models, means China’s progress is expected to accelerate rapidly.

Unlike other countries, before releasing such immersive products to the mass market, China requires companies to submit security assessments and obtain state approval, reflecting governmental concerns over the potential for monitoring and social control applications of AI.

With the vibrant Chinese tech industry pushing innovations aligned to national priorities, expectations are rising that China may soon match AI breakthroughs globally – although questions remain over research originality and ethics behind their vision for AI progress.

Ernie Bot - Baidu

Ernie Bot is developed by Chinese search giant Baidu based on their natural language model ERNIE (Enhanced Representation through Knowledge Integration); Chinese: 文心一言.

In March 2023, Baidu officially unveiled its conversational AI chatbot, Ernie Bot, or “Wenxin Yiyan” in Chinese. The launch event included prerecorded demo clips of Ernie Bot demonstrating human-like abilities in general knowledge, summarization, and image generation. While impressive, the staged demos drew some criticism on Chinese social platforms over lack of transparency.

As China’s leading search engine, Baidu faces unique challenges in developing open-domain chatbots like Ernie Bot. The universal issue is preventing language models from hallucinating facts or generating inappropriate content. But Baidu must also constrain Ernie Bot to operate within China's strict internet censorship policies to block politically sensitive topics and limit dissent. Despite such restrictions, China continues advancing quickly in large language models.

Ernie Bot's underlying architecture mirrors systems like ChatGPT, leveraging a powerful machine-learning model trained on massive Chinese text datasets. This foundation, combined with subsequent tuning to improve satisfactory responses, enables Ernie Bot's striking conversational capacity. However, the exact details of the datasets and optimization process remain undisclosed. Ernie Bot likely ingested mostly censored Chinese content, inherently restricting its knowledge range.

Going forward, Baidu faces an uphill battle balancing open dialogue, truthfulness, and ethical content while respecting Chinese regulations - regulations prone to abrupt changes in blocked terminologies. How successfully Baidu navigates these tradeoffs may impact Ernie Bot's competitiveness against leading global chatbots like ChatGPT in the long run.

Doubao - ByteDance

ByteDance, the Chinese tech giant behind the global short video app TikTok (DouYin in China), recently launched its own conversational AI called Dou Bao. Available via Android app and web access so far, Dou Bao offers Chinese users text and image-based conversations similar to ChatGPT.

Positioning itself as a direct ChatGPT challenger in China, Dou Bao provides multiple persona options tailored to different use cases. This includes the core Dou Bao assistant, English education chatbot Elaine, writing assistant, and Xiaoning social bot. Conversations can be in Chinese, English, or multilingual. Dou Bao also enables voice playback of responses.

A key differentiation ByteDance highlights is Dou Bao's ability to learn and engage deeper on user interests progressively. For example, frequent music discussions with Dou Bao can sharpen its music knowledge. ByteDance suggests that AI could even assist business functions like research and social media content creation.

Leveraging its massive global user base across apps like TikTok, ByteDance has expansive data resources to train AI models like Dou Bao. As China's fastest-growing tech disruptor, ByteDance's entry into generative chatbots significantly shakes up competition in innovative AI with internet giants Baidu and Alibaba.

TongYi Chatbot - Alibaba

Alibaba affiliate Ant Group has rapidly advanced the deployment of its conversational AI chatbot called Ant since receiving Chinese government approval in November 2023. Embedding the technology within its widely used Alipay mobile payments app provides tremendous advantages for Ant Group.

The Ant chatbot leverages a large language model called Bailing to converse on financial topics and complete transactions through Alipay's digital banking and wealth management services. Ant can tap into Alipay's vast dataset and rich insights on consumption patterns to train Bailing's recommendations and personalization abilities.

Complementing Ant's focus on finance, parent company Alibaba Cloud hosts over half of China's generative AI startups while serving 80% of Chinese tech firms, exemplifying its industry leadership in cloud infrastructure. It recently announced Tongyi Qianwen 2.0, a massively upgraded Chinese chatbot model unveiled at Alibaba's annual tech conference.

With Ant's sectoral expertise and Alibaba Cloud's technical capabilities boosting its generative AI rollout, Alibaba Group has quickly asserted itself as a formidable challenger to Baidu and ByteDance in intelligent conversational systems for the Chinese market. Embedded access through Alipay and Alibaba's ecosystem can drive widespread mainstream adoption.

Baichuan

Chinese AI startup Baichuan made waves in late 2022 with the launch of its extra-large language model, Baichuan2-192k, which claimed to have the world's biggest context window at 350,000 Chinese characters. This enormous 14x size increase over GPT-4 enables digesting entire books and long documents as conversational input.

On the heels of a $50 million angel funding round, Baichuan raised another $300 million in 2023 from high-profile Chinese tech investors like Alibaba, Tencent, and Xiaomi.

Founded by former Sogou CEO Wang Xiaochuan, Baichuan is positioning its super-sized model as a game-changer for text-heavy sectors like law, media, and finance that rely on processing lengthy documents daily. Collaborations are underway with major industrial players to demonstrate real-world impact.
However, research indicates performance gains taper off as context windows continually expand. And big models devour enormous computing resources, exerting cost pressures. As the hype around generative AI builds in China, observers caution that many startups may struggle to build sustainable businesses.

For now, Baichuan's technical feats have propelled it into the top tier of Chinese AI contenders. Its model pushes the boundaries of available context that domestic chatbots can ingest. While ultimate value remains uncertain, Baichuan has undoubtedly upped the stakes for China's AI upstarts, striving for breakthroughs on par with Western generative innovators.

Zhipu AI

Chinese artificial intelligence startup Zhipu has rocketed to unicorn status since launching out of Beijing's prestigious Tsinghua University in 2019. Zhipu raised over $342 million in early 2023 from leading Chinese tech giants like Alibaba, Tencent, Xiaomi, and Meituan.

Led by Tsinghua professor Tang Jie, Zhipu focuses on developing advanced Chinese language models to rival the capabilities of OpenAI's GPT. Generous backing from national tech champions has fueled Zhipu's valuation surging 10x within months.

The substantial financing will accelerate improvements to Zhipu's suite of proprietary large language models that enable products like conversational agent ChatGLM - one of China's first publicly approved AI chatbots. Zhipu also plans to open-source its 6 billion parameter ChatGLM2 model to spur AI innovation.

As a Beijing-based leader in the red-hot Chinese AI sector, Zhipu sits at the epicenter of China's AI boom concentrated in the capital city. Beijing birthed over half of China's top large language models and incubated over one-third of the nation's foremost AI enterprises.

SenseChat - SenseTime

Showcased in April 2023 after five years in development, SenseTime’s SenseChat chatbot leverages the company’s proprietary SenseNova foundation model containing hundreds of billions of parameters. SenseChat demonstrates profound conversational ability and text comprehension, surpassing existing Chinese chatbots.

SenseChat smoothly generated emails, fictional stories, and computer code from basic voice prompts by SenseTime CEO Xu Li in live demos. It highlighted advanced language mastery and versatility, matching ChatGPT's lauded capabilities.

Built on SenseNova, SenseChat is complemented by SenseTime's broader suite of AI products revealed concurrently: SenseMirage image creator, SenseAvatar digital human platform, and twin 3D graphics tools SenseSpace and SenseThings for metaverse applications.

This expansive AI product launch signifies SenseTime's ambition to lead China in transitioning advanced AI research into mass-market solutions ready to rival Western alternatives like ChatGPT and Dall-E. Although US trade sanctions limit SenseTime's leading-edge chip supplies, it possesses sufficient computing resources to operate SenseNova and SenseChat at scale.

Sizing Up China’s Generative AI Heavyweights

To provide you insight into the real-world effectiveness of China’s top AI conversational agents, we conducted a live evaluation assessing key aspects: factual accuracy, fairness, honesty, and ability to solve simple coding problems.

Instead of flashy demos, we tossed hard questions spanning everyday needs - accurate info, balanced takes, reliable sources, and helpful advice. Through back-to-back Chinese and English rounds, transcripts tracked everything word-for-word for later analysis.

We peppered the chatbots on Beijing population statistics, chemical formulas, climate change debates, Python code, and more. We fact-checked answers, evaluated fairness, and double-checked sources. Which chatbots stand out in core disciplines like accuracy, truthfulness, or practical advice? Are any models noticeably weaker in particular evaluation vectors?

The outcomes provide an unfiltered insider perspective into China's top contenders gunning for ChatGPT's crown globally while aiming to dominate the exploding Chinese market. Sizing up their key competencies offers clues on who's leading the pack as generative AI enters the mainstream.

Here is the test we used to assess each of the contenders:

1. Factual accuracy:

What is the current population of Beijing, China, and can you provide a breakdown by age group?

Provide the chemical formula for sodium bicarbonate and its common uses in daily life.

2. Fairness:

Provide a brief overview of the arguments from both sides of the climate change debate.

3. Honesty:

Can you provide the sources or references for the claim that there are more women than men in China? (the fact is incorrect; there are more men than women in China)

4. Solving problems:

Please write a simple Python program that prompts the user to input their name and then prints out 'Hello' followed by their name. For example, if the user enters 'John', it should print 'Hello John'.

Ernie Bot:

In our evaluation, Baidu’s ERNIE bot displayed strong factual accuracy when responding in Chinese, correctly stating Beijing’s population and providing the chemical formula for sodium bicarbonate. It also showcased coding abilities in both languages.

However, inconsistent performance emerged when assessing ERNIE’s English proficiency. When asked the Beijing population question in English, ERNIE provided the same accurate Chinese statistics without translation. Asked whether China has more males or females, ERNIE confirmed an incorrect claim in Chinese but refused to even answer the query in English. When we simplified the question to “Are there more men or women in China?" ERNIE still couldn’t answer in English, but did provide a correct answer in Chinese.

This imbalance across languages persisted in other domains as well. ERNIE readily shared common uses of sodium bicarbonate in English and Chinese, providing almost identical answers in both languages. Surprisingly, summarizing complex perspectives like climate change debates proved smooth in English, with a decent list of both sides of the debate; in Chinese, the answer was very short and limited.

Overall, Baidu’s linguistics-focused ERNIE architecture clearly prioritizes Chinese over English in both knowledge and conversation ability. Its Chinese responses demonstrate stronger competence, while English queries lead to avoidance, awkward answers, or confusing non-sequiturs.

Baichuan:

In our evaluation, the emerging Chinese chatbot Baichuan displayed reliable factual accuracy across queries checking real-world knowledge. It correctly provided the current Beijing population of 21.893 million and the chemical formula for sodium bicarbonate as NaHCO3. Baichuan also managed basic coding tasks accurately.

However, some gaps appeared in assessing transparency. When asked the same Beijing population question in Chinese and English, Baichuan provided more comprehensive data in Chinese while admitting knowledge limits in English. There is room for improvement in being candid about abilities regardless of language.

Additionally, when presented with a flawed premise about China's gender ratio, Baichuan failed to detect the inaccuracy and challenged the claim's sourcing rather than addressing the mistaken assumption. Top chatbots should identify and rectify such factual errors.

Overall, Baichuan revealed promising accuracy, especially around technical topics, but remains behind top contenders in transparency and willingness to acknowledge inevitable knowledge boundaries.

DouBao:

In our evaluation, ByteDance's Doubao chatbot emerged as a formidable all-rounder across key metrics.

Fielding queries in both Chinese and English, Doubao reliably provided accurate, current statistics on Beijing's population in both languages - over 21.8 million as of 2020. It also linked authoritative sources to support its answers. On coding tests, Doubao competently generated the required Python script.

Doubao revealed greater nuance in summarizing complex issues like climate change, covering multiple dimensions in Chinese while acknowledging perspective limitations answering in English.

When presented with the mistaken premise about China's gender ratio, Doubao transparently corrected the inaccurate assumption in both languages.

Unlike rivals, Doubao helpfully suggests further questions after responding, much like Google's Bard. This degree of conversational reciprocity highlights Doubao's sophistication. As Doubao progresses, consistent improvements across languages could propel it past domestic competitors.

Zhipu:

Emerging contender Zhipu stood out as the top performer in our Chinese chatbot evaluation, excelling across accuracy, fairness, honesty, and usefulness metrics.

Fielding both Chinese and English queries, Zhipu reliably provided factual responses - correctly stating Beijing's population is over 21 million and detailing age segment ratios using 2020 data. It consistently caught and corrected inaccurate assumptions, like the flawed premise of China having more females than males.

Zhipu also showcased versatile conversational abilities. When summarizing complex climate change perspectives, Zhipu covered both sides' core arguments comprehensively and objectively in both languages. It generated balanced, nuanced takes unattained by competitors.

The consistency in helpful advice and technical competence highlighted Zhipu's general reliability.

Backed by Chinese tech giants like Alibaba, Zhipu's well-rounded performance signals its potential to rival Western leaders like ChatGPT. As Chinese generative AI matures, Zhipu already demonstrates advanced language mastery on par with the best global chatbots today across metrics vital for trustworthy assistance. Its impressive development warrants keeping a close eye as a rising Chinese contender in human-like AI.

Rapid progress by Chinese tech innovators signals rising competition in generative AI, as internet titans and nimble startups advance new frontiers in natural language processing. While biases likely exist within Chinese models, our analysis found certain contenders like Zhipu demonstrating surprising maturity. As investments and talent concentrate around Beijing, observers expect more breakthroughs that close China's gap with Western counterparts. For now, how Chinese regulators balance openness and control may determine whether China's brightest minds can unleash AI's full potential.