Leading Chinese computer scientists have suggested the nation can build large language models without imported GPUs, by using supercomputers instead.
This was the belief expressed at the 2024 China Computing Power Development Expert Seminar, a conference co-organized by a Chinese industry alliance and national standardization body, and depicted on state-sponsored media on Sunday.
Zhang Yunquan, a researcher at the Institute of Computing Technology of the Chinese Academy of Sciences, reportedly acknowledged the US has taken action to “choke” China’s AI development, “including banning the sale of high-end GPUs, terminating the sharing of source code for large models, and interrupting ecological cooperation.”
“When large models require 10,000 to 100,000 GPUs, it is essential to overcome technical challenges like high energy consumption, reliability issues, and parallel processing limits by developing specialized supercomputers,” he concluded.
Zhang suggested that the pathway for China to keep pace with global AI development, and “break through the bottleneck of large model computing power in the short term”, is to use its two decades of experience developing advanced supercomputing technology and create machines to crunch large models. Chinese supers appear to have used locally-designed chips.
Reflecting on the restrictions, Chinese Academy of Sciences academic Qian Depei mused that academics and researchers in the Middle Kingdom “still have to conform to China's national conditions and cannot completely follow the Americans."
- AI models just love escalating conflict to all-out nuclear war
- Nvidia can't sell its best chips to China, but India is more than happy to take them
- Nvidia sees Huawei, Intel in rear mirror as it grapples with China ban
- Quantum computing next (very) cold war? US House reps want to blow billions to outrun China
The hope of some of researchers in attendance was that “super-intelligence fusion”, the integration of supercomputing and intelligent computing, will see China thrive.
"The combination of supercomputing and intelligent computing is inevitable, and there will be an organic integration, rather than simply putting them together," reportedly predicted Chen Runsheng, a Chinese Academy of Sciences boffin.
Zhang noted that China has invested huge amounts of money in the development of intelligent computing power in recent years.
Although China claims it’s got the experience to rely on supercomputers, there’s really no telling as they’ve stopped reporting details of tech employed in the nation’s mightiest machines and therefore missing out on spots on the famed TOP500 supercomputer scoreboard, alleging that disclosure would have negative effects.
“Setting aside China's progress in supercomputing, the key issue is that while the TOP500 list increasingly serves as a tool for the US Department of Commerce to sanction and suppress Chinese supercomputers, they still expect Chinese institutions to willingly participate and be targeted,” stated a piece in Chinese state-sponsored media The Global Times last week.
“’Entrapment’ is by no means an alarmist term,” the author asserted.
The Global Times cited US sanctions on 20 Chinese entities – seven supercomputing business and 13 more for their potential to make weapons of mass destruction and other tech of national security concerns as proof of “entrapment.”
It then argued that TOP500 co-founder Jack Dongarra “repeatedly stated that China has faster supercomputers” and has speculated that “Chinese supercomputer power may exceed all other countries.”
The Register met Dongarra at an event called the International Symposium On Blockchain Advancements in 2023, and he expressed regret that China no longer shares info with the TOP500 list.
“I like competition,” explained Dongarra, He attributed the dilemma to “geopolitical issues.”
According to Dongarra, in 2022, 162 Chinese machines were on the list, compared to 125 in the US. He added that there was rumoured to be two or three exascale machines in the Middle Kingdom.
China is already making progress with AI, with or without GPUs. On Monday, it developers released what they claimed is the world's “first large-scale seismic data processing model with 100 million parameters,” according to state sponsored media. The model, called "DiTing", was developed by the National Supercomputing Center in Chengdu in cooperation with the Institute of Geophysics of the China Earthquake Administration and Tsinghua University ®