The International Institute for Strategic Studies (IISS), a well-known research institution, recently released a report stating that although the United States restricts the export of high-performance AI chips to China, these control measures may push Chinese artificial intelligence researchers towards “areas with less computational requirements” and Guide them to develop a “new competitive advantage”.
In recent years, the capabilities of large language models have improved significantly, and OpenAI’s creation of GPT-3 in 2020 is an important milestone, the report said. These improvements are attributed to the creation of larger, more general model architectures and increases in dataset sizes and the amount of money technology companies spend to increase the computing power to train models. Empirical studies have shown that there is a strong relationship between dataset size, computational overhead, and parameter count for a given model, and in practice computational overhead is the strongest constraint on model improvement.
Furthermore, large language models are being developed by an increasing number of actors, with diffusion across multiple dimensions. Research on large-scale language models, for example, has largely taken place in the US, but researchers in other countries—notably China, and specific research institutions elsewhere, such as DeepMind in the UK—have devoted significant resources to building their own models. Furthermore, the types of institutions developing language models have expanded to include both large tech companies such as Google and Microsoft, as well as scattered collectives of researchers.
The report argues that the proliferation of large-model technology has two near-term implications for security: These models may produce higher-quality and more content for disinformation, and competition for the development of large models may increase geopolitical tensions.
The report also analyzed last October, when the U.S. government announced new export controls on advanced semiconductor chips flowing to China, in part because they are critical to the development of artificial intelligence. While the stated intent of export controls is to limit the development of artificial intelligence systems for surveillance or military applications, language models are also highly dependent on these advanced semiconductors. Part of the real motivation for imposing export controls may be to preserve the U.S. advantage in language models, either as part of a broader strategy to compete in AI technology or because the government specifically wants to curb Chinese language model development.
The report also cast doubt on the effectiveness of these measures. Although large language models have improved due to increased computing power, they cannot continue to do so at the current rate. Researchers are actively seeking to develop more computationally efficient ways to train similar models. An attempt by one country to limit another country’s computing power as a means of competing for AI development may incentivize the targeted country to develop a competitive advantage in these more computationally efficient methods of AI.
Furthermore, high-quality text such as books and academic journal articles may soon become a more pressing constraint on the development of language models than the availability of computing power.
Countries may also increasingly view the development of large models as a matter of national pride, a perception that may increase competition for their development. The report worries that, as big models move closer to the center of national technological competition, governments could become more aggressive in cutting off their own citizens’ access to language models developed in competing countries, further fragmenting the internet.
Post time: Apr-20-2023