ChatGPT ：一场并不虚假的革命

date

Mar 15, 2023

slug

the-not-false-revolution-of-chatgpt

status

Published

summary

托马斯-库恩（Thomas S. Kuhn）写道，科学进步既包括正常时期的小修改，也包括非正常时期的大革命。最近，人工智能的不正常进展正接近一个潜在的破坏性阶段，GPT-4 这个最先进的和令人震惊的语言模型被广泛认可就是证明。尽管如此，该模型引起了人们的欣喜和对其影响的争论，其中，Chromesky 的断言既正确又错误。

The !False Revolution of ChatGPT

Thomas S. Kuhn wrote that scientific progress involves both minor revisions during normal periods and major revolutions during abnormal periods. Recently, abnormal advances in artificial intelligence are approaching a potentially disruptive stage, as evidenced by the widespread recognition of GPT-4, the most advanced and astonishing language model. Nonetheless, the model elicits both satisfaction and contentious debate for its ramifications, wherein Chromesky’s assertions are both correct and wrong.

Chromesky is right and not

乔姆斯基的正误

Chromesky's perception of ChatGPT’s profound influence is indeed accurate. The latest results from March 15th reveal that the GPT-4 model has surpassed the vast majority of humans in a multitude of standardized examinations, including the Scholastic Assessment Test (SAT), the Graduate Record Examination (GRE), the Law School Admission Test (LSAT), and the Biology Olympiad. Furthermore, during the 100-plus days since ChatGPT's public release, we have witnessed a burgeoning proliferation of tools, prompts, elation, and astonishment. Merely hours after OpenAI unveiled GPT-4, I received a comment from an American and an inquiry from China. A friend pursuing a PhD at New York University lamented to me today, "We are nothing better than that (GPT-4). We are at “the beginning of the S-curve”, said Satya Nadella. Nadella is the chairman and CEO of Microsoft, a lead investor of OpenAI.

Chromesky 对于 ChatGPT 的强大影响的感受是正确的。3 月15日的最新结果显示， GPT-4 模型已经在大量专业考试上超越了绝大部分的人类（图 1），其中包含了被称为“美国高考”的 SAT、美国研究生入学考试 GRE 、（美国）法学院入学考试 LSAT 以及奥林匹克生物竞赛。除此以外，在 ChatGPT 开放使用的 100多天，我们还看到了如春笋般疯狂涌现的工具、提示（prompt）、欣喜和惊讶。就在 OpenAI 发布 GPT-4 的几小时后，我收到了一条来自美国的评论和一个来自中国的问题。在美国纽约大学攻读 PhD 的朋友今天对我感叹“we are nothing better than that（我们并不比这个高多少，‘这个’指 GPT-4）”。而在中国，已达到退休年龄的母亲问我：“这个 AI 我能用吗？” 我们正处于“（创新扩散的）S 型曲线起点”，萨提亚·纳德拉说。纳德拉是微软的董事长兼首席执行官，微软是 OpenAI 的主要投资者。

图 1. OpenAI 公布的 GPT-4 在人类专业考试的结果

In the domain of anticipating advancements in artificial intelligence, both Chromesky and Google have experienced shortcomings. In 2017, Google introduced the seminal paper, “Attention Is All You Need”, establishing the guiding framework for the GPT and Bert series of models. During that period, the Natural Language Processing sphere was primarily characterized by cumbersome RNNs and CNNs. Nonetheless, Google not only devised the innovative Transformer model but also facilitated the rapid ascent of GPT-X. Unfortunately, despite the preliminary investigations into the new model’s applicability by authors, these gifted individuals could not predict the extraordinary impact this paradigm would have on human society just half a decade later.

但在对于人工智能发展的预见上，Chromesky 甚至 Google 都失败了。2017 年Google 发表的论文 Attention Is All You Need 奠定了 GPT 及 Bert 系列模型的范式基础。当时的自然语言处理界以臃肿的 RNN、CNN 为主流，但 Google 不仅开创了一个新的 Transformer 模型，还让 GPT-X 的疯狂成为了可能。不过可惜，即便作者们初步测试了新模型在其他任务的可能性，这些天才也没有预见这个范式在五年后的今天给人类社会带来的疯狂。

图 2. Transformer 提出者比较谨慎地写到计划将模型拓展到其他任务

疯狂和不间断的疯狂

The astounding fact is that human cognition and artificial intelligence models bear striking similarities. Many individual genuinely immersed in AI studies starts with a tale of perceptrons. In 1943, Warren McCulloch and Walter Pitts simplified the functionality of human brain cells into cells that process and transmit synaptic signals and electrical impulses. Resembling a rudimentary logic gate, these cells convey specific information once accumulated signals surpass a predetermined threshold. This conceptual model of cells is known as MCP. A few years later, Frank Rosenblatt devised an automated algorithm based on the MCP model, learning to manage signal accumulation and transmission weights while producing classification outcomes. This elementary imitation of biological cells not only serves as the precursor to machine learning but also underpins the groundbreaking GPT-4 of 2023.

更疯狂的事实是，人类的思维与人工智能模型确实非常相似。很多真正学习过人工智能的人，都会从一个关于感知机的故事开始。1943 年，Warren McCulloch 和 Walter Pitts 将人的大脑细胞功能简化为处理和传递滑雪信号和电信号的细胞。这种细胞类似一个简单的逻辑门，当接收到的信号累计并超过了某一个阈值，细胞就会将某种信息传递出去。这种细胞的概念模型被称为 McCulloch-Pitts (MCP)。几年后，Frank Rosenblatt 在 MCP 模型的基础上创造了一种自动化算法，学习处理信号累计和传递的权重，并输出分类结果。这种对于生物细胞的简单模仿，不仅是机器学习的前身，也是 2023 年令我们疯狂的 GPT-4 的基础。

This resemblance is also evident in the training process. Every child learns to use chopsticks (or a knife and fork) through a cyclical phase comprising stimulation (observing parents eating), response (child's actions), feedback (environmental or parental feedback), and adjustment (modification based on feedback). This analogous process is not only manifest in the fundamental stages of machine learning—input, guess, loss function comparison, and iteration—but also in the reinforcement learning approach that ChatGPT employs for model enhancement. In essence, AI genuinely refines itself through human learning behaviors.

这种相似性还体现在训练的过程中。每一个小孩都会学习用筷子（或是刀叉）的方式，这个学习的过程大致需要刺激（看到父母吃饭）-反应（小孩行动）-反馈（环境或父母反馈）-修正（根据反馈改变）四过程组成的循环阶段。类似的过程不仅仅体现在机器学习通用的输入-猜测-比较 loss 函数-迭代的基本过程，还是 ChatGPT 选择用于模型提升的强化学习方向——AI 其实确乎是在人类的学习行为提升自己。

Indeed, as some critics have noted, the current state of AI is far from omniscient, yet the persistence of progress is foreseeable. Prior to GPT-4, users, editors, or reviewers could generally discern "what was not created by humans." However, as it sweeps through various human examinations, it becomes increasingly challenging to make such distinctions. Concurrently, the model development team gathers feedback data at an unprecedented pace, enhancing the efficacy of large language models. Nevertheless, as demonstrated in GPT-3's model paper, we have witnessed nearly linear performance improvements with the growth of model parameters (Figure 3), allowing us to predict that teams with greater computational power (permitting the training of more parameters) will effortlessly transform this persistence, following the acquisition of vast feedback data, into private barriers.

确实，正如一些批评家所言，现阶段的 AI 仍非“全知全能”，但进步的持续性是可预见的。在 GPT-4 之前，用户、编辑或者审稿人大致能够辨别“什么不是人类创造的”，而在它横扫各大人类考试之后，可以预见的是我们想要辨别会越来越困难。同时，模型开发团队，正在以前所未有的速度收集更多地“反馈性”数据，这可以用来提高大语言模型的效果。不过，正是因为在 GPT-3 的模型论文中，我们已经看到随着模型的参数量增长，近乎线性的效果提升（图 3），所以我们可以预测，拥有更高算力（允许训练更多参数）的团队，在拥有海量的反馈性数据之后，这种转化的持续性能够轻易地转化为私有的壁垒。

图 3. OpenAI 披露的 GPT-3 模型中参数量与准确率的关系

Reflection on revolution

反思革命

It is crucial to note that acknowledging the genuine existence of a revolution does not imply offering a “true promise,” as Chromesky has voiced concerns, since essential questions await exploration. The first issue is that privatization may intensify social inequality. From Rousseau's perspective, political inequality—distinguished from physical inequality—is characterized by a minority amassing privileges, such as wealth, honor, power, or obedience by prejudice. OpenAI, mirroring the so-called “founder of civil society” who first enclosed a piece of land and claimed ownership, persuading “simple” individuals to believe, has become the first to appropriate public or semi-public data and technology with potentially broad ramifications. In GPT-4's technical report, OpenAI, asserting "competition" and "security" concerns, declines to reveal model specifics. In February, OpenAI, already possessing openai.com (Open Artificial Intelligence Company), expends considerable resources to acquire ai.com (Artificial Intelligence Company) domain ownership—a move some netizens describe as "in pursuit of being not open." These two facts suffice to evoke concerns regarding the deepening of inequality.

必须指出，承认一种革命的真正存在并不是在给出一种“真正的承诺”——就像 Chromesky 所担忧的那样，重要的问题仍然有待探索。第一个问题是，私有化可能加剧社会不平等。在卢梭的眼中，有别于“自然的不平等”的“政治的不平等”的特点是，少数人通过损害他人利益而获取特权，如财富、尊贵、强大或屈服。正如“第一个将一块土地圈起来”并声称所有权继而让“天真”的人相信的，就是所谓“文明社会的创始人”一样，OpenAI 第一个将公有或部分公有的、具备广泛影响可能的数据和技术划归己有。GPT-4 的技术报告中，OpenAI 声称出于“竞争”和“安全”考虑，拒绝公布模型相关细节。今年二月，已经拥有 openai.com （开放的人工智能.公司）的 OpenAI 公司斥巨资购买了 ai.com （人工智能.公司）的域名所有权——被部分网友称为“为了不开放（open）”。这两个事实足以让人产生不平等深化的担忧。

图 4. “鉴于像 GPT-4 这样的大规模模型的竞争状况和安全影响，本报告没有包含关于架构（包括模型大小）、硬件、训练计算、数据集构建、训练方法或类似的进一步细节。”

The secondary issue manifests as an extension of the primary concern, where such privatization may potentially exacerbate micro-level digital divides and restructure geopolitical landscapes. Numerous studies corroborate the digital divide, revealing disparities in accessibility, utilization, and efficacy of digital technologies. Examining the situation from a micro perspective, accepting the two weak premises that “GPT-4 enhances productivity” and "varying individuals exploit GPT-4 differently" allows us to deduce that it bolsters the productivity of those who can employ it more efficiently, subsequently providing a competitive advantage for those capable of capitalizing on its potential. As these individuals often constitute the privileged echelons prior to the advent of such technology, the micro-level digital divide is expected to be magnified. From a macro viewpoint, nations have already imbued artificial intelligence technologies with geopolitical attributes, either passively (e.g., United States AI and artificial intelligence export bans) or actively (e.g., certain countries deliberately blocking specific technologies). Given that ChatGPT represents the inaugural large-scale language model technology, boasting hundreds of millions of users and unprecedented growth, the scope and persistence of such geopolitical implications may be unparalleled in history.

第二个问题则是第一个问题的表现，这种私有化很可能加剧微观数字鸿沟和重构地缘格局。数字鸿沟众多研究支持，一项数字技术的可及性、使用情况和使用效果都会存在差异。从微观来看，只要我们接受“GPT-4 提高生产力”和“不同人对于 GPT-4 的利用存在差异”这两个弱前提，我们就可以推断：它会提高能更高效地使用的人的生产力，继而让更能利用的人处于社会竞争的优势地位。而又因为这部分人，往往是技术出现前的优势阶层，所以微观层面的数字鸿沟将被放大。而从宏观来看，国家间或被动（美国AI与人工智能出口禁令）或主动（某些国家对于某些技术的主动屏蔽）已经为人工智能技术附加了地缘政治属性。考虑到 ChatGPT 是第一个拥有数亿用户、创造增长历史的大语言模型技术，这种地缘政治的影响面和持续性或将史无前例。

To sum up, GPT-4 marks a new chapter in the advancement of science and society. Despite not meeting utopian expectations, we should embrace the power, strive for solidarity, and most importantly, avoid historical tragedies, carefully.

总而言之，GPT-4标志着科学和社会进步的一个新篇章。尽管没有达到乌托邦式的期望，但我们应该拥抱力量，争取团结一致，最重要的是，小心避免历史悲剧的重演。

这篇文章的中英双语是一次刻意的尝试。我在不同段落随心所欲地选用了自己最想用的语言，而其他的一切均由 GPT-4 代我完成。第一段我首先用英文完成，为的是模仿 Chromesky 的开篇方式；第 2-8 段用中文，意在用第一语言快速写完，但我发现自己下意识地在“照顾”AI 转化为英文的难度（仿佛是为一位真实存在的合作者考虑）；最后一段用英文，是为了最后一句话的音节韵律。

The bilingual nature of this article represents a deliberate attempt on my part. I have freely chosen to use the language I most desired in different paragraphs, leaving the rest to be translated by GPT-4. I completed the first paragraph in English, emulating Chromesky's opening style. The subsequent paragraphs, from 2 to 8, were written in Chinese with the intention of quickly jotting down my thoughts in my first language. However, I found myself subconsciously "caring" about the difficulty of the AI's translation into English, almost as if I were considering a real-life collaborator. The final paragraph was written in English, for the sake of the syllabic rhythm of the last sentence.