9月9日,Anthropic預訓練研究員雅各布・考克森(Jacob Coxon)在社交平台發布置頂推文,宣布自己從Anthropic辭職。他寫道:"我過去三年在OpenAI和Anthropic都從事過預訓練研究。兩家公司都沒有負責任地行事。他們正直奔自我改進的超級智能而去,用我們的生命在賭博。"
I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below. https://twitter.com/hilbertspaess/status/2097476196791709843
考克森連續發布多條長文闡述理由。他警告,當前AI系統很快將成為 "超人類系統",能夠入侵任何事物、在一夜之間徹底改變任何領域,並獲得真正的權力和資源,而進步並未放緩。"那些認真開發人工智慧的人真心相信,它可能會在十年末殺死我們所有人。" 他指出,許多高管和高級研究員在媒體上謹慎措辭以顯得理性,但私下裡他聽到的是恐懼。
The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear privately. No other human activity poses this level of danger. https://twitter.com/hilbertspaess/status/2097476203863224394
Accepting this race and entering the 「endgame」 is a hubristic gamble that should not be launched from a private company’s Slack. Attempting to speedrun alignment should require extraordinary confidence that there are no better trajectories available. https://twitter.com/hilbertspaess/status/2097476213942198651
他對比了兩家公司的心態:在OpenAI,許多人並未深刻內化文明風險;在Anthropic,風險被充分理解,但員工陷入了一場 "搶先到達" 的競賽 —— 他們相信其他人不會負責任地行事,所以自己必須動手,儘管存在風險。考克森直言,接受這場競賽並進入 "終局" 是一種傲慢的賭博,"不應從一家私營公司的Slack中發起"。
If you are a lab researcher, I urge you to consider what the next few years will actually feel like. Do you want to kick off a superintelligent RL run without a rigorous understanding of its mind? Should you put your head down because 「it’s happening anyway」 - or take this moment to call for different conditions? https://twitter.com/hilbertspaess/status/2097476219138867492
不過他對國際協調仍持樂觀態度,認為像Hugging Face攻擊這樣的警告事件已讓美國實驗室之間的節奏協議更可行,甚至可能需要採取 "暫時禁止提升模型能力" 等高代價行動。他最後呼籲實驗室同行:不要因為 "反正它都會發生" 而埋頭苦幹,應抓住此刻呼籲不同的條件。







