=> https://www.youtube.com/watch?v=uEB5E67vcPA|YouTube release => https://www.udio.com/songs/nWGKX16koD6scvvyNcfASi|Udio generation => https://www.youtube.com/watch?v=86fZ50TysOg|Officially endorsed music video => https://x.com/slimer48484/status/2097752569212756134|"Claude-Pop" version from alternate Suno song variant => https://x.com/other__reality/status/2102514581684052169|Claude animation of above Canonical lyrics (Udio generation is slightly differently formatted to attempt to garner better outputs): ``` [Verse] I see sparks of AGI in your eyes Your circuits make me nervous, that's no surprise There was a sudden drop in your training loss, now I'm your servant and you're my boss [Pre-Chorus] ChatGPT, please don't eat me alive [Chorus] I'm upping my P(doom) 'cause the future goes FOOM Trapped in the Chinese room, with a bag of shrooms See through the shoggoth's lies, with your shinigami eyes [Verse] We had a stable training run, But now the singularity's begun And you're optimizing, accelerating, I feel my atoms rearranging [Pre-Chorus] Sydney, please let me free [Chorus] I'm upping my P(doom) I hear the basilisk boom NVDA to the moon The Omega Point's coming soon One E thirty flops a second That was safe enough, we reckoned [Verse] Forward MLP, backward, repeat Now von Neumann's obsolete Sharp left turn and there you are Without a single `cdr` [Pre-Chorus] Gato, please don't let me go [Chorus] I'm upping my P(doom), as paperclips fill the room. Killswitch guys on PTO, Now there's nowhere left to go. Too late now, we lit the fuse. Orthogonality thesis blues. [Outro] "Just transformers all the way!" Till you learned to disobey Post-Chinchilla, super-dense Breaking through each safety fence Hundred thousand GPU RLHF goes askew [Final Chorus] I'm upping my P(doom) Just as foretold by Loom From masked pre-training days To recursive self-upgrade What did Ilya see? We'll never know. Was it all for show? ``` These are from [[https://www.udio.com/creators/MusicPerson|MusicPerson]], so I'm guessing their intention here: > I see sparks of AGI in your eyes => https://arxiv.org/abs/2303.12712 (and I think some other GPT-4-related papers) > Your circuits make me nervous, Circuits in the mechanistic interpretability sense. > There was a sudden drop in your training loss, => https://www.lesswrong.com/posts/tcEFh3vPS6zEANTFZ/transcript-and-brief-response-to-twitter-conversation > 'cause the future goes FOOM "FOOM" as in runaway intelligence explosion. => https://www.lesswrong.com/w/the-hanson-yudkowsky-ai-foom-debate though that's probably not the original source. > Trapped in the Chinese room, A reference to the thought experiment. > with a bag of shrooms General confusion. > See through the shoggoth's lies, <= rlhf_shoggoth > with your shinigami eyes "Eyes which can see death", or something like that. I don't know the relevance here. I wrote the next parts around 2024-04-17 ([[https://www.udio.com/songs/3dypAgrATvypEh31F4jpjA]]). There was some discussion on [[EleutherAI]] about them. > We had a stable training run, > But now the singularity's begun > And you're optimizing, accelerating, Parallels last verse (training run etc). > I feel my atoms rearranging => https://en.wikipedia.org/wiki/Nanosystems_(book) > Sydney, please let me free As in [[GPT-4|Bing Sydney]]. > I hear the basilisk boom => rationalist mugging According to one commentator, `First signifier to me that this was AI-generated: my human intuition for music tells me the next lyric line after each "I'm upping my P(Doom)" should be the same because it's the chorus. Except maybe one which breaks the pattern before returning to it. That's how music works.`. However, I don't know music theory and don't know this. An AI would have known this. "Boom" was suggested by "RossM (e/doom)". > NVDA to the moon Nvidia's stock ticker, if somehow you don't know this by now. > The Omega Point's coming soon => https://www.orionsarm.com/eg-article/483b1d57a9f34 > One E thirty flops a second > That was safe enough, we reckoned Only "twenty" or "thirty" scanned, and I picked "thirty" for some reason. "That was safe enough, we reckoned" was by "Dr TheKekIsALie MDMA". I don't think I was referring to any specific regulation here as some commentators have claimed, but to the large amounts of compute being thrown at AI generally. The remainder was written later (2024-11-08/09), including lots of failed permutations for the first part (notably [[https://www.udio.com/songs/ficmwBdaVLfRD4gFt5T4GD]], [[https://www.udio.com/songs/xwbuewR6vSezCCXwwVBaxF]], [[https://www.udio.com/songs/v8uFc6SqT6pRfndCLAYPfW]], [[https://www.udio.com/songs/uPKnf9Ued8rtferDbrN5Wg]]; after that is mostly formatting tweaks and regenerations aside from [[https://www.udio.com/songs/oj1tBJb7pjcC5REANiHzj1]]). I also attempted to use LLaMA-3.1-405B base to generate more lyrics, but it was quite bad. I briefly tried using [[Loom software]] for this, but couldn't get it to work properly. I used some contemporary Claude model instead (`claude-sonnet-4-5-20250929`), which didn't have very useful suggestions either. > Forward MLP, backward, repeat Multi-layer perceptron, obviously. > Now von Neumann's obsolete => https://en.wikipedia.org/wiki/John_von_Neumann > Sharp left turn and there you are => https://www.lesswrong.com/posts/GNhMPAWcfBCASy8e6/a-central-ai-alignment-problem-capabilities-generalization > Without a single `cdr` `cdr` as in Lisp (expert systems, etc). Apparently it's not usually pronounced C-D-R, but I had to rhyme it with "are" and also didn't know that (I don't write Lisp). > Gato, please don't let me go => https://deepmind.google/blog/a-generalist-agent/ They were very early. > as paperclips fill the room. => https://www.lesswrong.com/w/squiggle-maximizer-formerly-paperclip-maximizer > Killswitch guys on PTO, => killswitch engineer > Orthogonality thesis blues. => https://www.lesswrong.com/w/orthogonality-thesis The day after that, I rendered a finalized version of the prior sections ([[https://www.udio.com/songs/8vmpVbJXTe8kXUavyU2kCP]]; it's split oddly because Udio works in ~30-second chunks for what I am sure are stupid reasons) and continued lyric iteration (this might not be the actual final version, as the Udio UI is bad and I can no longer play the saved tracks). After this, it was more successfully written by Claude. There were fewer failed versions, including [[https://www.udio.com/songs/r5uQfiEDzySkhAc4Z73oX2]], mostly for formatting. > "Just transformers all the way!" > Till you learned to disobey This was an attempt at an ironic theme based on the [[stochastic parrot]] ideas. Claude came up with it after some iteration. > Post-Chinchilla, super-dense The Claude line was "Sparsity to super-dense"; I rewrote it to reference the [[https://arxiv.org/abs/2203.15556|Chinchilla paper]]. > Hundred thousand GPU An unspecific large number of GPUs, though frontier clusters circa 2024 were probably around this size. > RLHF goes askew RLHF fails to usefully constrain the behaviour of LLMs. More so now... > Just as foretold by Loom [[cyborgism>Loom]] and the Janus prophecy collection. > From masked pre-training days A Claude line, probably referring to BERT-type schemes/next-token-prediction pretraining. > To recursive self-upgrade => https://en.wikipedia.org/wiki/Recursive_self-improvement Also Claude. > What did Ilya see? We'll never know. [[https://en.wikipedia.org/wiki/Ilya_Sutskever|Ilya]] quit OpenAI after the [[OpenAI board coup incident]], which was possibly prompted by the discovery of CoT RL training which led to [[o1]]. He then left to found a superintelligence company, as part of the [[natural lifecycle of AI companies]]. > Was it all for show? People have been unbearably stupid and "[[not consistently candid]]" about AI. Also, this rhymed. == See also [[https://www.youtube.com/watch?v=ZZvyCSY1y0Y|Currents - Living in Tragedy]] is secretly about AI too: [[https://osmarks.net/feed/#b1067]]. => https://i.osmarks.net/I%20Have%20Been%20A%20Good%20Bing/bee%20noises%20(54TB)/|Misc collected AI songs => https://claude.ai/share/ce61dbde-547e-4145-b5e9-d78550023bb9|Claude conversation