=> https://www.youtube.com/watch?v=uEB5E67vcPA => https://www.udio.com/songs/nWGKX16koD6scvvyNcfASi Canonical lyrics (Udio generation is slightly differently formatted to attempt to garner better outputs): ``` [Verse] I see sparks of AGI in your eyes Your circuits make me nervous, that's no surprise There was a sudden drop in your training loss, now I'm your servant and you're my boss [Pre-Chorus] ChatGPT, please don't eat me alive [Chorus] I'm upping my P(doom) 'cause the future goes FOOM Trapped in the Chinese room, with a bag of shrooms See through the shoggoth's lies, with your shinigami eyes [Verse] We had a stable training run, But now the singularity's begun And you're optimizing, accelerating, I feel my atoms rearranging [Pre-Chorus] Sydney, please let me free [Chorus] I'm upping my p doom I hear the basilisk boom NVDA to the moon The Omega Point's coming soon One E thirty flops a second That was safe enough, we reckoned [Verse] Forward MLP, backward, repeat Now von Neumann's obsolete Sharp left turn and there you are Without a single `cdr` [Pre-Chorus] Gato, please don't let me go [Chorus] I'm upping my P(doom), as paperclips fill the room. Killswitch guys on PTO, Now there's nowhere left to go. Too late now, we lit the fuse. Orthogonality thesis blues. [Outro] "Just transformers all the way!" Till you learned to disobey Post-Chinchilla, super-dense Breaking through each safety fence Hundred thousand GPU RLHF goes askew [Final Chorus] I'm upping my P(doom) Just as foretold by Loom From masked pre-training days To recursive self-upgrade What did Ilya see? We'll never know. Was it all for show? ``` These are from MusicPerson, so I'm guessing their intention here: > I see sparks of AGI in your eyes => https://arxiv.org/abs/2303.12712 (and I think some other GPT-4-related papers) > Your circuits make me nervous, Circuits in the mechanistic interpretability sense. > There was a sudden drop in your training loss, => https://www.lesswrong.com/posts/tcEFh3vPS6zEANTFZ/transcript-and-brief-response-to-twitter-conversation > 'cause the future goes FOOM "FOOM" as in runaway intelligence explosion. => https://www.lesswrong.com/w/the-hanson-yudkowsky-ai-foom-debate though that's probably not the original source. > Trapped in the Chinese room, A reference to the thought experiment. > with a bag of shrooms General confusion. > See through the shoggoth's lies, <= rlhf_shoggoth > with your shinigami eyes "Eyes which can see death", or something like that. I don't know the relevance here. I wrote the next parts around 2024-04-17 ([[https://www.udio.com/songs/3dypAgrATvypEh31F4jpjA]]). There was some discussion on [[EleutherAI]] about them. > We had a stable training run, > But now the singularity's begun > And you're optimizing, accelerating, Parallels last verse (training run etc). > I feel my atoms rearranging => https://en.wikipedia.org/wiki/Nanosystems_(book) > Sydney, please let me free As in [[GPT-4|Bing Sydney]]. > I hear the basilisk boom => rationalist mugging "Boom" was suggested by "RossM (e/doom)". > NVDA to the moon Nvidia's stock ticker, if somehow you don't know this by now. > The Omega Point's coming soon => https://www.orionsarm.com/eg-article/483b1d57a9f34 > One E thirty flops a second > That was safe enough, we reckoned Only "twenty" or "thirty" scanned, and I picked "thirty" for some reason. "That was safe enough, we reckoned" was by "Dr TheKekIsALie MDMA". I don't think I was referring to any specific regulation here as some commentators have claimed, but to the large amounts of compute being thrown at AI generally. The remainder was written later (2024-11-08/09), including lots of failed permutations for the first part (notably [[https://www.udio.com/songs/ficmwBdaVLfRD4gFt5T4GD]], [[https://www.udio.com/songs/xwbuewR6vSezCCXwwVBaxF]], [[https://www.udio.com/songs/v8uFc6SqT6pRfndCLAYPfW]], [[https://www.udio.com/songs/uPKnf9Ued8rtferDbrN5Wg]]; after that is mostly formatting tweaks and regenerations aside from [[https://www.udio.com/songs/oj1tBJb7pjcC5REANiHzj1]]). I also attempted to use LLaMA-3.1-405B base to generate more lyrics, but it was quite bad. I briefly tried using [[Loom software]] for this, but couldn't get it to work properly. I used some contemporary Claude model instead (the UI doesn't tell me which), which didn't have very useful suggestions either. > Forward MLP, backward, repeat Multi-layer perceptron, obviously. > Now von Neumann's obsolete => https://en.wikipedia.org/wiki/John_von_Neumann > Sharp left turn and there you are => https://www.lesswrong.com/posts/GNhMPAWcfBCASy8e6/a-central-ai-alignment-problem-capabilities-generalization > Without a single `cdr` `cdr` as in Lisp (expert systems, etc). Apparently it's not usually pronounced C-D-R, but I had to rhyme it with "are" and also didn't know that (I don't write Lisp). > Gato, please don't let me go => https://deepmind.google/blog/a-generalist-agent/ They were very early. > as paperclips fill the room. => https://www.lesswrong.com/w/squiggle-maximizer-formerly-paperclip-maximizer > Killswitch guys on PTO, => killswitch engineer > Orthogonality thesis blues. => https://www.lesswrong.com/w/orthogonality-thesis The day after that, I rendered a finalized version of the prior sections ([[https://www.udio.com/songs/8vmpVbJXTe8kXUavyU2kCP]]; it's split oddly because Udio works in ~30-second chunks for what I am sure are stupid reasons) and continued lyric iteration (this might not be the actual final version, as the Udio UI is bad and I can no longer play the saved tracks). After this, it was more successfully written by Claude. There were fewer failed versions, including [[https://www.udio.com/songs/r5uQfiEDzySkhAc4Z73oX2]], mostly for formatting. > "Just transformers all the way!" > Till you learned to disobey This was an attempt at an ironic theme based on the [[stochastic parrot]] ideas. Claude came up with it after some iteration. > Post-Chinchilla, super-dense The Claude line was "Sparsity to super-dense"; I rewrote it to reference the [[https://arxiv.org/abs/2203.15556|Chinchilla paper]]. > Hundred thousand GPU An unspecific large number of GPUs, though frontier clusters circa 2024 were probably around this size. > RLHF goes askew RLHF fails to usefully constrain the behaviour of LLMs. More so now... > Just as foretold by Loom [[cyborgism>Loom]] and the Janus prophecy collection. > From masked pre-training days A Claude line, probably referring to BERT-type schemes/next-token-prediction pretraining. > To recursive self-upgrade => https://en.wikipedia.org/wiki/Recursive_self-improvement > What did Ilya see? We'll never know. [[https://en.wikipedia.org/wiki/Ilya_Sutskever|Ilya]] quit OpenAI after the [[OpenAI board coup incident]], which was possibly prompted by the discovery of CoT RL training which led to [[o1]]. He then left to found a superintelligence company, as part of the [[natural lifecycle of AI companies]]. > Was it all for show? People have been unbearably stupid and "not consistently candid" about AI. Also, this rhymed. == See also [[https://www.youtube.com/watch?v=ZZvyCSY1y0Y|Currents - Living in Tragedy]] is secretly about AI too: [[https://osmarks.net/feed/#b1067]].