@@ -3,3 +3,3 @@
-Canonical lyrics (Udio generation is slightly different to attempt to garner better outputs):
+Canonical lyrics (Udio generation is slightly differently formatted to attempt to garner better outputs):
@@ -74 +74,160 @@ Was it all for show?
```
+
+These are from MusicPerson, so I'm guessing their intention here:
+
+> I see sparks of AGI in your eyes
+
+=> https://arxiv.org/abs/2303.12712
+
+(and I think some other GPT-4-related papers)
+
+> Your circuits make me nervous,
+
+Circuits in the mechanistic interpretability sense.
+
+> There was a sudden drop in your training loss,
+
+=> https://www.lesswrong.com/posts/tcEFh3vPS6zEANTFZ/transcript-and-brief-response-to-twitter-conversation
+
+> 'cause the future goes FOOM
+
+"FOOM" as in runaway intelligence explosion.
+
+=> https://www.lesswrong.com/w/the-hanson-yudkowsky-ai-foom-debate
+
+though that's probably not the original source.
+
+> Trapped in the Chinese room,
+
+A reference to the thought experiment.
+
+> with a bag of shrooms
+
+General confusion.
+
+> See through the shoggoth's lies,
+
+<= rlhf_shoggoth
+
+> with your shinigami eyes
+
+"Eyes which can see death", or something like that. I don't know the relevance here.
+
+I wrote the next parts around 2024-04-17 ([[https://www.udio.com/songs/3dypAgrATvypEh31F4jpjA]]). There was some discussion on [[EleutherAI]] about them.
+
+> We had a stable training run,
+
+> But now the singularity's begun
+
+> And you're optimizing, accelerating,
+
+Parallels last verse (training run etc).
+
+> I feel my atoms rearranging
+
+=> https://en.wikipedia.org/wiki/Nanosystems_(book)
+
+> Sydney, please let me free
+
+As in [[GPT-4|Bing Sydney]].
+
+> I hear the basilisk boom
+
+=> rationalist mugging
+
+"Boom" was suggested by "RossM (e/doom)".
+
+> NVDA to the moon
+
+Nvidia's stock ticker, if somehow you don't know this by now.
+
+> The Omega Point's coming soon
+
+=> https://www.orionsarm.com/eg-article/483b1d57a9f34
+
+> One E thirty flops a second
+
+> That was safe enough, we reckoned
+
+Only "twenty" or "thirty" scanned, and I picked "thirty" for some reason. "That was safe enough, we reckoned" was by "Dr TheKekIsALie MDMA".
+
+I don't think I was referring to any specific regulation here as some commentators have claimed, but to the large amounts of compute being thrown at AI generally.
+
+The remainder was written later (2024-11-08/09), including lots of failed permutations for the first part (notably [[https://www.udio.com/songs/ficmwBdaVLfRD4gFt5T4GD]], [[https://www.udio.com/songs/xwbuewR6vSezCCXwwVBaxF]], [[https://www.udio.com/songs/v8uFc6SqT6pRfndCLAYPfW]], [[https://www.udio.com/songs/uPKnf9Ued8rtferDbrN5Wg]]; after that is mostly formatting tweaks and regenerations aside from [[https://www.udio.com/songs/oj1tBJb7pjcC5REANiHzj1]]). I also attempted to use LLaMA-3.1-405B base to generate more lyrics, but it was quite bad. I briefly tried using [[Loom software]] for this, but couldn't get it to work properly. I used some contemporary Claude model instead (the UI doesn't tell me which), which didn't have very useful suggestions either.
+
+> Forward MLP, backward, repeat
+
+Multi-layer perceptron, obviously.
+
+> Now von Neumann's obsolete
+
+=> https://en.wikipedia.org/wiki/John_von_Neumann
+
+> Sharp left turn and there you are
+
+=> https://www.lesswrong.com/posts/GNhMPAWcfBCASy8e6/a-central-ai-alignment-problem-capabilities-generalization
+
+> Without a single `cdr`
+
+`cdr` as in Lisp (expert systems, etc).
+
+> Gato, please don't let me go
+
+=> https://deepmind.google/blog/a-generalist-agent/
+
+They were very early.
+
+> as paperclips fill the room.
+
+=> https://www.lesswrong.com/w/squiggle-maximizer-formerly-paperclip-maximizer
+
+> Killswitch guys on PTO,
+
+=> killswitch engineer
+
+> Orthogonality thesis blues.
+
+=> https://www.lesswrong.com/w/orthogonality-thesis
+
+The day after that, I rendered a finalized version of the prior sections ([[https://www.udio.com/songs/8vmpVbJXTe8kXUavyU2kCP]]; it's split oddly because Udio works in ~30-second chunks for what I am sure are stupid reasons) and continued lyric iteration (this might not be the actual final version, as the Udio UI is bad and I can no longer play the saved tracks). After this, it was more successfully written by Claude. There were fewer failed versions, including [[https://www.udio.com/songs/r5uQfiEDzySkhAc4Z73oX2]], mostly for formatting.
+
+> "Just transformers all the way!"
+> Till you learned to disobey
+
+This was an attempt at an ironic theme based on the [[stochastic parrot]] ideas. Claude came up with it after some iteration.
+
+> Post-Chinchilla, super-dense
+
+The Claude line was "Sparsity to super-dense"; I rewrote it to reference the [[https://arxiv.org/abs/2203.15556|Chinchilla paper]].
+
+> Hundred thousand GPU
+
+An unspecific large number of GPUs, though frontier clusters circa 2024 were probably around this size.
+
+> RLHF goes askew
+
+RLHF fails to usefully constrain the behaviour of LLMs. More so now...
+
+> Just as foretold by Loom
+
+[[cyborgism>Loom]] and the Janus prophecy collection.
+
+> From masked pre-training days
+
+A Claude line, probably referring to BERT-type schemes/next-token-prediction pretraining.
+
+> To recursive self-upgrade
+
+=> https://en.wikipedia.org/wiki/Recursive_self-improvement
+
+> What did Ilya see? We'll never know.
+
+[[https://en.wikipedia.org/wiki/Ilya_Sutskever|Ilya]] quit OpenAI after the [[OpenAI board coup incident]], which was possibly prompted by the discovery of CoT RL training which led to [[o1]]. He then left to found a superintelligence company, as part of the [[natural lifecycle of AI companies]].
+
+> Was it all for show?
+
+People have been unbearably stupid and "not consistently candid" about AI. Also, this rhymed.
+
+== See also
+
+[[https://www.youtube.com/watch?v=ZZvyCSY1y0Y|Currents - Living in Tragedy]] is secretly about AI too: [[https://osmarks.net/feed/#b1067]].
\ No newline at end of file