Edit ‘autogollark’ by osmarks

Edit ‘autogollark’ (by osmarks at 05 Aug 26 08:43 UTC)

Hyphae affected: autogollark

diff --git a/autogollark.myco b/autogollark.myco
index 22313c6..dd84643 100644
--- a/autogollark.myco
+++ b/autogollark.myco
@@ -27,3 +27,3 @@ Autogollark currently comprises the dataset, the search API server and the [[htt
* {RL (also include reasoning, of course). Probably hard though (sparse rewards). [[https://arxiv.org/abs/2403.09629]] (bad?). [[https://arxiv.org/abs/2503.22828]] would probably work. [[https://arxiv.org/abs/2505.15778]] [[https://arxiv.org/abs/2505.24864]] [[https://arxiv.org/abs/2509.06160]]
-* Unclear whether model could feasibly learn tool use "from scratch", so still need SFT pipeline.
+* Unclear whether model could feasibly learn tool use "from scratch", so still need SFT pipeline. Also, enormous compute costs. Though back in the Mistral-something days I don't think it was too expensive to have something "okay".
}
@@ -40,3 +40,3 @@ Autogollark currently comprises the dataset, the search API server and the [[htt
* https://github.com/d0rc/egg.c and https://eshyperscale.github.io/. Does this actually work (at scale)? Why? Would be really nice for using AMX units. But no pretrained models.
-* Maybe compute grants are available for training.
+* Maybe compute grants are available for training. Probably not enough especially now.
* Substantial bandwidth bottleneck on CPU (230GB/s nominal; 200GB/s benchmarked; 100GB/s per NUMA node, which llama.cpp handles awfully). Specdec/MTP would be useful. Can anything use AMX well though?