@@ -15,3 +15,3 @@ Autogollark currently comprises the dataset, the search API server and the [[htt
* Due to general personality stability. Need finetune or similar.
-* One proposal: use internal finetune to steer big model somehow. Possibly: use its likelihood (prefill-only) to evaluate goodness of big model output wrt. gollark personality, and if it is too bad then use finetune directly.
+* One proposal: use internal finetune to steer big model somehow. Possibly: use its likelihood (prefill-only) to evaluate goodness of big model output wrt. gollark personality, and if it is too bad then use finetune directly. But issues if we go for a custom tokenizer.
* Is GCG code salvageable? NanoGCG, maybe.
@@ -36,5 +36,6 @@ Autogollark currently comprises the dataset, the search API server and the [[htt
* https://arxiv.org/abs/2507.01335
-* https://github.com/d0rc/egg.c and https://eshyperscale.github.io/. Does this actually work? Why?
+* https://github.com/d0rc/egg.c and https://eshyperscale.github.io/. Does this actually work (at scale)? Why? Would be really nice for using AMX units.
+* Maybe compute grants are available for training.
}
-* MCTS over conversations with non-gollark simulacra? Should find //something// to use spare parallelism on local inference. Best-of-n? https://arxiv.org/abs/2505.10475
+* Search over conversations with non-gollark simulacra? Should find //something// to use spare parallelism on local inference. Best-of-n? https://arxiv.org/abs/2505.10475
* {Longer context, mux several channels.