6

Source document

[2025-04-07 06:12] traves_theberge: **trains on benchmarks** (files: raw.png) [2025-04-07 06:27] traves_theberge: (files: Gn3fvTIXMAAmHmK.png) [2025-04-07 06:28] traves_theberge: i suspect most of these models are trained on the benchmarks. [2025-04-07 06:29] traves_theberge: but does that make them useful is another question " look at me i can do what i was trained to do" [2025-04-07 06:33] lisamegawatts: https://github.com/mlabonne/llm-autoeval [2025-04-07 06:42] lisamegawatts: https://huggingface.co/datasets/gorilla-llm/Berkeley-Function-Calling-Leaderboard?ref=blog.promptlayer.com [2025-04-07 06:42] lisamegawatts: ^possible smp benchmark [2025-04-07 07:16] lisamegawatts: https://x.com/Yuchenj_UW/status/1909061004207816960 [2025-04-07 07:20] lisamegawatts: So it seems most of the programming benchmarks are aimed specifically at python [2025-04-07 07:23] lisamegawatts: https://youtu.be/zmTbZS3-eg0?si=g2MzovnfXYoWQNkP [2025-04-07 07:25] lisamegawatts: I think maybe instead of training it on code directly, train it on coding textbooks so it learns how to do things, <@806444151422976035> if you have any recommended javascript [2025-04-07 07:30] lisamegawatts: these guys used t5, outdated but very small model [2025-04-07 07:40] ajaxdavis: nah, suppose just douglas crawford [2025-04-07 07:46] lisamegawatts: https://youtu.be/Jr_nGkCG3og?si=o716oFZe7bauEwy2 they use jax instead of python and got 4000% improvement and say they did not even optimize jax [2025-04-07 07:48] jonathan.poczatek: jax == good [2025-04-07 07:54] jonathan.poczatek: https://developers.google.com/machine-learning/crash-course/llm/tuning this is a great summary [2025-04-07 08:01] jonathan.poczatek: https://github.com/neural-maze/ava-whatsapp-agent-course [2025-04-07 08:03] jonathan.poczatek: https://github.com/argilla-io/argilla [2025-04-13 03:45] lisamegawatts: https://x.com/tunguz/status/1911142310160855541?s=46&t=OS71yaTltL7EclsAy81ZcQ [2025-04-20 12:48] lisamegawatts: https://www.nature.com/articles/s42256-025-01000-2 [2025-04-21 01:04] lisamegawatts: https://x.com/saboo_shubham_/status/1913243850765672926?s=46&t=OS71yaTltL7EclsAy81ZcQ [2025-04-25 19:01] traves_theberge: https://github.com/tensorzero/tensorzero [2025-04-26 23:03] lisamegawatts: https://www.reddit.com/r/LocalLLaMA/comments/1k7o89n/we_compress_any_bf16_model_to_70_size_during/?rdt=37369 [2025-04-28 06:17] lisamegawatts: https://github.com/mlflow/mlflow [2025-04-28 07:12] lisamegawatts: A line in roo code caught my eye: If there's a real world job posting for something you want a custom mode to do, try asking Code mode to Create a custom mode based on the job posting at @[url] <@806444151422976035> cool feature for json resume maybe? replace my job with ai and it writes the prompt.... [2025-04-28 07:37] ajaxdavis: could you explain it again lol