| Post Reply | RetroFed Forum Forum Index » Qwen |
| Author | Message | ||
|---|---|---|---|
maria ~
lemmy.blahaj.zone
Smorty@lemmy.blahaj.zoneJoined: 2023-07-05 |
im so sleepy. agents driven by language models (LMs) call functions to do stuff. Functions like these:
this is not a simplification btw. so far, LMs were told to generate any of these formats to call a function:
wow these all have problems with something, hm? worst of all: we wanna save tokens wherever possible. so if an LM has to generate a full Introducing: my new formatwowie let's have a look at this format! [write
] now isnt that simple?
now, of course, this is a new format. so language models suck at generating it, right? WRONG even a local 2-bit quant of qwen 3.6 35B-A3B aligned to it super easily. and! even a dense Qwen3 4B model at Q4 quant worked with it flawlessly. I'm tired and need to sleep. now congratulate me! say "horray wow ur such a genius ohmygod we are gonna save so many tokens and thus möney". go, go head. im not gonna ask an LM to do it, that much is clear. or, even better: tell me what SUCKS about this, im always open for critical feedback. id rather be wrong than believe im right all the time. |
||
| 5 Original | |||
ThotDragon
lemmy.blahaj.zone
thoughtfuldragon@lemmy.blahaj.zoneJoined: 2024-04-24 |
Not a fan of LLMs but have you considered INI format? This is pretty similar. |
||
| 3 | |||
maria ~
Moderator
Smorty@lemmy.blahaj.zoneJoined: 2023-07-05 |
Not a fan of LLMs but have you considered INI format? This is pretty similar. https://en.wikipedia.org/wiki/INI_file
thanks for sharing, but the goal here was not to make yet another key-value format, but to have natural feeling multiline-strings with minimal escaping. thats why i settled for the code blocks: they tokenize well, are universally understood as "this is some text", are easy to write and rarely ever have to deal with escape sequences. in this case, the thank u for engaging with the post btw. i really appreciate it <3 |
||
| 2 | |||
|
Another Catgirl
lemmy.blahaj.zone
anothercatgirl@lemmy.blahaj.zone Joined: 2023-06-15 |
I want to use this in my project! how do I apply it in practice? |
||
| 1 | |||
|
Another Catgirl
lemmy.blahaj.zone
anothercatgirl@lemmy.blahaj.zone Joined: 2023-06-15 |
I want to use this in my project! how do I apply it in practice?
for context, I downloaded qwen3:4b with ollama and it runs fast on my gpu. I wanna make tools for my LLM to be able to play Dwarf Fortress for me. |
||
| 1 | |||
maria ~
Moderator
Smorty@lemmy.blahaj.zoneJoined: 2023-07-05 |
for context, I downloaded qwen3:4b with ollama and it runs fast on my gpu. I wanna make tools for my LLM to be able to play Dwarf Fortress for me.
ohgosh playing a whole game is a whole different level of complexity >o< it can work, but only under very short time horinzons and it will likely get stuck immediately trying to do the same thing. (im assuming you mean the text-based version) see claude opus trying to play runescape here and skip to 6:55 where it starts. you will notice: its interesting to watch, but gets stuck quickly. and that model was SOTA at the time. the model would also have difficulties telling what us where. sounds weird, but due to tokenization, it cant "infer" which characters are above which others. it will be guessing wrong a lot. you would be babysitting the model. right now, LMs and VLMs are being trained for computer use (where they click on a screen and use a computer "like a human would"), but exclusively for work tasks and not games. try this first: run test it saying "yo what files and dies are in this directory?" and it will run if you really - REALLY want it to work with the game, the best option would be:
theres no reason to use my format, its just a tolen-efficiency thing. with your fast GPU-throughput, this shouldnt be a problem. ohgosh long response- |
||
| 2 | |||
|
Another Catgirl
lemmy.blahaj.zone
anothercatgirl@lemmy.blahaj.zone Joined: 2023-06-15 |
ohgosh playing a whole game is a whole different level of complexity >o< it can work, but only under very short time horinzons and it will likely get stuck immediately trying to do the same thing. (im assuming you mean the text-based version) see claude opus trying to play runescape here and skip to 6:55 where it starts. you will notice: its interesting to watch, but gets stuck quickly. a...
I specifically wanna use your format though! |
||
| 2 | |||
maria ~
Moderator
Smorty@lemmy.blahaj.zoneJoined: 2023-07-05 |
I specifically wanna use your format though!
then i have "bad" news. i made thiup, it's not a standard. meaning: you would have to write the entire agentic scaffolding yourself. which honestly, is a fun project. im sure ur super duper smart and already know this, but: all that an agent is, is just:
so umm... if thats what u feel like doing, good luck! 💖 |
||
| 2 | |||
|
Another Catgirl
lemmy.blahaj.zone
anothercatgirl@lemmy.blahaj.zone Joined: 2023-06-15 |
then i have "bad" news. i made thiup, it's not a standard. meaning: you would have to write the entire agentic scaffolding yourself. which honestly, is a fun project. ive done it, its nice being able to see exactly what's goin on. im sure ur super duper smart and already know this, but: all that an agent is, is just: parsing the tokens as they come in notice when the LM writes a tool call parse...
that is a need, I'm gonna stop playing modded minecraft and work on this instead. it sounds like a lot of fun! |
||
| 2 | |||
gandalf_die_12te
lemmy.blahaj.zone
gandalf_der_12te@lemmy.blahaj.zoneJoined: 2024-06-10 |
thanks for sharing, but the goal here was not to make yet another key-value format, but to have natural feeling multiline-strings with minimal escaping. thats why i settled for the code blocks: they tokenize well, are universally understood as "this is some text", are easy to write and rarely ever have to deal with escape sequences. in this case, the [ and ] also serve as tool-call delimiters, ...
if you are looking to embed code into free-flowing text output, you can go with a format such as this:
you write the commands with indentations (>) and basically ignore every other line. some languages do it that way. i think PHP uses opening tags to denote where code starts/ends, the rest is just ignored by the PHP interpreter. |
||
| 1 | |||
maria ~
Moderator
Smorty@lemmy.blahaj.zoneJoined: 2023-07-05 |
if you are looking to embed code into free-flowing text output, you can go with a format such as this: this is some free-flowing text output from the LLM to write to a file. > write("filename", "content") > someotherfunction() this is more free-flowing text you write the commands with indentations (>) and basically ignore every other line. some languages do it that way. i think PHP use...
I... think u didnt read my comment. for me, the point was to have natural feeling multiline strings, but yesyes, if those are not a concern, ur format very much rules ~ |
||
| 1 | |||
| Post Reply |
Back to top |