All Roads Lead Back to GLM

The title of the previous entry was set to be this one, but ended up changing thanks to OMP and DSV4F-0731 being such a solid combination. Then, I heard that GLM 5.3 was coming and had the opportunity to try it out in beta, and I was a fan. I figured I would be writing the next entry all about GLM 5.3. Then a model codenamed "Ox Alpha" was on OpenRouter and rumors were swelling that it was a GLM-style model. Knowing 5.3 was coming, I didn't really think Z.ai was putting another model out, and mostly thought it was probably just some other company that fine-tuned a GLM model and that was why it was finger-printing as a GLM model.

But then Ox Alpha was released and it turned out to be GLM 5.3 Flash! People had been asking Z.ai to release a smaller "flash" or "air" model for a while, and this was an unexpected answer to that request. Just as people had been asking for a Flash/air variant, they had also been asking for vision and Z.ai answered that as well with adding a vision tower to GLM 5.3 Flash. I started playing around with this model locally and was instantly hooked.

It's got the same "feel" as GLM 5.2, performs very similarly. but also has vision ability. The full GLM 5.3 also is an improvement over our benchmark scores for 5.2, but I only have benchmarked it from the Z api, and have not tested it locally just yet. So first, let's get the benchmarks out of the way!

Seeing these results, you might propose that we should just use DSV4F 0731, it's better, and faster. Of course just 1 benchmark isn't enough to justify that, and DSV4F lacks vision (though there is now an experimental vision variant), the big thing was that it really felt to me that GLM 5.3 Flash WAS faster to use in practice. And it is. Your time to solution, and time to failure, is much faster:

So, even though DSV4F (might) be slightly better on this bench, and is more than 2x faster at generating tokens, it uses up so many more tokens to do it, that, in the end, GLM 5.3 Flash is faster in practice. I think once you have a human in the loop working with the agent on problems, this is way more valuable. It just feels so darn good to dev with GLM 5.3 Flash

Unfortunately, however, it was during my time testing this model that one of my RTX Pro 6Ks just totally died. This GPU was about 2 months old, and it had been having some sort of random firmware issue, causing the card to jump to ERR and need the computer to reboot to get it back. Eventually, this reboot procedure stopped helping and I was never able to revive the card again. I am currently going through the RMA process, so we'll see how that goes.

Besides being another GLM-feeling LLM, GLM 5.3 Flash has that vision component, and I have long wondered about using multimodal LLMs with robotics.

An obvious question you might wonder is: well why not use VLA or VLM? The hard part about robotics isn't object detection, that's long solved. This also isn't a solution for gait/locomotion...yet, but I actually don't think this is far away either and I've done some experimentation with LLMs in this space in the past and it does show promise.

I've trained VLAs. They're super finicky and you're always running into sim2real issues, especially around the camera. You also have to build the whole training pipeline in a simulator, and, if everything does work, you still just have a robot that does this 1 single thing after weeks of work.

If you use teleop, this overcomes the "2real" problem, but now you need to painstakingly collect teleop data, and it's only good at that specific task and that particular robot. There is a growing set of egocentric training data for "general purpose" VLAs and world action models (for humanoid form factors), but I'm really starting to wonder: Why? I think we might just sidestep this whole area of research entirely.

Here, we have GLM 5.3 Flash commanding the high level SDK for an XGO mini2sw, very much like how a VLA might, but this is just a multimodal LLM, and it's getting the job done. I've done tasks like this with actual VLAs, this is working better, faster, and more intelligently with no further training. And thats what you get. It's not like it's just some vision/action model. It's more like an intelligence that happens to also have vision. I think this could be the more likely future of robotics, and just like we have coding harnesses, we could potentially just be a robotics harness away from a lot of general tasks being solved with robotics and models like GLM 5.3 Flash

You may also be wondering about closed frontier models like GPT 5.6 Sol. I did attempt to use GPT 5.6 Sol for this task. The average solution time for GLM 5.3 Flash is 20 seconds from the go command to picking up the object. The average solution time for GPT 5.6 Sol is 4.5 minutes, and the outlier times were 20+ minutes. The main issues with GPT 5.6 sol were: slow api as well as constant model adjustments. It just never was quite happy with positioning and would forego a perfectly fine position that would have likely worked to pick up for an attempt to get it even better. It was a very frustrating experience to watch haha.

I also have tried to use DSV4F-0731 and Qwen 3.8 27B as an LLM + vision LLM to solve this task, and it works very good. There's also the experimental vision DSV4F. I don't have that model downloaded and haven't tested that, but I suspect it'd probably work for this task too.

Okay, that's all for now! I may eventually do the full testing of GLM 5.3