This is why I created: https://github.com/alisorcorp/warrant

It keeps documentation from going out of date by embedding re-runnable verification checks directly inside markdown files (that the agents use, not humans). I use it with a handoff workflow to force Claude/GPT to re-verify facts before handing off documentation to the next session.

Yea Qwen3.8 wasn't fun to use on my 64GB M4 Max either (better than these numbers though), so my new daily driver is Ornith-1.5-35B-A3B-MLX-4bit. I recommend giving that a whirl if you're on similar hardware, it's definitely better than Qwen3.6 35b-a3b which was my go-to before.

https://huggingface.co/ornith-ai/Ornith-1.5-35B-A3B

I tried Ornith-1.5-35B-A3B-MLX-4bit on a 32G M2-Pro mac mini with pi and OpenCode. I didn’t get very good results coding in Python, Racket, and TypeScript. I have seen several positive comments like yours so I was probably doing something wrong. I amgetting the new 64G mac mini in 4 weeks, and I made a note to try Ornith-1.5-35B-A3B-MLX-4bit again.

Can you tell more about your experience with Ornith? I've come across it, and the benchmarks on its landing page are unbeleavably good. But it isn't featured on more established benchmarks like ArtificialAnalysis and it's not on OpenRouter, so I wrote it off as scam.

Sora 2 was way better than Veo in terms of quality. It just didn't have much of a use case for businesses outside of one-shot zany UGC-looking clips.

https://youtu.be/QaiecWzeHFM?si=UV6eF-m514nCOFe0&t=156

There's over 3 million models on HuggingFace. So many quants. I'd imagine less than 0.5% would actually be seeded.

Sure there is. You've been able to search/sort/filter/group for a while now.

Would love to know - how?

I was already rolling around the idea of a 128GB M5 Max MBP. Now this!

A 4-bit MLX quant with 128k window should fit perfectly, in the 50-70 tok/s range.

Just out of curiosity, why run "local-local" when you could just set up a Mini or Studio at home and query it over http? [edit] whole conversation about this in another thread https://news.ycombinator.com/item?id=49433413

I’m personally considering retiring my MBP for a Studio + 15" Air whenever this MBP ages out.

I actually do this with my MBP - it's a LLM server when I'm working - and then when I'm not it's just a really great machine for video editing and other media work.

This is the way.

I’m doing that. Mac Mini M4 Pro with 48G RAM as a headless llama.cpp server.

I much prefer using " thin clients " as the interface to the big VMs running in my homelab

I have a 128GB M5 Max, and it sucks at this stage. 50-70 tok/s might be something...

50-70tk/s is what I get on my m5 max on a 5-6bit Qwen 3.8 27B?

I don't know what black magic you're up to but I see more like 30-35t/s on a 16" M5 Max using 3.8:27b Q4, regardless of whether it's mlx or gguf.

qwen3.5:122b-a10b is significantly faster at around 60-65.

No magic, just oMLX with MTP. You can look through the speed the community is getting here: https://omlx.ai/benchmarks/performance?model=qwen3.8&chip=&c...

With MTP? I get 25-30 TPS on a strix halo. 50+ on a M5 max should very doable. Dflash (2) will push your TG even further

I tried 8-bit, perhaps I should try 6-bit.

How do you folks code at 40-50 tps? With an extremely lightweight harness (pi) and just 8k system and tools context, and ~40tps on qwen 3.8 27B 4-bit on low thinking mode, it still takes me nearly 30-45 mins for a basic coding session...

Does it work? yeah... But I'd pick a subscription anyday...

There is a finite amount of time left for these companies to become next Facebook/Google, hoarding our interaction and privacy will be a point of contention pretty soon. At that moment, Qwen will be the knight in shining armor.

Do you find subscriptions to be meaningfully faster? I didn't really feel too much of a speed difference compared to Opus.

As someone who uses Opus daily for professional work and Qwen3.8-27B for all my private stuff, yes, Opus sub is faster for me, but I'm only rocking an R9700. If you're lucky enough to have sold a kidney on the blackmarket and purchased a 5090 and you're running ninfer, then actually... I think you'd be seeing fairly comparable performance!

When you hear stories like "Opus 5 thought for 20 minutes and then denied my request" it really puts wind in this sails of Local LMs

IDK, prefill speed is a bigger concern for most wokflows, like agent coding, and I heard that this is quite low on macs?

That was mainly before the M4 generation when they didn't have matmul instructions.

M5 prefill is much faster than M4.

I've seen benchmarks that show 4-5x faster of M5 Max vs. M4 Max.

For local models you're likely using M5 Max, prefill is low thousands of tokens per second, as opposed to, say high hundreds with M4 Max.

For larger dense models, some fraction of that, but similar multiple.

Yes, I have the M5 Max. But there was no matmul acceleration before the M4 which made things a lot slower.

I have actual obscure RGB corner floor lamps with BLE in my office. I had Claude find the BLE protocol and use my Office 365 calendar feed to change their color to red when I'm in a scheduled meeting (event), and back to warm white when the meeting is over, so co-workers know I'm busy without doing "are you busy" sign language through my window. The script is polled every 60 seconds. Tinkering with stuff like this is a lot of fun.

Very interesting, I missed your original post[0], will definitely look into it further. What's new since you've posted?

[0] https://news.ycombinator.com/item?id=48631726

We've added some things like CI, Windows support, and starting some work on path-based permissioning for monorepos. These are still in the early stages so not documented that great yet and need to be tested a little bit more. Should have more updates in the next couple weeks.

Yep, terrible speller -- bad at pronunciation too (by the way I just autocorrected that word) -- and I often wonder how people, who are much more intelligent than me on paper, can't figure out random tasks that just seem obvious.

Yes. "Weak detail recall but strong fluid reasoning, conceptual/gist memory, and adaptive problem-solving" is how GPT sums up what I feel. And "high fluid intelligence" is a label it gave that I won't be using anytime soon... But I like the sound of it.

You asked an LLM to reason about you, for you, and it said you’re an adroit reasoner? It must have context we lack!

You might benefit from (proper, not the multiple choice kind) aptitude testing.

What are the benefits? What would/could I do with the results?

Save money on future dates by causing the woman to leave before she even orders dinner after you pull out your Mensa© Membership card.

Most people come in for educational or career guidance, but I think it's best used as a way of understanding how your mind works, especially in relation to others'. The on-the-flying reasoning isn't pitted against good memory (there also isn't a single "memory", but we have different strengths for remembering words, stories, ideas, shapes, faces, numbers, melodies, and rhythms), but against a more methodical and well-prepared presenter. The on-the-fly reasoners are also quicker to see patterns and make connections among disparate data. It's great for trial law, but not so great (or distracting at best) for tax law, where you can't just wing it.