• 0 Posts
  • 16 Comments
Joined 3 years ago
cake
Cake day: March 22nd, 2024

help-circle


  • But Lemmy/Piefed could be structured to mitigate it. For instance, there could be a “community notes” header like Twitter (one of its few useful UX innovations, IMO). Or perhaps Piefed’s repost consolidation feature could be extended to include “reposts” of an original source, like tabloids running newspaper headlines.

    …But yeah. You are not wrong. Ultimately I think it’s a user sentiment issue. I’ve discussed this in other comment sections, and a good chunk of Lemmy outright say “Posts fitting with my views are more important than factual accuracy.” Others questioned why we should even care about primary sources, or sources with a reputable track record.

    Hence users like the headline above. And it makes me sad that, even when pointed out, even after everything we’ve escaped from and how we’ve seen the internet deteriorate before our eyes, folks here don’t see that as the priority issue.





  • I mean, this isn’t true from my perspective. All the Chinese LLM labs are still developing full steam ahead. Model/architectures are getting updated so quickly it’s a meme amongst inference framework devs, because they can’t even support one model before the next one is already out.

    …They’ve always been thrifty, though. The difference is they don’t need a trillion dollar bubble to fund their modest, more sustainable dev/inference costs.



  • Like others said, it looks better next quarter than a bunch of new projects. Short term gain vs long.

    But also, their expenses from all that hosting and usage are high. They’ve masked it creatively, but that only works so long.

    Maybe they’re trying to make their balance sheet look less catastrophic in the short term (again, at the expense of the long term).

    Whatever it is, that will be the underlying theme. Things must look good immediately, at the expense of sustainability.







  • It doesn’t literally mean “90% failure rate vs 100%”

    In the HLE benchmark, for example, we may be talking about 45% test score for MiMo Pro or GLM or something, vs 55% for Sonnet and 61% for maxed out Claude.

    But further complicating that, those particular open weight models do better than Claude in one particular hallucination benchmark, meaning Claude/ChatGPT might be worse at RAG/extraction type workloads. It’s like this across the board, with different models trading blows in different areas.

    By “90%,” he really means “it’s so close the better model depends on the particular application, workload, harness, sampling configuration” and such, and that none of these models are close to “100% accuracy” across the board.


    One other thing: Anthropic, OpenAI, and Grok have all been accused of “dumbing down” models under high demand to speed up inference. If your application depends on them, and they are the only provider, that could throw a wrench in the workflow.

    But open weights? Pick your provider. You get an API as consistent as you are willing to pay for.

    In other words, that “100%” isn’t really representing the potential inconsistency one gets being locked to a single provider.