Father, Hacker (Information Security Professional), Open Source Software Developer, Inventor, and 3D printing enthusiast

  • 0 Posts
  • 8 Comments
Joined 3 years ago
Cake day: June 23rd, 2023
  • Riskable@programming.devtoFediverse@lemmy.world•LLM vigilance on the Fediverse is lackingEnglish
    1·
    2 days ago

    I’m not misrepresenting anything. If you take any model and just change the temperature a little bit (it’s a setting that controls the influence of the RNG) you can have wildly different output.

    Different AI providers using the exact same model will have outputs so different using the same exact prompts you’d think they were in different universes.

    They absolutely do not “have the same hallmarks.”

    This stuff is benchmarked to absurd degrees every time a new model comes out. Just look it up already! You’re so obviously arguing from an ignorant position it’s silly.

  • Riskable@programming.devtoFediverse@lemmy.world•LLM vigilance on the Fediverse is lackingEnglish
    1·
    3 days ago
    1. They do not all write the same. There’s benchmarks for this and if you just do a basic search or actually test them yourself you’ll see that their output differs wildly. Not only between vendors, but been versions of the same model. For example, Claude stopped using em dashes for most output for a while but now they’ve come back.
    2. There’s vastly more than “a handful of models”! Are you living under a rock‽ Go to HuggingFace today and then again tomorrow. There’s going to be a dozen new ones! Probably more!
    3. Changing the back end configuration even a tiny bit can vastly change the output of a model.
  • Riskable@programming.devtoFediverse@lemmy.world•LLM vigilance on the Fediverse is lackingEnglish
    4·
    3 days ago

    FYI: Pangram sucks. It’s about as accurate as throwing darts these days.

    A long, long time ago (i.e. 2024) there was only one LLM that was widely used by the populace: ChatGPT. It only had a few versions and they didn’t differ much in how they worded output.

    Now it’s 2026 and we have hundreds of thousands of AI models. That includes over 1000 commercial vendor models!

    Another thing that’s really important that you need to understand is that even if two providers run the same model, their configurations will be different and thus, their output won’t be the same.

    To summarize: There’s no fucking way Pangram can work properly anymore. It’s useless garbage.