• 1 Post
  • 116 Comments
Joined 1 year ago
cake
Cake day: June 4th, 2023

help-circle





  • LLMs cannot:

    • Tell fact from fiction
    • Accurately recall data from its training set
    • Count

    LLMs can

    • Translate
    • Get the general vibe of a text (sentiment analysis)
    • Generate plausible text

    Semantics aside, they’re very different skills that require different setups to accomplish. Just because counting is an easier task than analysing text for humans, doesn’t mean it’s the same it’s the same for a LLM. You can’t use that as evidence for its inability to do the “harder” tasks.









  • My online persona is definitely different from IRL, and it differs IRL depending on who I’m interacting with. But these are all the real me. My ability to communicate via text is generally better than spoken, so that is reflected in how I write, what I write about, as well as how little I speak in person.

    Secondly, in person communication has clearer continuity. If I have multiple conversations with a given person, I learn a bit about them and their communication style, allowing me to adjust how I speak to be better understood by that person. Online, I rarely remember who I’m talking to, so I just write in whatever way feels most natural to me.

    The real time nature of in person communication also limits what you can bring up and when. Anything you say requires the other party to respond immediately, and if you recognize that they’re not in the mood to think particularly hard, then you don’t bring up difficult topics. Online conversations don’t come with this kind of information, but it does give you the flexibility to answer whenever you want, or not at all, so many things that I would not deem acceptable in an IRL setting can be acceptable online.

    So in summary, different situations do call for different behaviours. But that’s not problematic any more than behaving differently at a party and at a funeral is problematic.





  • It sounds like you don’t like how LLMs are currently used, not their power consumption.

    I agree that they’re a dead end. But I also don’t think they need much improvement over what we currently have. We just need to stop jamming them where they don’t belong and leave them be where they shine.


  • Yeah, they operate very opaquely, so we can’t know the true cost, but based on what I can know with certainty given models I can run on my own machines, the numbers seem reasonable. In any case, that’s not really relevant to this discussion. Treat it as a hypothetical, then work out the math later to figure out where we want to be and what threshold we should be setting.


  • Indeed. Though what we should be thinking about is not just the cost in absolute terms, but in relation to the benefit. GPT-4 is one of the more expensive models to run right now, and you can accomplish very good results with their smaller GPT-4o mini at 0.5% of the energy cost[1]. That’s the cost of running 0.07 LED bulbs over an hour, or running 1 LED bulb over 0.07 hours (i.e. 5min). If that saves you 5min of time writing an email while the room is lit with a single LED bulb and your computer is drawing energy, that might just be worth it, right?

    [1] Estimated by using https://huggingface.co/spaces/genai-impact/ecologits-calculator and the pricing difference between GPT-4o, 4o mini, and 3.5 (https://openai.com/api/pricing/). The assumption I’m making is that the total hardware and energy cost scales linearly with the API pricing.