Basically a deer with a human face. Despite probably being some sort of magical nature spirit, his interests are primarily in technology and politics and science fiction.

Spent many years on Reddit before joining the Threadiverse as well.

  • 0 Posts
  • 1K Comments
Joined 2 years ago
cake
Cake day: March 3rd, 2024

help-circle

  • If you’d like to analogize with special-purpose programs versus general purpose ones, you could consider an LLM to be more like an “operating system” rather than a single application running on that system. The LLM isn’t specialized in a particular task, it supports running any task you want to throw at it.

    In this analogy, the specialized applications that are running on the LLM would be the things called “agents” and “harnesses.” They’re the parts that hold the code that is tailored specifically to the particular task that they’re for. So if I wanted to set up a system using an LLM to, for example, read court transcripts and automatically search legal databases for statutes and case law relevant to whatever’s being discussed in them, I wouldn’t simply copy and paste the transcript into the LLM’s context and expect something useful to come back. I’d have to embed the LLM inside a system that prompts it correctly, has tools that can search legal databases, mechanisms for storing intermediate results, scripts to check the formatting of inputs and outputs, and so forth. None of that accessory stuff needs to be an ML model, it can just be conventional programming. Trying to train a specialized ML model from the ground up to do all that stuff without the associated harness helping it would be hugely wasteful.

    I suppose another analogy could be the hardware itself. For every task that we use general-purpose von Neumann architecture CPUs for we could create a specialized chip on a purpose-specific circuit board. But instead for most tasks we find it’s much cheaper and more convenient to create general-purpose computers and then program them for these specific tasks. Nowadays you’ll often find that simple home appliances with just a few buttons and a few functions will have a full blown microcontroller inside them with firmware. Probably lots of unused inputs/outputs and ram and whatnot. That’s because it’s far cheaper and easier to build a factory that stamps out millions of general-purpose programmable chips than it would be to have hundreds of factory lines that each do a run of ten thousand custom-designed chips. It’s genuinely less wasteful doing it that way.












  • I didn’t say anything about the “prevent instances from being overloaded” part being good or bad. I didn’t even give an opinion on ActivityPub, just pointed out the practical limitations and incompatible design goals.

    Personally, I’ve got no problem with websites implementing rate caps and whatnot to ensure that their traffic remains within the limits they can handle, or throttling specific IPs. I am very concerned with how Cloudflare in particular has become the single centralized “gatekeeper” for vast swaths of the Internet, though. If they decide that some particular client isn’t allowed to see stuff then poof, a big chunk of the Internet is cut off. That’s worrisome IMO.




  • That’s a very narrow view of data scraping, there’s lots of ways to get data.

    The Fediverse is built on ActivityPub, which is an open protocol that’s designed to broadcast data with no limitations or restrictions. If you don’t want your data to end up in the hands of anyone who wants it - including those nefarious AI trainers - then that’s an inherently incompatible goal with ActivityPub.

    If you’re just worried about specific instances being overloaded with requests, then sure, all the usual rate limiting DDOS-prevention Cloudflare tricks will work. But the data itself isn’t “protected.” Someone who wants it could simply run an instance of their own specifically to collect it.