A human doing a clean room reimplementation is not supposed to have seen the original source code for something they are reimplementing. If that source code is in the training data for the LLM, that seems like a similar situation.

The difference is the original source code, and similar source code that is incompatibly licensed, was in the LLM training data, making it not a clean room.

Is it really feasible to run it 100% locally? I know there’s plenty of people with very powerful rigs indeed, but still.

There are small models you can run 100% locally on a mediocre computer or even a phone, and I mean you could be 100% offline and it will still work.

The better local models require a higher end gaming GPU or an ARM Mac with a decent amount of RAM, but nothing too extreme. You could get a computer to run them for about $2000 to $3000.

the software doesn’t come from nowhere and ultimately still relies on gas-turbine-powered datacenters and stolen IP and stolen personal data, no?

The small local models typically start with a full infustrial-size model and then they “distill” it to smaller models. The concerns over training data are still valid. I’m not sure how much the concerns over power usage are about training vs running the full industrial size models commercially. Running a small local model uses a tiny fraction of the power it takes to run an industrial size model, and the training is a one-time cost (except these companies never stop training because they want to make next year’s model better and faster).

I do think it’s worth pointing out that running a model locally keeps your conversations with it private. Remember, it can be done 100% offline. So your data won’t be stolen.

It’s something to hassle with on Windows too. Less so on macOS because it’s easier for them to optimize everything when they control the whole software and hardware stack.

Less durable is not the same as fragile. All smartphones today are way more durable than any smartphone from 10 years ago.

It’s a tradeoff of a little less durability for a bigger screen when you want one.

Yeah its definitely useful for topics you personally don’t have much experience in. It’s much less useful when trying to use it for topics you personally are an expert in.

I am bad at remembering names. Remembering a set of pronouns is at least as hard as remembering a name. Pronouns help when I can’t remember names. Having to remember custom pronouns defeats the purpose of pronouns.

All that being said, I’ve never met someone IRL who hasn’t picked one of the three most common pronouns sets (he, her, or they), so I see this as largely hypothetical. The internet does tend to be a space for people to test their more extreme beliefs.

This is basically the same as asking someone who knows a bit about everything to recite obscure information from memory. I doubt most humans would do better.

Sure, but a good web resource or even a good reference book would provide better help faster. Unfortunately it’s getting harder to harder to find those good web resources as search results get overtaken by AI slop.

iterate until the result is correct, then give me a one-page document explaining what I need to know, with cited evidence — far faster than I’d do it

I have gotten it to do this in certain situations, like the other day I wanted to simplify a math formula, so I gave it a loop with a Python script that checked its solution against the original reference version. This worked pretty well.

But I had to write code specifically for that situation. Even with a good skeleton to start with, it’s a non-negligible amount of work to get that set up. I feel like the scenario in which this is useful is kinda narrow: when I have a very good idea of exactly what I want, but some step along the way is a hassle. General software engineering, like making a whole app, is far too open ended, and most of the sub-problems I encounter in software engineering seem either too open ended or too small to benefit from this approach.

That also doesn’t account for the speed of models. My experience is a ~20GB locally hosted model takes like 1-5 minutes to produce a good length response, and the few times I have used online models they are often slower. A few minutes per iteration, accounting for debugging when it goes off track, is not exactly fast or hassle free.

I just feel like the trade off where using AI vs doing it all myself is pretty limited in when AI offers an advantage.

Eh idk I still have it make mistakes constantly.

Recent example: I asked AI to help me come up with a search/replace string for editing a file in vim. My prompt was something like “I am editing a text file in vim and I need to replace <description of pattern> with <description of pattern>. How can I do this with :s/ ?” And it spat out something that didn’t work. It was not valid syntax (for vim, I think it was valid standard regex, or close to it), and it didn’t quite follow the pattern I was trying to describe (although ofc that could be my fault to some extent). But it was useful in that it pointed me in the right direction to come up with a correct formula with some follow up googling and experimentation.

That has been very typical of my experiences with AI. Useful sometimes, but absolutely not “does everything to the point you don’t have to think about it” that seems to be a common opinion.

For context I’m using ~20GB local models, so not Claude, which people who pay for LLMs swear by.

I’ve never encountered such a thing in the US. Closest might be name tags with space for pronouns on them.

This is already well known to be very unhealthy

It’s good to get comfortable in the command line, including over ssh.

Especially considering ai and documentation along the way?

If I am going to ask AI or Google about something, I just do that to help me find the answer and then apply the answer myself. That way I learn, and can double check the AI isn’t hallucinating, at least on in obvious ways.

As for documentation, I keep a notes folder with detailed notes on manual configuration I’ve done, how and why, and the things I’ve learned along the way. I’ve found it’s both useful to help remember the things I’ve learned, and it is useful to go back to refer to.

One day, in the name of organizing, my mom put all of our game CDs into a a compact container and threw the original boxes away.

Yeah that was a sad day…

This is a given unless AI generated hell completely takes over entertainment

You can hack blu ray players, which can help get around DRM but also let you make your own rips of both movies and games.

https://github.com/RibShark/OmniDrive

I have bash scripts of the form rsync -av source dest for this. Make it -avz if copying over a network.

Interesting. I didn’t know how that worked, but I might give it a try. I have a 3090 with a lot of RAM.

You’ll want a GPU with hardware encoding and decoding for popular codecs. Most GPUs these days do that, even the budget ones, but you do need one.

Minecraft servers want fast single core CPU performance and a decent amount of RAM.

The rest of it could be RAM hungry too.

I feel this way about writing code

Those people see AI as a transformational technology that is going to be central to how everything is done 10 years from now for better or for worse, and trying to ignore that is just putting your head in the sand.