• 0 Posts
  • 37 Comments
Joined 1 year ago
cake
Cake day: June 12th, 2023

help-circle
  • Similar to previous reply about MATE with font size changes, I do that with plasma. I hadn’t seen plasma big screen you linked, I’ll definitely try that one out. I’ve wondered about https://en.m.wikipedia.org/wiki/Plasma_Mobile? Like these sort of niche projects don’t always get a lot of attention, if the bigscreen project doesn’t work out, I’d bet the plasma mobile project is fairly active and given the way it scales for displays might work really well on a tv

    Speaking of scaling since you mentioned it. I have noticed scaling in general feels a lot better in Wayland. If you’d only tried it in X11 before, might want to see if Wayland works better for you.





  • Ran Asahi for several months, tried it out again recently. It’s good/fine, I just don’t love fedora.

    There’s some funkiness with the more complicated install, the AI acceleration doesn’t work, no thunderbolt / docking station.

    MacBooks are great hardware but I don’t think they’re the best option for Linux right now. If you’re never going to boot into macOS then I’d look for x13, new Qualcomm, isn’t there a framework arm64 option now or was that a RISC module?

    I’m also assuming you’re not looking to do any gaming? Because gaming on ARM is not really a thing right now and doesn’t feel like it will be for a long while.







  • Pumpkin Escobar@lemmy.worldto196@lemmy.blahaj.zoneThe Rule
    link
    fedilink
    English
    arrow-up
    9
    ·
    4 months ago

    There’s quantization which basically compresses the model to use a smaller data type for each weight. Reduces memory requirements by half or even more.

    There’s also airllm which loads a part of the model into RAM, runs those calculations, unloads that part, loads the next part, etc… It’s a nice option but the performance of all that loading/unloading is never going to be great, especially on a huge model like llama 405b

    Then there are some neat projects to distribute models across multiple computers like exo and petals. They’re more targeted at a p2p-style random collection of computers. I’ve run petals in a small cluster and it works reasonably well.



  • Taking ollama for instance, either the whole model runs in vram and compute is done on the gpu, or it runs in system ram and compute is done on the cpu. Running models on CPU is horribly slow. You won’t want to do it for large models

    LM studio and others allow you to run part of the model on GPU and part on CPU, splitting memory requirements but still pretty slow.

    Even the smaller 7B parameter models run pretty slow in CPU and the huge models are orders of magnitude slower

    So technically more system ram will let you run some larger models but you will quickly figure out you just don’t want to do it.







  • Pumpkin Escobar@lemmy.worldtoUnixporn@lemmy.ml*Permanently Deleted*
    link
    fedilink
    English
    arrow-up
    5
    ·
    edit-2
    7 months ago

    Agreed, the meta+arrow shortcuts to move windows around are great. That defaults to half/quarter windows. You can also define a custom layout (meta+t to configure). The meta+arrow shortcuts still work on half/quarters of the screen, but you can shift+drag a window to drop it into one of the custom layout tiles/areas… gives a lot of flexibility.