• 4 Posts
  • 1.86K Comments
Joined 2 years ago
cake
Cake day: March 22nd, 2024

help-circle
  • brucethemoose@lemmy.worldtolinuxmemes@lemmy.worldBeware
    link
    fedilink
    arrow-up
    9
    ·
    edit-2
    16 hours ago

    Thats just poor distro support, kind of like CUDA in the past… ROCM should “just work” if it’s shipped right. But it’s not really a priority with maintainers.

    Now, if you’re trying to run CUDA stuff with ROCM, that’s a whole different story. The bast majority of GPU software has extremely poor ROCM support compared to CUDA, and some of this is definitely from AMD footgunning.


  • brucethemoose@lemmy.worldtolinuxmemes@lemmy.worldBeware
    link
    fedilink
    arrow-up
    7
    ·
    edit-2
    16 hours ago

    because it tends to include a previous version of the driver, which causes install/uninstall havok

    To be fair, this is a packaging/distro problem, as CUDA should always work (and be kept in sync with) the newest graphics driver.

    ROCM and OpenVINO (AMD and Intel) are even more of a pain, actually.


  • I don’t even know what they’re using the SSDs for.

    Most businesses are too stupid to train their own models from scratch, and won’t use “foreign” ones so they won’t finetune them either.

    On the inference side… SSDs aren’t used for much. Just storing Docker stuff/dependencies and model weights for the initial load, and that’s it. Maybe some data for bulk processing, but that’s no different than existing software. The one niche may be KV cache swapping for re-using prompt prefixes, but this is limited and being obsoleted by new attention mechanism.

    So WTF do they even need SSDs and HDDs for? Honestly it feels like FOMO purchasing.