• 0 Posts
  • 12 Comments
Joined 10 months ago
cake
Cake day: February 19th, 2025

help-circle
  • Image working in an old building, there is no coffee machine, there is no warm water and if you want to do something new, chances of you getting in trouble because you forgot to mess with CMake or free memory, are high. But the building works since it has been tested for 30 years.

    Rust is a new building that you can move over to, there is a coffee machine that is a bit complicated at first but once you understand it it is that bad, there is warm water and you don’t have to mess with CMake or allocate/free memory for everything. But the building is new, there will be issues here and there and sometimes the promised warm water wont work since someone fucked it up, but in general it is just sooooo much more comfy to work in.

    Rust is not about making Programming languages fast or memory safe. If you truly want to do that, I recommend doing crack and writing in assembly. It is about making programming easier without sacrificing speed.










  • The smallest Modells that I run on my PC take about 6-8 GB of VRAM and would be very slow if I ran them purely with my CPU. So it is unlikely that you Phone has enough RAM and enough Cores to run a decent LLM smootly.

    If you still want to use selfhosted AI with you phone, selfhost the modell on your PC:

    • Install Ollama and OpenWebUI in a docker container (guides can be found on the internet)
    • Make sure they use your GPU (Some AMD Cards require an HSA override Flag to work
    • Make sure the docker container is secure (Blocking the Port for comunication outside of your network should work fine as long as you only use the AI Modell at home)
    • Get youself an openwight modell (I recomend llama 3.1 for 8 GB of VRAM and Phi4 if you got more or have enough RAM)
    • Type the IP-Adress and Port into the browser on your phone.

    You now can use selfhosted AI with your phone and an internet connection.