Unfortunately Nvidia is, by fair, the best choice for local LLM coder hosting, and there are basically two tiers:

• Buy a used 3090, limit the clocks to like 1400 Mhz, and then host Qwen 2.5 coder 32B.

• Buy a used 3060, host Arcee Medius 14B.

Both these will expose an OpenAI endpoint.

Run tabbyAPI instead of ollama, as it’s far faster and more vram efficient.

You can use AMD, but the setup is more involved. The kernel has to be compatible with the rocm package, and you need a 7000 card and some extra hoops for TabbyAPI compatibility.

Aside from that, an Arc B570 is not a terrible option for 14B coder models.

The i7-6700 has an Intel iGPU that will handle heavy transcoding just fine using Quicksync.

It will even do really fast object detection with OpenVINO, with minimal CPU usage. At least in Frigate both of those things work extremely well.

Ryzen 5600g or any higher G model its an APU or 6700xt for a real GPU 6700 is best bang for buck but it fluctuates all the time. All amd systems are much less finnicky than nvidia. Especially on Linux.

I'm glad you posted this because I need similar advice. I want a GPU for Jellyfin transcoding and running Ollama (for a local conversation agent for Home Assistant), splitting access to the single GPU between two VMs in Proxmox.

I would also prefer it to be AMD as a first choice or Intel as a second, because I'm still not a fan of Nvidia for their hostile attitude towards Linux and for proprietary CUDA.

(The sad thing is that I probably could have accomplished the transcoding part with just integrated graphics, but my AMD CPU isn't an APU.)

You can try a Coral TPU with CodeProjectAI. I used it for a bit but I have the USB version and it has heating/disconnect issues at times.

I don’t like using CPU for AI so plan to offload the AI to an unRAID server with a nVidia 2070 super to combine Plex decoding and AI tasks to that box.