itguyeric/infra
1
0
Fork 0
Code Issues 28 Pull requests Projects 1 Releases Packages Wiki Activity Actions

Self-hosted AI: local LLMs with Ollama + Open WebUI #33

New issue
Open
opened 2026-08-19 02:26:06 +00:00 by itguyeric · 1 comment
itguyeric commented 2026-08-19 02:26:06 +00:00
Owner
Copy link

Run local models on the fleet instead of reaching for a cloud API. Ollama to manage and serve models, Open WebUI as the chat front end. Uses the RTX for inference, so plan GPU allocation since Plex and a possible Immich ML container also want it.

Doubles as the local engine for other services: Paperless-AI extraction (#28) and anything we wire up later. New bootc image with GPU passthrough like the Plex box, behind SWAG, LAN-only or over Tailscale.

Run local models on the fleet instead of reaching for a cloud API. Ollama to manage and serve models, Open WebUI as the chat front end. Uses the RTX for inference, so plan GPU allocation since Plex and a possible Immich ML container also want it. Doubles as the local engine for other services: Paperless-AI extraction (#28) and anything we wire up later. New bootc image with GPU passthrough like the Plex box, behind SWAG, LAN-only or over Tailscale.
itguyeric added this to the Homelab project 2026-08-19 02:26:06 +00:00
itguyeric commented 2026-08-19 02:50:43 +00:00
Author
Owner
Copy link

Blocked by: Need AI-capable server

Blocked by: Need AI-capable server
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
No results found.
Labels
Clear labels   
area/ci
Forgejo Actions, runners, Ansible, bootc images

  
area/media
Plex, arr stack, books, YouTube

  
area/network
OPNsense, DNS, Tailscale, SWAG

  
area/observability
Monitoring, alerting, logs

  
area/platform
New services and platform-level builds

  
area/security
Auth, secrets, access control

  
area/storage
NFS, LVM, disks, swap, backups

  
area/web
Hugo site, blog content

  
type/project
Multi-session effort, likely spawns tasks

  
type/task
Discrete piece of work with a clear done

No labels
area/ci
area/media
area/network
area/observability
area/platform
area/security
area/storage
area/web
type/project
type/task
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Homelab
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
itguyeric/infra#33
No description provided.