local-llm
jamesob/local-llm
A comprehensive hardware and software configuration guide for building and operating a high-performance local LLM and speech-to-text inference cluster.
1.8k 107No licenseUpdated 2026-07-10
61
GoodOverview
This repository provides a detailed blueprint and runner configurations for setting up state-of-the-art local AI infrastructure, featuring a custom 4x GPU rig with PCIe peer-to-peer switching, Docker-based model serving, and integrated local tooling for agentic workflows.
Capabilities
- ▸Hardware bill of materials for local AI rigs at various price points
- ▸PCIe switch configuration, BIOS bifurcation, and kernel parameter tuning for multi-GPU setups
- ▸Ready-to-run Docker Compose serving configurations for large models like GLM and Qwen
- ▸Local speech-to-text pipeline setup using cohere-transcribe
- ▸Custom scripts for measuring GPU peer-to-peer bandwidth and latency
Best for
Building high-performance local GPU clusters for running state-of-the-art large language models, Configuring low-latency peer-to-peer GPU interconnects using custom PCIe switches, Deploying local speech-to-text transcription harnesses, Setting up containerized model serving runners with Docker Compose
Works with
OpenCode
Evoa Score breakdown= Σ (score × weight)
61Task usefulness20%88 → +17.6
実タスクにどれだけ役立つか（機能の豊富さ・用途の明確さ）。 — AIによるcapabilities/use-cases解析
Code quality15%85 → +12.8
実装・指示の品質。 — AIによるSKILL.md/README解析
Maintenance15%85 → +12.8
リポジトリがどれだけ活発に保守されているか。 — GitHub 最終push日時の新しさ
Documentation12%33 → +4.0
ドキュメントの充実度・分かりやすさ。 — README/独自要約の情報量
Security15%50 → +7.5
危険・不審な挙動が無いか。 — AIによるセキュリティレビュー
Originality10%95 → +9.5
ありふれたラッパーではない独自性。 — AIによる独自性判定
Popularity8%82 → +6.6
コミュニティの採用度。 — GitHub Stars/Forks（対数スケール）
Compatibility5%25 → +1.3
対応AIエージェントの広さ。 — AIによる対応エージェント判定
Weighted total71.9 / 100
ライセンス不明/制限あり（Red）のSkillは総合スコアに0.85倍の補正を適用します。 ランキングはこのScoreのみで決まり、広告で変わりません。 算出方法の詳細 →
Security considerations
Instructions include disabling IOMMU and PCIe ACS at the kernel/runtime level to enable peer-to-peer GPU traffic, which alters standard system security isolation boundaries.
Categories
Summary and analysis are original content generated by AI Skills Rank. The skill's source text is not reproduced here — view it on the linked repository.