From 97c35e1f2f1da1119fa438361d874cdcd4992daf Mon Sep 17 00:00:00 2001
From: admin9webs <admin@9webs.online>
Date: Fri, 10 Jul 2026 16:11:17 +0800
Subject: [PATCH] Add mlops/llm-inference

---
 skills/mlops/llm-inference/SKILL.md | 43 +++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)
 create mode 100644 skills/mlops/llm-inference/SKILL.md

diff --git a/skills/mlops/llm-inference/SKILL.md b/skills/mlops/llm-inference/SKILL.md
new file mode 100644
index 0000000..bea7e24
--- /dev/null
+++ b/skills/mlops/llm-inference/SKILL.md
@@ -0,0 +1,43 @@
+---
+name: llm-inference
+description: "LLM inference: local GGUF inference (llama.cpp) and high-throughput serving (vLLM)."
+version: 2.0.0
+author: Hermes Agent
+license: MIT
+platforms: [linux, macos]
+metadata:
+  hermes:
+    tags: [llm, inference, llama-cpp, vllm, gguf, serving, deployment]
+    related_skills: [huggingface-hub]
+---
+
+# LLM Inference
+
+Two inference approaches: local GGUF inference for development and vLLM for production serving.
+
+## Section 1: llama.cpp — Local GGUF Inference
+
+Run LLMs locally using llama.cpp with GGUF quantized models. Supports CPU, GPU, and mixed inference.
+
+**When to use:** Local development, testing models, resource-constrained environments, offline inference.
+**Key features:** GGUF quantization, CPU/GPU/mixed, OpenAI-compatible server mode, HF Hub model discovery.
+**See:** `references/llama-cpp.md`.
+
+## Section 2: vLLM — High-Throughput Serving
+
+Deploy LLMs for production with vLLM. OpenAI API compatibility, PagedAttention, continuous batching, and quantization support.
+
+**When to use:** Production LLM deployment, high-throughput serving, multi-model serving, needs OpenAI API compatibility.
+**Key features:** PagedAttention, continuous batching, tensor parallelism, GPTQ/AWQ quantization, OpenAI API server.
+**See:** `references/serving-llms-vllm.md`.
+
+## Choosing Between Them
+
+| Need | Tool |
+|------|------|
+| Local dev/testing | llama.cpp |
+| Resource-constrained | llama.cpp (with quantization) |
+| Production API serving | vLLM |
+| High throughput needed | vLLM |
+| OpenAI API compatibility | Both (via server modes) |
+| Multi-GPU serving | vLLM (tensor parallelism) |