From f64dcb03538ed97eb1da9b8c9286d143d493e3b9 Mon Sep 17 00:00:00 2001 From: admin9webs Date: Fri, 10 Jul 2026 16:11:14 +0800 Subject: [PATCH] Add mlops/huggingface-hub --- skills/mlops/huggingface-hub/SKILL.md | 103 ++++++++++++++++++++++++++ 1 file changed, 103 insertions(+) create mode 100644 skills/mlops/huggingface-hub/SKILL.md diff --git a/skills/mlops/huggingface-hub/SKILL.md b/skills/mlops/huggingface-hub/SKILL.md new file mode 100644 index 0000000..9f4568a --- /dev/null +++ b/skills/mlops/huggingface-hub/SKILL.md @@ -0,0 +1,103 @@ +--- +name: huggingface-hub +description: "HuggingFace hf CLI: search/download/upload models, datasets." +version: 1.0.0 +author: Hugging Face +license: MIT +tags: [huggingface, hf, models, datasets, hub, mlops] +platforms: [linux, macos, windows] +--- + +# Hugging Face CLI (`hf`) Reference Guide + +The `hf` command is the modern command-line interface for interacting with the Hugging Face Hub, providing tools to manage repositories, models, datasets, and Spaces. + +> **IMPORTANT:** The `hf` command replaces the now deprecated `huggingface-cli` command. + +## Quick Start +* **Installation:** `curl -LsSf https://hf.co/cli/install.sh | bash -s` +* **Help:** Use `hf --help` to view all available functions and real-world examples. +* **Authentication:** Recommended via `HF_TOKEN` environment variable or the `--token` flag. +* **Gated repos:** Before downloading from gated models (for example `black-forest-labs/FLUX.1-dev`), verify a token is configured and that the Hugging Face account has accepted the model agreement. A 401 `GatedRepoError` means the account/token lacks access; do not keep retrying unauthenticated downloads. + +--- + +## Core Commands + +### General Operations +* `hf download REPO_ID`: Download files from the Hub. +* `hf upload REPO_ID`: Upload files/folders (recommended for single-commit). +* `hf upload-large-folder REPO_ID LOCAL_PATH`: Recommended for resumable uploads of large directories. +* `hf sync`: Sync files between a local directory and a bucket. +* `hf env` / `hf version`: View environment and version details. + +### Authentication (`hf auth`) +* `login` / `logout`: Manage sessions using tokens from [huggingface.co/settings/tokens](https://huggingface.co/settings/tokens). +* `list` / `switch`: Manage and toggle between multiple stored access tokens. +* `whoami`: Identify the currently logged-in account. + +### Repository Management (`hf repos`) +* `create` / `delete`: Create or permanently remove repositories. +* `duplicate`: Clone a model, dataset, or Space to a new ID. +* `move`: Transfer a repository between namespaces. +* `branch` / `tag`: Manage Git-like references. +* `delete-files`: Remove specific files using patterns. + +--- + +## Specialized Hub Interactions + +### Datasets & Models +* **Datasets:** `hf datasets list`, `info`, and `parquet` (list parquet URLs). +* **SQL Queries:** `hf datasets sql SQL` — Execute raw SQL via DuckDB against dataset parquet URLs. +* **Models:** `hf models list` and `info`. +* **Papers:** `hf papers list` — View daily papers. + +### Discussions & Pull Requests (`hf discussions`) +* Manage the lifecycle of Hub contributions: `list`, `create`, `info`, `comment`, `close`, `reopen`, and `rename`. +* `diff`: View changes in a PR. +* `merge`: Finalize pull requests. + +### Infrastructure & Compute +* **Endpoints:** Deploy and manage Inference Endpoints (`deploy`, `pause`, `resume`, `scale-to-zero`, `catalog`). +* **Jobs:** Run compute tasks on HF infrastructure. Includes `hf jobs uv` for running Python scripts with inline dependencies and `stats` for resource monitoring. +* **Spaces:** Manage interactive apps. Includes `dev-mode` and `hot-reload` for Python files without full restarts. + +### Storage & Automation +* **Buckets:** Full S3-like bucket management (`create`, `cp`, `mv`, `rm`, `sync`). +* **Cache:** Manage local storage with `list`, `prune` (remove detached revisions), and `verify` (checksum checks). +* **Webhooks:** Automate workflows by managing Hub webhooks (`create`, `watch`, `enable`/`disable`). +* **Collections:** Organize Hub items into collections (`add-item`, `update`, `list`). + +### Gated model fallback research + +When a needed file is in a gated repo and no authorized HF token is available, do not stop after the first 401. Search for public mirrors and verify by filename, size, and checksum before using them: + +```python +from huggingface_hub import HfApi, model_info, hf_hub_download +api = HfApi() +for m in api.list_models(search='flux vae', limit=50): + print(m.id) + +for repo in ['candidate/repo']: + info = model_info(repo, files_metadata=True) + for f in info.siblings: + if f.rfilename == 'ae.safetensors': + print(repo, f.rfilename, f.size) + +path = hf_hub_download('candidate/repo', 'ae.safetensors', local_dir='.') +``` + +For files mirrored from gated upstreams, record the mirror repo, original upstream, byte size, and sha256 in the destination README. Treat the mirror as a pragmatic artifact source, not proof of license/permission. + +--- + +## Advanced Usage & Tips + +### Global Flags +* `--format json`: Produces machine-readable output for automation. +* `-q` / `--quiet`: Limits output to IDs only. + +### Extensions & Skills +* **Extensions:** Extend CLI functionality via GitHub repositories using `hf extensions install REPO_ID`. +* **Skills:** Manage AI assistant skills with `hf skills add`.