Skip to Content
Running ModelsModel sources

Run models

Skippy can serve a model from its catalog, an exact Hugging Face repository reference, or an existing local file. The source identifies the model bytes; the runtime and serving settings determine how those bytes execute.

Use a catalog  name for the shortest supported path, a Hugging Face reference when you need a particular repository and quantization, and a local path for models you already manage.

Choose a model source

skippy serve --model Qwen3-0.6B-Q4_K_M skippy serve --model unsloth/Qwen3-8B-GGUF:Q4_K_M skippy serve --model /models/qwen3-8b-q4_k_m.gguf

Remote GGUF files resolve to an immutable revision and are cached before loading. Serving the same exact reference reuses the Hugging Face cache.

Discover and manage models

Skippy uses the Mesh LLM catalog for curated names and recommendations by default:

skippy models recommended

recommended shows curated starting points from the selected catalog.

You can provide your own catalog by creating a Hugging Face dataset repository with the same format as meshllm/catalog , then setting the repository as an override:

export SKIPPY_MODEL_CATALOG=your-org/your-catalog skippy models recommended

Catalog caches are kept separately by repository, so changing the override does not mix entries from different catalogs.

Use search when you know a family or keyword but not the exact repository:

skippy models search qwen --limit 10

Search queries Hugging Face for GGUF repositories by default. Once you have a candidate, resolve the exact artifact, quantization, revision, and expected files before downloading it:

skippy models show unsloth/Qwen3-8B-GGUF:Q4_K_M skippy models download unsloth/Qwen3-8B-GGUF:Q4_K_M

show is read-only. download verifies and stores the selected files in the shared Hugging Face cache so later serving can reuse them without another download.

Inspect and maintain the local cache separately:

skippy models installed skippy models updates --check skippy models cleanup --unused-since 30d

installed reports the model references Skippy can resolve locally. updates --check compares cached references with upstream revisions without downloading changes. cleanup finds managed model files that have not been used within the selected period.

Cleanup and deletion preview their targets by default. Review the selected files before adding --yes; Skippy never deletes a remote repository.

Automatic resource planning

For a local GGUF, Skippy sizes one unified KV pool from model metadata, weight footprint, and available device memory, then chooses context and concurrency defaults. Override those decisions only when you have a measured reason:

skippy serve \ --model unsloth/Qwen3-8B-GGUF:Q4_K_M \ --ctx-size 32768 \ --generation-concurrency 2

Inspect the resolved launch before model load with:

skippy serve --settings serve.toml --print-effective-config

How model resolution works

Skippy resolves a remote reference to an immutable repository revision, selects the requested artifacts, verifies the downloaded files, and records the resulting model identity. Local files are read directly but still pass model admission and memory planning before the server becomes ready.

For local GGUF models, the planner uses model metadata, weight footprint, and available device memory to choose a context size and generation concurrency. An explicit --ctx-size or --generation-concurrency overrides the matching automatic choice. If the requested model or context does not fit, Skippy fails the load instead of silently substituting another model.

Model command reference

TaskCommand
Show recommended modelsskippy models recommended
Search Hugging Faceskippy models search QUERY
Resolve an exact artifactskippy models show REFERENCE
Download before servingskippy models download REFERENCE
List cached modelsskippy models installed
Check cached refs for updatesskippy models updates --check
Preview unused-file cleanupskippy models cleanup --unused-since 30d

Skippy honors the standard Hugging Face cache and authentication settings. Model deletion and cleanup are previews until --yes is supplied.