Skip to main content

LocalAI

@dsh-ai-providers/localai is the dedicated LocalAI integration maintained by Firefox2100 as part of this project. It is separate from the general OpenAI-compatible provider, so LocalAI-specific controls and capabilities are not presented as OpenAI features. It is not an official DSH or LocalAI package.

The package depends on the provider-neutral @dsh-ai-providers/openai-compatible transport library, not on the OpenAI provider plugin. The two plugins can be installed independently while sharing their common protocol implementation.

Capabilities​

The package registers:

  • streamed chat and vision language models;
  • embeddings;
  • reranking;
  • decisions;
  • text-to-speech;
  • file and live speech-to-text;
  • image generation.

Enable only the services implemented by the models loaded in your LocalAI instance.

Connection​

Open Settings > Providers > LocalAI connections. The default connection is http://localhost:8080/v1. Change it when LocalAI runs on another host or behind a proxy. An API key is optional; when authentication is enabled, set a credential name on the connection and store its value through DSH's credential service.

A reusable LocalAI connection in DSH settings

Connections are shared by every capability. Each service page selects a connection and an exact model name independently, and keeps its LocalAI-native request fields beside that service.

Language models​

Language models use LocalAI's OpenAI-compatible POST /chat/completions stream. Routes and models support the common sampling fields plus top_k, min_p, repetition penalty, reasoning handling, custom headers, and arbitrary top-level request fields in sampling.body.

Use sampling.body for LocalAI-native or backend-specific options such as constrained grammar and backend sampler controls. Route values apply first and model values override them.

Model administration​

LocalAI routes support the optional model administration API exposed as ctx.aiModels: status uses GET /backend/monitor, loading uses POST /backend/load, and unloading uses POST /backend/shutdown, all relative to the configured /v1 base URL. Status reports LocalAI's backend state and, when supplied, its memory use in bytes.

These endpoints require administrative authorization when LocalAI authentication is enabled, so the connection credential must have that access. Loading and unloading change shared server state; use them only when the caller owns the model lifecycle and has accounted for concurrent requests.

Reranking​

Reranking calls POST /rerank relative to the configured /v1 base URL. Set rerankModel to the model name served by LocalAI. rerankBody is merged into each request for backend-specific parameters; the query, documents, model, top_n, and return_documents fields remain controlled by the service call.

Decisions​

Decisions call POST /systemone relative to the configured /v1 base URL, the System One API through which LocalAI serves decision models that return typed answers with probabilities in one pass. Set decisionModel to the name of a model declared with known_usecases: [decisions] in LocalAI. decisionBody is merged into each request for backend-specific fields; the model, state and questions stay controlled by the service call.

Decision models need a LocalAI release and llama-cpp backend build that support them. Use the connection test in the Decisions tab to confirm that your installation can serve the model. See Decisions for consumer usage.

Speech, embeddings, and images​

These services use the corresponding OpenAI-compatible LocalAI endpoints. Model IDs, voices, formats, and image options depend on the LocalAI backend configuration and should be entered exactly as served. embeddingBody, ttsBody, and imageBody add backend-specific JSON fields while the service keeps ownership of canonical fields such as the input, model, voice, and prompt. Live transcription is available when the deployment provides the compatible realtime WebSocket endpoint.

LocalAI image generation always calls POST /images/generations; it does not use /images/edits. Reference handling is configured as one of three modes: ignore references, send the first image in files for image-to-image generation, or send every image in ref_images for a backend that supports references. highRes performs a second image-to-image pass at the configured scale. The provider also supports per-request steps, negativePrompt, and seed fields, subject to the loaded backend.

When to use this package​

Use @dsh-ai-providers/localai for LocalAI deployments. Keep @dsh-ai-providers/openai for OpenAI or other compatible servers. Both packages may be installed together because they register different provider IDs.