Skip to main content

Creating or extending a provider

A provider plugin implements one or more provider-neutral contracts against a vendor API. Use @dsh-ai-providers/openai and @dsh-ai-providers/jina as implementation references, and @dsh-ai-providers/openai-compatible and @dsh-ai-providers/jina-cohere as examples of protocol libraries that several vendors share without depending on each other, but expose only the capabilities the target API actually supports.

1. Define validated configuration​

Use a Schemastery Config schema for connections, model names, and capability options. Keep secrets in DSH's credential service; store only credential references in plugin configuration.

Normalize and validate base URLs at the configuration boundary. Reject unsupported protocols and malformed values before making a request.

2. Implement capability factories​

Create an AiProvider with a stable ID and one factory for each supported capability. A factory receives enough context to resolve current configuration and returns the core interface implementation.

Unsupported capabilities should be absent, not represented by methods that always fail. Optional operations such as voice discovery may return an empty result when the vendor has no discovery endpoint.

Factories must read live configuration when called. Do not capture a connection, credential value, or model at registration time: doing so makes settings appear to update while requests continue using stale state. Keep the implementation's provider, model, and capability identity accurate because consumers and diagnostics can inspect them.

Routes, retries and discovery​

A route (LlmRouteSpec) may carry baseUrl, the endpoint its configuration fixes, and retry, a DSH retry policy; the runtime reports both to DSH. A model may carry systemPromptUpdate: 'in-history' when its chat template reads a changed system message anywhere in the conversation. Override LlmService.interrogate(draft, signal) to list the models of an endpoint that is not configured yet (draft.baseUrl, and a key for this one question in draft.apiKey); the default throws an unsupported error. discover(route) stays the way to list the models of a configured route.

Vendor controls in sampling​

The aim of a vendor package is to expose what the vendor's API can do while the configuration keeps one shape. Follow this convention for the controls of a model request:

  • A route, a model and a policy all carry the same sampling object, merged field by field in that order, then the request's own values. LlmSampling in core names the fields every provider has (temperature, topP, topK, minP, the penalties, seed, stop, maxTokens, toolChoice) and body, the escape hatch for any other top-level request field. Use those names and meanings for anything that matches, and map them to the vendor's wire names yourself.
  • A vendor's own control is a further optional field of the same flat object, in camelCase, named after the vendor's parameter (numCtx for num_ctx). Declare it in an interface that extends LlmSampling, and in the same sampling schema of the route and the model. Absent always means the server's default; never fill in a default the vendor does not document.
  • Give a control a field only when it is a stable setting of the vendor's API. What is experimental, or specific to one model, belongs in body, and in a reasoning level's body when it selects a level of thinking.
  • Fields outside LlmSampling travel through the runtime's merge untouched, so a policy of a consumer who knows the vendor can set them by returning the vendor's extended type. A consumer that does not know the vendor never needs to.
  • A control the vendor cannot honour is not invented: document what is ignored, and keep fail-loud behaviour for what would change the meaning of the request.
  • Document each field with its wire name in the provider's page.

The Ollama provider is the reference: OllamaSampling extends LlmSampling, and its page lists every field with its request name.

Streaming responses​

@dsh-ai-providers/core provides the readers of streamed bodies, so a vendor does not write its own: sseData(body, activity, signal) yields the data of each server-sent event, and ndjsonLines(body, activity, signal) yields each line of a newline-delimited stream. Both call activity for every part received (for a silence watchdog), end on the signal even when the underlying read ignores it, and read the last event or line when the stream ends without a final newline.

Optional LLM model controls​

Set LlmService.modelControl only when the provider has a real model-administration API. List the implemented methods in operations, and omit methods the provider cannot perform.

import {LlmService, type LlmModelControl} from '@dsh-ai-providers/core'

class VendorLlmService extends LlmService {
  readonly modelControl: LlmModelControl = {
    operations: ['status', 'unload'],
    status: (route, model, signal) => this.modelStatus(route, model, signal),
    unload: (route, model, signal) => this.unloadModel(route, model, signal),
  }
}

Resolve the named route and its connection on every call so settings changes take effect immediately. Validate the model name before sending it, pass the signal to the transport, and map the vendor's state into LlmModelRuntimeStatus. Report memory in bytes only when the response supplies a valid, non-negative value.

Model administration usually changes shared process state. Do not implement automatic unloading inside stream(), and do not imply that an aborted HTTP request rolled back a load or unload already accepted by the server.

3. Register through the context​

Register with ctx.aiProviders inside the plugin scope. Use ctx.effect() when registration returns or requires a disposer, so hot reload removes the provider cleanly.

ctx.effect(() => ctx.aiProviders.register(provider))

Give errors stable AiError categories and preserve the vendor response as diagnostic context without exposing credentials.

Map a failure according to what the caller can do next. Malformed caller data is invalid-input; a feature the vendor cannot perform is unsupported; credential, quota, and model refusals are rejected; transport and server failures are unavailable; an aborted request is cancelled. Reserve not-configured for selection or configuration that prevents the service from being resolved.

4. Expose settings metadata​

Describe reusable connections and capability options for the settings UI. Supply English and Simplified Chinese labels. Mark startup-only options explicitly; other configuration should support live updates.

Language-model integrations may also expose route factories and localized request options. Route IDs must remain stable because other DSH configuration can refer to them.

5. Test the contract​

Test request and response mapping, credential resolution, base URL handling, abort propagation, stream cleanup, error normalization, malformed responses, live configuration changes, and unload cleanup.

Run the workspace tests and type checks before publishing. Avoid asserting only against a public cloud service; compatible local servers are an important supported use case.

Review before publishing​

  • The provider ID and every LLM route ID are stable and unique.
  • Configuration stores credential references, never secret values.
  • Every external response is validated before it crosses the core contract.
  • AbortSignal reaches the transport, and streams release resources on completion, failure, return, and cancellation.
  • Registration and every listener or timer are scoped through ctx and unload cleanly.
  • UI labels and option descriptions exist in English and Simplified Chinese.
  • Provider-specific options are exposed only when the provider implements them.
  • modelControl.operations agrees with the methods the service implements.
  • Model-control tests cover route resolution, authorization, cancellation, malformed status responses, and unsupported operations.
  • Tests cover a compatible local or stub server so routine verification does not require paid credentials.