Let a user who wants a bigger model than their Mac can run point transcription at an OpenAI-compatible HTTP endpoint — any endpoint, any provider — without weakening the default local posture. Local stays the default; the remote path is explicitly opt-in, visibly indicated while active, and carries no vendor names in code.
Transcription today is 100 % on-device (WhisperKit; Lightning in bench). That is the right default and the product’s identity, but two real user groups want more:
The agent layer already made exactly this trade once: network use is allowed, but only user-configured, never default (VISION.md privacy contract, clauses 3–4). This spec extends the same carve-out to transcription.
Approval note: this change touches two AGENTS.md hard-rule areas (a network call in the dictation path; the privacy contract). Both were explicitly approved by the maintainer commissioning this spec.
public struct RemoteProfile: Sendable, Equatable {
var baseURL: URL // e.g. https://api.openai.com/v1
var model: String // e.g. whisper-1; sent only if non-empty
var auth: RemoteAuth
}
public enum RemoteAuth: Sendable, Equatable {
case none // local gateways, whisper.cpp server
case bearer // Authorization: Bearer <secret>
case header(name: String) // <name>: <secret> — corporate gateways
}
No secret lives in the profile or in UserDefaults. Secrets go in the Keychain via CredentialStore, keyed by endpoint host (KeychainCredentialStore, one generic-password item per host). Keying by host is the security invariant: an engine can only ever attach the credential saved for the host the request targets, so editing the URL can never leak the old host’s key to a new one.
RemoteEngine implements the existing TranscriptionEngine protocol (no protocol changes). Per request:
AudioResampler, AVFoundation only — the recorder writes device-native rate, often 48 kHz stereo, which is ~6× the upload weight for zero quality gain). Above compressAboveBytes (700 KB, ~22 s) the WAV is further encoded to 32 kbps mono AAC/m4a — ~7× smaller, so long dictations stay inside gateway body caps and upload faster. Measured against a live endpoint, WAV and AAC transcripts matched on clean, noisy, and Chinese speech. Encoding runs through AVAssetWriter (AVAudioFile cannot write compressed formats), and any encoder failure falls back to the WAV rather than failing the dictation.POST {base}/audio/transcriptions (path not appended if the user already pasted the full path), multipart: file, model, optional language, response_format=json.{"text": …}; plain-text bodies accepted as a fallback for non-conforming servers.Transport rules: HTTPS required (plain HTTP allowed for loopback only, for local whisper.cpp/Ollama-style servers); HTTP redirects refused — audio and credential go only to the configured host, never wherever a 3xx points; URLs with embedded user:pass@ rejected (and auto-stripped in Settings) so no secret can ride into UserDefaults; ephemeral URLSession so no response ever lands in the on-disk URL cache; 60 s request timeout. Requests never identify the app to the endpoint: the User-Agent is user-configurable (remoteUserAgent, folded to one header line; a neutral default when empty — the CFNetwork default would embed the bundle name) and the multipart boundary is app-neutral. Errors surface through the existing EngineError path — a misconfigured remote errors, it never silently falls back to local.
EngineKind is untouched: the app constructs RemoteEngine directly (as it already does WhisperKitEngine), because makeEngine(model:) cannot carry profile config. CLI/bench remote support is out of scope.
Transcription picker (On this Mac / Remote endpoint (experimental)), and when remote: endpoint URL, model, user agent, auth method, and a Keychain-backed API-key field tied to the URL’s host. UserDefaults keys: transcriptionBackend, remoteEndpoint, remoteModel, remoteUserAgent, remoteAuthMethod, remoteAuthHeaderName.path=remote; failures log a diagnostics event too (backend kind + error, never transcript).SpeechModelCatalog.canDelete); an in-flight dictation protects them all. Switching back to local when the selected model’s weights aren’t on disk must not warm silently: the warm path’s only surface is the menu-bar banner, which is hidden behind the Settings window the user is standing in, so a multi-gigabyte fetch would run unannounced and unstoppable. It goes through the existing speech-model download sheet instead (confirm → progress → Cancel). Cancelling puts back only the warm-up the sheet displaced, never re-enters the warm path’s own fetch, and never rewrites the user’s backend choice: local stays selected with no engine, the hotkey refuses the recording up front instead of losing the utterance to the nil-engine guard at stop, and a Settings row offers the download again (the picker can’t — the model is already the selection). The same rule covers the picker: a not-yet-downloaded model always confirms in the sheet, even while a launch warm-up is already downloading another one (that transfer is stood down when the sheet’s download starts).modelID becomes "<model> @ <host>" for remote runs). Pending-recovery re-transcription stays local..error (it used to hide as if the dictation had evaporated), showing DictationFailure.message(for:) — the engine’s user-facing line, wrapper and server body stripped. A remote profile that cannot work (no key, not signed in, expired Access session) is caught by DictationFailure.remotePreflightMessage before recording starts, so the user reads the instruction instead of losing an utterance to it.While a remote profile is active, the recording overlay must never claim locality:
globe · remote chip is shown during recording and transcribing (same treatment as the SPEC-031 kickoff chip, which is the contract’s existing network indicator); a kickoff recording with a remote backend shows both chips — each network hop gets disclosed;via <host> instead of On your Mac (polishing keeps On your Mac — polish stays local);Sending to <host>… / transcribing via <host> instead of the local claims);VISION.md privacy-contract clause 1 and README wording change from unconditional to default-scoped in the same PR; the sentence pattern follows the agent carve-out already in VISION.md.
This is the first feature that can send audio off-device, so the change is fenced:
via <host> subline).transcriptionBackend unset or local, no code in this spec executes on the dictation path (code review + existing tests green).RemoteEngine sends a well-formed OpenAI-compatible multipart request with bearer / custom-header / no auth (unit tests: request shape, auth headers, full-path URL not doubled).user:pass@ rejected; redirects refused without a second request (unit tests).EngineError (unit test).via <host> subline during a remote dictation, and On your Mac never appears while remote is active (manual: configure endpoint, dictate, observe overlay).<model> @ <host> (manual).On this Mac with the selected model deleted opens the download sheet, not a silent transfer; Download shows progress and hot-swaps the engine when it lands; Cancel starts nothing, leaves a “isn’t on your Mac yet — Download” row in Settings, and makes the dictation hotkey beep with an error rather than record (manual).RemoteAuth is a closed enum designed to grow cases without touching transport..ogg would mean bundling libopus + libogg for a marginal size win).RemoteProfile, but that is its own spec.docs/i18n/*) — machine-translated; the “nothing leaves” → “by default” sweep lands with the next translation refresh (English README, zh-CN README, docs site, VISION, and in-app copy are amended in this spec).detectedLanguage from remote (verbose_json is not universally supported; SPEC-035 normalisation falls back to system language).