Part 6 of 7 in the Building ZumiScribe series. Try ZumiScribe, free in your browser.

The jobs a browser tab cannot do

ZumiScribe is a web page with no server of its own and no account: you bring a key for a speech service, or use a transcriber that runs on your device. In Chrome or Edge you can record, fix speakers, summarise and export with nothing installed, and I wanted that to stay the default. But as the feature list grew, several items turned out to be things a tab cannot do:

Each needs a program on the device. The answer is ZumiScribe Local: a small menu-bar app for the Mac, and an early test version for Android. Neither replaces the web page; they make local things available to it.

Why the page stays in Chrome

Why not put ZumiScribe inside the app's own window? Because the Mac app's built-in web view (WKWebView) has no folder picker, so no File System Access API, which the library and audio files rely on, and no Chrome text reader. So ZumiScribe stays in Chrome or Edge, opened from the website, and the local app never serves the pages.

What it replaced

Before the app, running everything locally meant separate programs: the speech engine on 8765 and 8766, a glossary on 8767, the follow-along relay on 8780, the AssemblyAI token helper on 8781, and Ollama on 11434 for summaries. Most needed Python, and each had its own start command.

The plan, agreed on 22 October 2026, was one app holding the small services, written in Rust inside Tauri so no Python is needed for them. The speech engine stays a separate program because it needs Apple's MLX libraries, which are Python. Ollama stays outside: the app only checks that it is reachable.

One port, and keeping private routes private

All the small services share one port, 8780, with different paths. The catch is that they do not share an audience. The relay has to be reachable by phones, and sometimes through a tunnel; the glossary, the token helper and the engine controls must never be. So the rule is by path, checked on every request. /follow/* and /ws/* are public, and rooms already need a key (and an optional PIN) to publish. Everything else is private, and must pass four checks:

  1. The connection comes from this computer (the peer address is loopback), so a phone or another machine on the network is refused even if it forges headers.
  2. The Host header is 127.0.0.1, localhost or [::1], which also stops DNS-rebinding tricks.
  3. No forwarding headers are present. This one matters because a tunnel also connects from this computer, so the peer address alone proves nothing. Tunnels add headers such as X-Forwarded-* or CF-*, or keep the public Host.
  4. If there is an Origin, it must be one of ZumiScribe's own.

The tests simulate a tunnel and a machine on the network and expect every private path to answer 403; the first skeleton had 19 Rust tests over real connections. I also wrote down the weak spot: a tunnel that rewrites Host and strips every header would defeat check 3, though not checks 1 and 4 for a browser, and the private paths hold no keys. If I ever doubt it, a second loopback-only port is a small change.

By default the app listens only on 127.0.0.1. "Let phones on my network follow" is a switch that rebinds it to all interfaces, and only then does macOS ask about incoming connections. If an older script already holds 8780, the app takes the next free port (8781 to 8799) and says which.

The browser's rules for talking to localhost

An https page normally cannot call plain http, but Chrome treats http://127.0.0.1 as a secure context, so a page on https://zumilabs.com may call it. Chrome also asks for a private-network header, and both the Mac app and the Android bridge answer with Access-Control-Allow-Private-Network: true. Safari and Firefox are not supported targets.

The same rule shaped the company server. That exception covers only the local machine, so a server has to be https and wss, behind a proxy with a certificate. Check my setup tests it, with a line that reads "This page is https but ... is not".

Then there is Chrome's local network permission: a page that talks to things on the device triggers a question about letting the site "access other apps and services on this device". Earlier versions looked for the local services before the page had said why, so the prompt could arrive unexplained. For a privacy-focused tool, that is a poor first impression.

So in early November I changed it. Nothing on the computer is probed before the welcome has explained it, or before Settings is opened. The prompt appears only if you choose ZumiScribe Local or the local engine, after the welcome has said why, and the page shows that it is checking. The install page says what to expect: choose Allow, because that is how the page talks to the app on your own Mac.

A setup wizard, and a page you can screenshot

Installing is where people give up, so the first launch is a step-by-step setup window, and every step has a test with a plain pass or fail and what to do about a fail. The plan had eight steps (your Mac, the services, Chrome, the speech engine, phones, summaries, AssemblyAI, finish); it has grown to twelve, adding the Voz + Who and Granite models and other engines you run yourself. Some tests do real work: the engine step transcribes a spoken sentence with speaker labels, and the phone step sends a test caption to a phone.

The Chrome step opens the website's Check my setup page, because the app calling its own /health proves little: failures really show from the browser, where the origin, the https rules and Chrome's permission apply. The page checks that the services answer, the secure connection rules are met, the page's origin is allowed, a test caption goes to the relay and back, the glossary and helper answer, the engine's WebSocket opens and Ollama is reachable, with a microphone test and a live level meter. Each failed line says what to do, and Copy a report gives versions and results, with no keys or text. The aim was a page that a user or an administrator can screenshot first when something is wrong, for the Mac app and a company server alike.

The engine manager

The speech engine (Parakeet for words, Sortformer for speakers) is the heavy part: about 6 GB, Apple silicon only. The app's job is that the user never meets Python. One click downloads a pinned uv, which fetches a private Python, creates two environments from pinned requirement files and downloads the two models from Hugging Face, all inside the app's Application Support folder. The app then starts the engine, watches it, restarts it if it dies, keeps the last log lines and stops it on Quit. Remove deletes the folder.

Files the app downloads itself do not get the browser's quarantine flag, so no extra macOS warnings appear. On 22 October a clean install into an empty folder took 340 seconds, the engine was ready four seconds after starting, and a spoken test sentence came back with a speaker label. If an install fails, Try again carries on from what was already downloaded.

Installing without a signed download

There is no Apple Developer account yet, so the app is ad-hoc signed, and I documented two routes plainly. The quiet one is a Terminal command: curl fetches install.sh, which downloads the zip, checks its checksum, puts the app in Applications and opens it. A file fetched by curl is not marked as downloaded from the internet, so macOS shows nothing. The other is the zip from the browser, with the Open Anyway steps (macOS 15 and later: System Settings, Privacy & Security, Open Anyway; macOS 14 and earlier: right-click, Open).

Signing is a switch for later: with an identity and notarisation settings in the environment, package.sh signs, Tauri notarises, and install.sh is no longer needed. The first version, 0.1.0, was published on 22 October, a download of about 3 MB.

The same pieces as a container for a company

Some companies want nothing installed on anyone's computer. The services are one Rust library with two thin shells, the Tauri app and a headless zumiscribe-server, so the server is the same code with different settings. It ships as a services image of about 53 MB with a compose.yaml and an optional Caddy proxy for automatic https; I built and ran it, including https through Caddy. The systemd unit has not been tried on Linux yet.

In server mode private routes cannot rely on loopback, and there is no sign-in yet: I deferred it until a company asks and will test it. Until then a server is for a trusted network only. It warns at startup when it listens on a non-local address with no sign-in, and /health reports "auth": "none", which Settings shows. The allowed-origins list protects against other web pages, not other people on the network, and the guide says so. A speech engine on a GPU server is planned, not built, and Voz + Who never runs on a server because its licence forbids a hosted service.

Windows: a plan, not a product

Windows is a plan dated 10 November, with nothing built. The services should build as they are, since they already build for Linux. The plan lists about ten Mac-only places in the app, such as opening folders, start at login and Cmd shortcuts; other apps' sound needs a rewrite with WASAPI loopback; and Parakeet's MLX build does not run on Windows, so it gets its own spike while a first release would ship with cloud transcribers and Voz + Who. Testing happens without a PC: GitHub Actions for the installer, a Windows on Arm VM, and one real laptop before telling anyone it works.

Android: from a spike to 0.8

The Android app began as a spike with one question: can a phone transcribe a conversation, with speaker labels, entirely on the phone? It runs Parakeet and speaker labelling through sherpa-onnx on the phone's CPU, with models of about 680 MB downloaded once.

Version 0.4 added the part that matters here: a browser bridge. A foreground service, with a notification and a Stop button, listens on 127.0.0.1 only, on 8765 and 8766, with the same protocols as the Mac engine, so ZumiScribe in the phone's browser sees just another engine. An origin list and a Host check mean it answers only ZumiScribe's pages, through the phone's own address. A wake lock is held only while transcribing. 0.5 was a new look with a download progress bar. 0.6 added on-device caption translation with ML Kit through the same bridge, which roughly doubled the APK from 14 MB to about 27 MB. 0.7 reorganised it into Home, Models and an advanced screen, and made speaker labels optional. 0.8 renamed that screen Test, added a clear progress card and let you cancel a recording. It is still a test version, installed from a file and signed with a debug key.

What went wrong along the way

What I would tell someone building a helper for a web app

Keep the web app the main thing and the helper small. Decide early which routes are public and which are private, keep the rule in one place, and test it with a fake tunnel and a fake neighbour, not only the happy path. Send Chrome's private-network header, and do not touch localhost until the user has asked for something that needs it and you have said why. Give every setup step a real test, and give people a page they can screenshot. Write the unsigned install steps honestly. And build the core as a library with thin shells from the first day: that is why the company server was a new shell and a configuration, not a rewrite.

ZumiScribe

Transcripts that know who said what

Live captions, speaker labels, pictures in context and summaries, in your browser. There is no ZumiScribe server: transcripts stay in your browser or your own folder, and you choose which transcriber hears the audio, including ones that run on your own computer.

Try ZumiScribe