Introducing RecallWhisper: Private Speech, Local Memory

Important thoughts rarely arrive when it is convenient to write them down. They appear during a walk, in a conversation, or halfway through another task. RecallWhisper is an Android app designed to capture those moments, turn speech into searchable text, and create useful summaries without handing long-term control of your personal archive to a cloud platform.

The app records through a native Android foreground service, detects conversational pauses with Silero VAD, and stores its durable data on the phone. Audio files are encrypted with AES-256-GCM, with their keys protected by Android Keystore. Transcripts, summaries, processing state, and raw API responses are kept in the app's local Room database.

RecallWhisper still needs transcription and summarization models, but it talks directly to OpenAI-compatible APIs. You can use compatible hosted services or run both APIs on hardware you control. This guide introduces the normal app workflow and then builds a private processing stack with OpenASR, X-ASR, llama.cpp, and Qwen3.5.

How RecallWhisper works

The processing flow is deliberately simple:

Microphone → encrypted local WAV → transcription API
           → local raw transcript → summarization API
           → local summary/search/export

The phone remains the permanent home of the recording and its results. The configured transcription server receives audio for speech recognition, while the summarization server receives transcript text. When both servers are self-hosted, the entire workflow can remain on infrastructure you control.

Key features include:

• A native Kotlin foreground recorder and Android Quick Settings tile
• Silero voice activity detection with a configurable 5–60 second pause tolerance
• Separate API URLs, bearer tokens, and model IDs for transcription and summarization
• Wi-Fi-only processing by default, with optional cellular processing
• Local audio playback, transcript search, deletion, and JSON export
• An API playground for model discovery, prompt editing, parameter tuning, and diagnostics

Using the app

1. Install RecallWhisper

Visit the RecallWhisper repository for the current source and project downloads. To build the Android APK yourself, install Flutter and run:

flutter analyze
flutter test
flutter build apk --release --split-per-abi

The split build produces separate APKs for Android CPU architectures, allowing you to install the smaller artifact that matches your device.

2. Connect the processing APIs

Open Settings and configure the transcription and summarization services separately. Each base URL must include /v1, but should not include a final endpoint such as /audio/transcriptions or /chat/completions; RecallWhisper adds that route itself.

HTTPS is required by default. Allow insecure HTTP is available for a controlled development network, but enabling it sends audio and bearer tokens across the network without encryption. It should never be used on public or untrusted networks.

3. Record naturally

Start the recorder in RecallWhisper or from its Quick Settings tile. The foreground service keeps capture active while you use other apps. Silero VAD separates speech from silence, and the conversation pause setting controls how long RecallWhisper waits before closing the current segment.

A short pause tolerance creates smaller, faster segments. A longer tolerance is useful when a conversation contains natural gaps and should remain grouped together.

4. Let the phone process recordings

RecallWhisper queues each completed segment for transcription and then summarization. Processing uses Wi-Fi by default; cellular access must be enabled explicitly. The local processing state makes it possible to see whether a segment is queued, being processed, completed, or needs attention.

5. Review and manage your memory

Use the app to play the encrypted local recording, read its transcript and summary, search transcript text, or delete material you no longer need. Export all data as JSON opens Android's document picker and writes metadata, raw transcription results, transcripts, summaries, checksums, and processing state to a file you choose. Audio stays separately encrypted in the app's private storage.

Build a self-hosted processing stack

The reference setup uses one specialized service for each job:

RecallWhisper
  ├─ /v1/audio/transcriptions → OpenASR + X-ASR zh/en
  └─ /v1/chat/completions     → llama.cpp + Qwen3.5 4B GGUF

Keeping the services separate lets you choose the best speech model and language model independently. It also explains why RecallWhisper has two sets of URLs, tokens, and model names.

1. Start OpenASR with X-ASR zh/en

Install the current openasr binary from the OpenASR releases, then download the recommended Q8 bilingual model:

openasr pull xasr-zh-en:q8
openasr transcribe sample.wav --model xasr-zh-en

The X-ASR model card also provides FP16 and Q4 variants. Q8 is a practical default; choose Q4 when memory is limited.

Create a bearer token and start the service on port 9099:

openasr apikey create --name recallwhisper
export OPENASR_TOKEN='replace-with-the-created-token'
openasr serve --help
openasr serve \
  --addr 127.0.0.1:9099 \
  --model xasr-zh-en \
  --pairing-admin-token-env OPENASR_TOKEN

Save the generated API key: it becomes the transcription token in RecallWhisper. Binding to 127.0.0.1 keeps the plain HTTP service private on the host while a reverse proxy provides HTTPS. If OpenASR runs on a different machine from the proxy, follow the OpenASR server documentation for authenticated remote serving and TLS pairing.

OpenASR exposes the OpenAI-compatible /v1/models and /v1/audio/transcriptions routes used by the app.

2. Start llama.cpp with Qwen3.5 4B

The official Qwen3.5 4B model does not include GGUF files. This example uses the 2.74 GB Q4_K_M quantization from the Unsloth Qwen3.5-4B GGUF repository.

Create a working directory and download the model:

mkdir -p recallwhisper-llm/models
cd recallwhisper-llm

curl -fL \
  'https://huggingface.co/unsloth/Qwen3.5-4B-GGUF/resolve/main/Qwen3.5-4B-Q4_K_M.gguf?download=true' \
  -o models/Qwen3.5-4B-Q4_K_M.gguf

Create compose.yaml, replacing <SET_YOUR_LLAMA_API_KEY> with a strong, unique value:

services:
  llama:
    image: ghcr.io/ggml-org/llama.cpp:server
    # NVIDIA: ghcr.io/ggml-org/llama.cpp:server-cuda
    # AMD/Intel: ghcr.io/ggml-org/llama.cpp:server-vulkan
    restart: unless-stopped
    ports:
      - "127.0.0.1:8181:8080"
    volumes:
      - ./models:/models:ro
    environment:
      LLAMA_API_KEY: <SET_YOUR_LLAMA_API_KEY>
    command:
      - --model
      - /models/Qwen3.5-4B-Q4_K_M.gguf
      - --alias
      - qwen3.5-4b
      - --host
      - 0.0.0.0
      - --port
      - "8080"
      - --ctx-size
      - "16384"
      - --parallel
      - "1"
      - --reasoning
      - "off"

Start the container:

docker compose up -d

The host listens only on 127.0.0.1:8181; port 8080 remains internal to the container. The --alias value becomes the model ID returned by /v1/models, and --reasoning off ensures that RecallWhisper receives final summary text in choices[0].message.content instead of only a reasoning field.

The official llama.cpp Docker guide covers the available server images. See the llama.cpp server reference when you need GPU offload, TLS, a larger context, or more concurrent requests.

3. Put both APIs behind HTTPS

Do not expose the two plain HTTP ports directly to the internet. If DNS for asr.example.com and llm.example.com points to the server, Caddy can obtain trusted certificates and proxy both services:

asr.example.com {
    reverse_proxy 127.0.0.1:9099
}

llm.example.com {
    reverse_proxy 127.0.0.1:8181
}

The Caddy reverse-proxy guide explains the surrounding server setup. Keep authentication enabled in OpenASR and llama.cpp even when Caddy is in front of them.

For private-LAN HTTP instead, bind each required port to the LAN interface, restrict access with a firewall, and enable Allow insecure HTTP in RecallWhisper. This is a development convenience, not a safe public deployment.

4. Enter the self-hosted settings

In RecallWhisper, save the following values with your own domains and secrets:

Open Debug and API playground before enabling continuous processing. Run the authenticated transcription health check, load the summarization model list, and send a short JSON-mode test. This catches URL, token, model, and response-format problems before they affect real recordings.

Troubleshooting

• 404 Not Found: Make sure the configured base URL ends in /v1 and does not include the endpoint that RecallWhisper appends.
• 401 Unauthorized: Check transcription and summarization tokens independently; the two credentials are intentionally separate.
• Cleartext HTTP rejected: Use trusted HTTPS, or enable insecure HTTP only on a controlled private network.
• TLS certificate error: Install a certificate Android trusts. The insecure HTTP option does not disable HTTPS certificate validation.
• Model not found: Query /v1/models and match the configured name exactly to xasr-zh-en or the llama.cpp --alias.
• No final summary: Keep Qwen reasoning disabled so final text appears in choices[0].message.content.
• llama.cpp container exits: Run docker compose logs llama, check the GGUF path and permissions, and update the image if its build predates Qwen3.5 support.

A private memory system you can inspect

RecallWhisper does not try to become another account-based cloud notebook. It gives Android a dependable ambient recorder, keeps the durable archive local, and lets standard APIs handle the expensive model work. That division makes the system practical on a phone while leaving the processing backend replaceable.

For the most private deployment, self-host both APIs, keep their unencrypted ports on loopback, use trusted HTTPS, and retain separate credentials. The result is a voice memory workflow that remains searchable and useful without surrendering ownership of the archive it creates.