Initial commit: add CLAUDE.md and transcribe.py

2026-05-31 01:05:48 +08:00
parent 50f8d158c4
commit a727899ee5
2 changed files with 111 additions and 0 deletions
@@ -0,0 +1,60 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project
+
+cohere-transcribe — live speech-to-text using the Cohere ASR model (`CohereLabs/cohere-transcribe-03-2026`) via HuggingFace Transformers. Captures microphone audio, runs voice activity detection (VAD) to segment speech, transcribes each segment, and either prints text or injects it into the focused window via `wtype` (Wayland).
+
+## Development Environment
+
+Nix flake provides the dev shell (Python 3.14, portaudio, CUDA toolkit, wtype, uv). Direnv activates it automatically. Python deps managed by uv.
+
+```bash
+# Install/sync Python deps
+uv sync
+
+# Run the CLI (installed as entry point)
+uv run cohere on          # start daemon (background, types into focused window)
+uv run cohere off         # stop daemon
+uv run cohere status
+uv run cohere transcribe --stream   # live transcribe to terminal
+uv run cohere transcribe --mic 5    # record 5s then transcribe
+uv run cohere transcribe file.wav   # transcribe file
+
+# Run mic tests
+uv run python tests/test_mic.py
+```
+
+## Architecture
+
+Two modes share the same model/VAD pipeline but differ in output:
+
+- **Daemon mode** (`cohere on`): runs as a background process, transcribes speech segments and injects text into the focused window via `wtype`. State tracked in `~/.local/state/cohere/state.json`. The daemon is spawned by the CLI (`cli.py`) which launches `daemon_main.py` as a detached subprocess.
+- **Stream/one-shot mode** (`cohere transcribe`): runs in foreground, prints transcriptions to stdout with timestamps.
+
+### Key modules
+
+- `model.py` — model loading (`load_model`) and transcription (`transcribe_audio`). Single source of truth for `MODEL_ID` and `SAMPLE_RATE` (16kHz).
+- `vad.py` — RMS-based voice activity detection with `VADStateMachine`. Calibrates ambient noise threshold at startup. Configurable silence duration triggers segment boundaries.
+- `stream.py` — streaming transcription loop: audio callback feeds VAD, completed segments go to a transcription worker thread via queue.
+- `daemon.py` — same streaming pattern as `stream.py` but outputs via `wtype` instead of print. Also contains daemon lifecycle management (state file, PID tracking, start/stop).
+- `cli/cli.py` — Typer CLI with `on`/`off`/`status`/`transcribe` commands.
+- `transcribe.py` — original standalone script (not part of the package).
+
+### Data flow
+
+```
+Microphone → sounddevice.InputStream (50ms frames)
+  → VADStateMachine.process_frame()
+  → speech segment detected → Queue
+  → transcription_worker thread → transcribe_audio()
+  → output (wtype or print)
+```
+
+## Conventions
+
+- Package uses src layout (`src/cohere_transcribe/`), built with hatchling.
+- Entry points: `cohere` and `cohere-transcribe` both map to `cohere_transcribe.cli:main`.
+- VAD constants are in `vad.py` (frame size, pre-roll, silence limits, max segment length).
+- Daemon state lives at `~/.local/state/cohere/` (state.json, daemon.log).
@@ -0,0 +1,51 @@
+import sys
+import numpy as np
+import sounddevice as sd
+from transformers import AutoProcessor, CohereAsrForConditionalGeneration
+from transformers.audio_utils import load_audio
+from huggingface_hub import hf_hub_download
+
+# Load model
+print("Loading model...")
+processor = AutoProcessor.from_pretrained("CohereLabs/cohere-transcribe-03-2026")
+model = CohereAsrForConditionalGeneration.from_pretrained(
+    "CohereLabs/cohere-transcribe-03-2026",
+    device_map="auto"
+)
+
+def transcribe_audio(audio, language="en"):
+    inputs = processor(audio, sampling_rate=16000, return_tensors="pt", language=language)
+    inputs.to(model.device, dtype=model.dtype)
+    
+    outputs = model.generate(**inputs, max_new_tokens=256)
+    text = processor.decode(outputs, skip_special_tokens=True)
+    return text
+
+def record_audio(duration, samplerate=16000):
+    print(f"Recording for {duration} seconds...")
+    audio = sd.rec(int(duration * samplerate), samplerate=samplerate, channels=1, dtype='float32')
+    sd.wait()
+    return audio.flatten()
+
+# Parse arguments
+if len(sys.argv) > 1 and sys.argv[1] == "--mic":
+    duration = int(sys.argv[2]) if len(sys.argv) > 2 else 5
+    try:
+        mic_audio = record_audio(duration)
+        print("Transcribing...")
+        text = transcribe_audio(mic_audio)
+        print(f"\nTranscription:\n{text}\n")
+    except OSError as e:
+        print(f"Microphone error: {e}")
+        print("Hint: Run with nix-shell for PortAudio support")
+else:
+    print("Loading demo audio...")
+    audio_file = hf_hub_download(
+        repo_id="CohereLabs/cohere-transcribe-03-2026",
+        filename="demo/voxpopuli_test_en_demo.wav",
+    )
+    audio = load_audio(audio_file, sampling_rate=16000)
+
+    print("Transcribing...")
+    text = transcribe_audio(audio)
+    print(f"\nTranscription:\n{text}\n")