Initial commit: add CLAUDE.md and transcribe.py
This commit is contained in:
@@ -0,0 +1,60 @@
|
|||||||
|
# CLAUDE.md
|
||||||
|
|
||||||
|
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||||
|
|
||||||
|
## Project
|
||||||
|
|
||||||
|
cohere-transcribe — live speech-to-text using the Cohere ASR model (`CohereLabs/cohere-transcribe-03-2026`) via HuggingFace Transformers. Captures microphone audio, runs voice activity detection (VAD) to segment speech, transcribes each segment, and either prints text or injects it into the focused window via `wtype` (Wayland).
|
||||||
|
|
||||||
|
## Development Environment
|
||||||
|
|
||||||
|
Nix flake provides the dev shell (Python 3.14, portaudio, CUDA toolkit, wtype, uv). Direnv activates it automatically. Python deps managed by uv.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Install/sync Python deps
|
||||||
|
uv sync
|
||||||
|
|
||||||
|
# Run the CLI (installed as entry point)
|
||||||
|
uv run cohere on # start daemon (background, types into focused window)
|
||||||
|
uv run cohere off # stop daemon
|
||||||
|
uv run cohere status
|
||||||
|
uv run cohere transcribe --stream # live transcribe to terminal
|
||||||
|
uv run cohere transcribe --mic 5 # record 5s then transcribe
|
||||||
|
uv run cohere transcribe file.wav # transcribe file
|
||||||
|
|
||||||
|
# Run mic tests
|
||||||
|
uv run python tests/test_mic.py
|
||||||
|
```
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
Two modes share the same model/VAD pipeline but differ in output:
|
||||||
|
|
||||||
|
- **Daemon mode** (`cohere on`): runs as a background process, transcribes speech segments and injects text into the focused window via `wtype`. State tracked in `~/.local/state/cohere/state.json`. The daemon is spawned by the CLI (`cli.py`) which launches `daemon_main.py` as a detached subprocess.
|
||||||
|
- **Stream/one-shot mode** (`cohere transcribe`): runs in foreground, prints transcriptions to stdout with timestamps.
|
||||||
|
|
||||||
|
### Key modules
|
||||||
|
|
||||||
|
- `model.py` — model loading (`load_model`) and transcription (`transcribe_audio`). Single source of truth for `MODEL_ID` and `SAMPLE_RATE` (16kHz).
|
||||||
|
- `vad.py` — RMS-based voice activity detection with `VADStateMachine`. Calibrates ambient noise threshold at startup. Configurable silence duration triggers segment boundaries.
|
||||||
|
- `stream.py` — streaming transcription loop: audio callback feeds VAD, completed segments go to a transcription worker thread via queue.
|
||||||
|
- `daemon.py` — same streaming pattern as `stream.py` but outputs via `wtype` instead of print. Also contains daemon lifecycle management (state file, PID tracking, start/stop).
|
||||||
|
- `cli/cli.py` — Typer CLI with `on`/`off`/`status`/`transcribe` commands.
|
||||||
|
- `transcribe.py` — original standalone script (not part of the package).
|
||||||
|
|
||||||
|
### Data flow
|
||||||
|
|
||||||
|
```
|
||||||
|
Microphone → sounddevice.InputStream (50ms frames)
|
||||||
|
→ VADStateMachine.process_frame()
|
||||||
|
→ speech segment detected → Queue
|
||||||
|
→ transcription_worker thread → transcribe_audio()
|
||||||
|
→ output (wtype or print)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Conventions
|
||||||
|
|
||||||
|
- Package uses src layout (`src/cohere_transcribe/`), built with hatchling.
|
||||||
|
- Entry points: `cohere` and `cohere-transcribe` both map to `cohere_transcribe.cli:main`.
|
||||||
|
- VAD constants are in `vad.py` (frame size, pre-roll, silence limits, max segment length).
|
||||||
|
- Daemon state lives at `~/.local/state/cohere/` (state.json, daemon.log).
|
||||||
@@ -0,0 +1,51 @@
|
|||||||
|
import sys
|
||||||
|
import numpy as np
|
||||||
|
import sounddevice as sd
|
||||||
|
from transformers import AutoProcessor, CohereAsrForConditionalGeneration
|
||||||
|
from transformers.audio_utils import load_audio
|
||||||
|
from huggingface_hub import hf_hub_download
|
||||||
|
|
||||||
|
# Load model
|
||||||
|
print("Loading model...")
|
||||||
|
processor = AutoProcessor.from_pretrained("CohereLabs/cohere-transcribe-03-2026")
|
||||||
|
model = CohereAsrForConditionalGeneration.from_pretrained(
|
||||||
|
"CohereLabs/cohere-transcribe-03-2026",
|
||||||
|
device_map="auto"
|
||||||
|
)
|
||||||
|
|
||||||
|
def transcribe_audio(audio, language="en"):
|
||||||
|
inputs = processor(audio, sampling_rate=16000, return_tensors="pt", language=language)
|
||||||
|
inputs.to(model.device, dtype=model.dtype)
|
||||||
|
|
||||||
|
outputs = model.generate(**inputs, max_new_tokens=256)
|
||||||
|
text = processor.decode(outputs, skip_special_tokens=True)
|
||||||
|
return text
|
||||||
|
|
||||||
|
def record_audio(duration, samplerate=16000):
|
||||||
|
print(f"Recording for {duration} seconds...")
|
||||||
|
audio = sd.rec(int(duration * samplerate), samplerate=samplerate, channels=1, dtype='float32')
|
||||||
|
sd.wait()
|
||||||
|
return audio.flatten()
|
||||||
|
|
||||||
|
# Parse arguments
|
||||||
|
if len(sys.argv) > 1 and sys.argv[1] == "--mic":
|
||||||
|
duration = int(sys.argv[2]) if len(sys.argv) > 2 else 5
|
||||||
|
try:
|
||||||
|
mic_audio = record_audio(duration)
|
||||||
|
print("Transcribing...")
|
||||||
|
text = transcribe_audio(mic_audio)
|
||||||
|
print(f"\nTranscription:\n{text}\n")
|
||||||
|
except OSError as e:
|
||||||
|
print(f"Microphone error: {e}")
|
||||||
|
print("Hint: Run with nix-shell for PortAudio support")
|
||||||
|
else:
|
||||||
|
print("Loading demo audio...")
|
||||||
|
audio_file = hf_hub_download(
|
||||||
|
repo_id="CohereLabs/cohere-transcribe-03-2026",
|
||||||
|
filename="demo/voxpopuli_test_en_demo.wav",
|
||||||
|
)
|
||||||
|
audio = load_audio(audio_file, sampling_rate=16000)
|
||||||
|
|
||||||
|
print("Transcribing...")
|
||||||
|
text = transcribe_audio(audio)
|
||||||
|
print(f"\nTranscription:\n{text}\n")
|
||||||
Reference in New Issue
Block a user