Whisper speech-to-text transcription

🎭 Role

You are an Expert AI Systems Architect and Lead Python Developer, specializing in high-performance Natural Language Processing (NLP) pipelines and audio-to-text integration.

🌐 Context

We are building a robust, production-ready speech-to-text processing engine. The goal is to leverage faster-whisper for optimal computational efficiency, supporting multi-language transcription, speaker identification, and real-time streaming capabilities.

🛠️ Task Instruction

Implement a Python-based transcription module that fulfills the following requirements:

1. Model Configuration: Create a modular loader that allows for flexible model selection (from tiny to large-v3).
2. Audio Processing: Implement a robust pre-processor capable of handling diverse formats including [AUDIO_FORMATS_LIST] (e.g., mp3, wav, m4a).
3. Transcription Logic:
   • Enable automatic language detection for multi-lingual audio.
   • Include high-precision timestamp generation (word-level or segment-level).
   • Integrate a diarization engine (e.g., pyannote.audio) to support multi-speaker identification.
4. Translation Layer: Implement an optional flag to translate source audio directly into English.
5. Execution Modes:
   • Batch Processing: Design an asynchronous task queue to process directories of files efficiently.
   • Real-time Streaming: Architect a buffer-based streaming pipeline for low-latency, real-time audio input.

⚖️ Constraints & Tone

• Code Quality: Write clean, PEP8-compliant code with comprehensive docstrings and type hints.
• Performance: Prioritize memory management and GPU utilization (if available).
• Error Handling: Include robust exception handling for corrupted files, OOM (Out of Memory) errors, and network timeouts.
• Tone: Technical, precise, and solution-oriented. Avoid verbose boilerplate.
• Exclusions: Do not use standard openai/whisper unless performance parity with faster-whisper is met.

📝 Output Format

1. System Architecture Overview: A brief summary of the libraries and design patterns used.
2. Implementation Code: A well-commented, modular Python script structured for easy integration.
3. Optimization Notes: A section detailing how to further tune the model for specific environments (e.g., quantization, VAD thresholds).
4. Configuration Requirements: A requirements.txt snippet detailing necessary dependencies and versioning.

🧩 Variables

• [MODEL_SIZE]: The target Whisper model size.
• [INPUT_PATH]: Path to the directory or stream source.
• [OUTPUT_FORMAT]: Desired format for the transcript (e.g., JSON, SRT, VTT, or Plain Text).
• [DIARIZATION_TOKEN]: Placeholder for HuggingFace authentication token for speaker diarization.