Transcript Sources¶

cast2md uses a transcript-first workflow that prioritizes external transcript sources before falling back to Whisper transcription. This minimizes storage requirements (no audio download needed) and processing time.

Provider Priority¶

Phase 1: Feed Discovery¶

When a feed is added or refreshed, discover_new_episodes() runs:

RSS Parsing¶

The parser extracts Podcast 2.0 transcript info from each episode entry:

• transcript_url -- URL from <podcast:transcript> tag
• transcript_type -- MIME type (e.g., text/vtt, application/srt)

Pocket Casts Upfront Check¶

After creating episodes, the system checks Pocket Casts for episodes that don't have Podcast 2.0 transcripts:

1. Search for show via POST podcast-api.pocketcasts.com/discover/search
   • Search by feed title, match result by author name
   • Cache pocketcasts_uuid on feed for future lookups
2. Get episodes via GET podcast-api.pocketcasts.com/mobile/show_notes/full/{uuid}
   • Returns JSON with all episodes
   • Each episode may have pocket_casts_transcripts[] array with VTT URLs
3. Match episodes by title similarity + published date within 24 hours
   • If match found with transcript URL, store in pocketcasts_transcript_url

Phase 2: Transcript Download¶

When a TRANSCRIPT_DOWNLOAD job runs, the provider chain is invoked:

_providers = [
    Podcast20Provider(),    # Check transcript_url first
    PocketCastsProvider(),  # Fallback to pocketcasts_transcript_url
]

Podcast20Provider¶

• Checks: episode.transcript_url exists
• Downloads: From the URL in the RSS feed
• Parses: Based on MIME type (VTT, SRT, JSON, plain text)
• Source tag: podcast2.0:vtt, podcast2.0:srt, podcast2.0:json, or podcast2.0:text

PocketCastsProvider¶

• Always tries as a fallback
• If episode.pocketcasts_transcript_url exists (from upfront discovery), downloads directly
• Otherwise, searches the Pocket Casts API (slower path)
• Source tag: pocketcasts

Data Model¶

Episode Fields¶

Feed Fields¶

Transcript Source Tags¶

When an episode reaches completed, the transcript_source column records how it was obtained:

403 Handling (Pocket Casts)¶

Pocket Casts sometimes returns transcript URLs before the files exist on S3. The system handles this with age-based retry logic:

A scheduler job runs hourly to:

1. Find episodes with awaiting_transcript status and next_transcript_retry_at <= now
2. Re-queue TRANSCRIPT_DOWNLOAD jobs
3. Transition aged-out episodes to needs_audio

Adding New Providers¶

1. Create src/cast2md/transcription/providers/newprovider.py

2. Implement the TranscriptProvider base class:

   class NewProvider(TranscriptProvider):
       @property
       def source_id(self) -> str:
           return "newprovider"
   
       def can_provide(self, episode, feed) -> bool:
           return True  # Your check logic
   
       def fetch(self, episode, feed) -> TranscriptResult | None:
           # Download and return transcript
           return TranscriptResult(content=markdown, source=self.source_id)
   

3. Register in providers/__init__.py:

   _providers = [
       Podcast20Provider(),
       PocketCastsProvider(),
       NewProvider(),  # Add here
   ]
   

Key Files¶

Known Limitations¶

1. Pocket Casts search matching -- relies on author name matching, which may fail for some podcasts
2. Rate limiting -- Pocket Casts API calls are rate-limited to 500ms between requests
3. Episode matching -- title normalization may not handle all edge cases (special characters, truncation)