Every YouTube download — /force and the normal clip path alike — was
failing at the data-transfer stage with "HTTP Error 403: Forbidden".
Extraction succeeded and a format was selected; only the media fetch
died, which is why it read as a per-video problem rather than a broken
extractor.
The cause was YouTube retiring the cookieless `android_vr` client that
yt-dlp 2026.6.9 pinned. Its media URLs now 403 for every video, so the
failure was global rather than specific to the video that surfaced it.
2026.8.19 re-points at `visionos` and downloads work again.
Refresh the cookie-withholding note, which named android_vr as the
client whose URLs work, and record the 403-on-data-transfer signature as
the tell for a retired client so the next occurrence goes straight to a
yt-dlp upgrade instead of the cookie or format logic.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QH4jMHfUTDhm41BrRtkPc3
YouTube's only muxed mp4 is format 18 (360p), so the muxed-first selector
was posting 360p even when 1080p was available. Split streams are now
preferred for YouTube and merged locally; every other host keeps the
muxed-first chain unchanged (verified identical selection for Instagram,
X and Reddit).
Three things the obvious version got wrong:
- A height<=1080 cap excludes portrait video: a 1080p Short or reel is
1080x1920. Resolution is capped with -S res:1080 instead, which keys
off the smaller dimension.
- h264 is a hard first preference, not an -S sort key. At equal
resolution YouTube's VP9/AV1 renditions are smaller and win on size,
but neither reliably plays in an mp4 container on older Signal
clients. Dropping a resolution rung to stay on h264 is the better
trade, and is usually smaller too: a 10-min video takes 480p h264
(37 MB) over 720p AV1 (78 MB).
- The audio cap matters. YouTube's 388k track is 29 MB by itself on a
10-min video, enough to push an otherwise-fine merge past the 100 MB
Signal limit; abr<=160 takes the 129k track instead.
The 88 MB video cap makes yt-dlp step down automatically on long videos,
so a 3-min video lands at 1080p (80 MB) and a 10-min one at 480p.
Clips get the same treatment — they're YouTube-only, and 60s at 1080p is
only ~29 MB. /force is deliberately left muxed-first: it exists for the
biggest, slowest videos, where 1080p would risk the rsync timeout.
Measured, old -> new: 3-min 360p/11 MB -> 1080p/80 MB; 10-min 360p/27 MB
-> 480p/37 MB; portrait Short 360x640/6 MB -> 1080x1920/23 MB. Verified
end to end: real downloads decode as h264+aac, clip lands at exactly
60.000s.
Reddit links now download and post like the other platforms.
Two changes were needed, not just the URL pattern:
- Regex: yt-dlp's Reddit extractor only matches /comments/ permalinks, so
the app's /r/<sub>/s/<code> share links and redd.it / v.redd.it
shortlinks are listed explicitly — they redirect to the canonical URL,
which yt-dlp follows. Without them the link is silently dropped.
- Format selector: Reddit publishes video and audio as separate DASH/HLS
streams with no muxed rendition, so every `best...` term failed and
yt-dlp aborted with "Requested format is not available". Added a
bv*+ba merge tail (and a bare bv* for silent videos) to the normal and
/force selectors. Muxed hosts still resolve at the earlier terms, so
YouTube/IG/TikTok/X selection is unchanged.
Posts with no video report "No media found", already in the silent list,
so ordinary Reddit links shared in chat draw no reply.
signalbot logs every received envelope at INFO via `[Raw Message] {json}`. On a
busy account the delivery/read receipts, typing indicators, and read-sync
messages flood journald hard enough to rotate the journal down to ~1 minute,
burying real traffic and making failures impossible to debug after the fact.
Add a logging filter on the `signalbot` logger that suppresses content-free
envelopes (anything without a `dataMessage`) at INFO, while still emitting them
if the logger is explicitly set to DEBUG. Real message content stays at INFO.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two YouTube regressions stacked up:
- yt-dlp 2026.3.17 hit "HTTP Error 403: Forbidden" on the actual media
download. Bumping to 2026.6.9 fixes the data-download 403, but its EJS
n-sig challenge solver now requires Node >= 22, and Debian 13 only ships
Node 20 ("node-20.x (unsupported)" -> only image formats resolve). bot.py
now points --js-runtimes at a standalone /opt/node22 via the new
JS_RUNTIME constant.
- With a cookies.txt present, yt-dlp skips the cookieless android_vr/android
clients (whose URLs work) and falls back to the `tv` client, whose media
URLs 403. New _cookie_args(url) withholds cookies for YouTube hosts only,
while still sending them to Instagram/TikTok/X.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The share sheet in the Instagram app now emits instagram.com/reels/<id>
(plural), which INSTAGRAM_URL_PATTERN didn't match — only /reel/ and /p/.
A non-matching URL is silently dropped (regex never matches -> no download,
no error reply), so reels links appeared to do nothing. Broaden the pattern
to (?:share/)?(?:reels?|p|tv) to also cover IGTV (/tv/) and the /share/
prefix. Verified against should-match / should-not-match URL sets; other
platform patterns unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wrap the forced yt-dlp download (and its node/ffmpeg children) and the rsync
upload with 'nice -n 19 ionice -c 3' so the known-big, slow /force job yields the
CPU and disk to everything else under load while still finishing fast when idle.
Normal short downloads stay at full priority. Best-effort: prefix is empty if the
tools are absent.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When a posted video is too long (>1h, probed up front) or a normal download
fails as too-big/timeout, the bot offers /force. Replying /force downloads the
full video off-loop (separate slot, no size cap, --max-filesize 10G), uploads it
via rsync/SSH to a web docroot, and posts a tokenized link auto-deleted after 24h
by an hourly sweep (also evicts oldest past 10 GB; remote-clock TTL; only ever
removes its own token dirs).
Hosting target is env-configured (FORCE_REMOTE/FORCE_REMOTE_DIR/FORCE_BASE_URL/
FORCE_SSH_PORT/FORCE_SSH_KEY) so published code carries no infra details; the
feature self-disables when unset (probe is skipped too, so normal posting is
unchanged). Staged on disk (not tmpfs), self-heals orphaned staging dirs, and
re-arms the offer on transient failure. See signal-bot.service.example.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Switch storage backend from in-memory to SQLite (silences the library warning).
recent_urls (the edit-redelivery dedup map) now survives restarts: it's tiny and
TTL-bounded, stored via the library KV store, with wall-clock timestamps and
prune-on-load. last_video is deliberately NOT persisted — each value is a base64
video up to ~133 MB, so persisting it would bloat the DB; it stays in memory.
DB lives at bot-state.db (gitignored).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
bot.py:
- Run all yt-dlp/ffmpeg/ffprobe/file-IO/base64 work off the event loop via
asyncio.to_thread, bounded by a Semaphore(2); the loop no longer freezes
bot-wide during downloads/encodes.
- download_attachments=False; VideoTracker fetches only video attachments lazily
instead of the library base64-ing every attachment of every message.
- last_video keyed per (group, sender) with group-latest fallback so /speed and
/rev stop silently targeting a stranger's video; proactive TTL sweeps free the
big base64 blobs and bound recent_urls growth.
- _reencode guards 0/NaN ffprobe duration and adds -maxrate/-bufsize; /rev now
size-checks + re-encodes + faststart like the other paths.
- Twitter URL regex no longer merges space-separated links (keeps i/web/status);
dedupe repeated URLs within a message; mark-handled only on success/no-media so
a corrective edit can retry; 'unsupported url' surfaced instead of silently dropped.
- All sends wrapped (catch SendMessageError); base64 decode guarded; edit/sync
attachment envelopes handled; /cookies temp file created 0600.
deploy:
- Pin signalbot==1.1.0, yt-dlp floor, add missing yt-dlp-ejs.
- Pin signal-cli-rest-api by digest + add /v1/health healthcheck.
- Restart=always (the library never exits non-zero when wedged).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
/clip <seconds> in a link's message overrides the default 60s clip
window (capped at 600s); with a ?t= it sets the window length, without
one it clips from the start. /help lists every command with examples.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When a YouTube link carries a t=/start= offset, download only a 60s
window around it via yt-dlp --download-sections instead of the whole
video. Hour-long uploads shared at a timestamp previously failed the
100 MB Signal limit even after re-encoding.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Group-chat bot that downloads videos from X/Instagram/YouTube/TikTok links
via yt-dlp and posts them back, plus /speed and /rev video toys.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
jamesandClaude Opus 5