From 4308a2f112e92b54e9370886cb39c815aa6fd212 Mon Sep 17 00:00:00 2001 From: Mike Mueller Date: Sun, 3 May 2026 17:43:23 +0200 Subject: [PATCH] =?UTF-8?q?feat:=20M6=20web=20client=20QoL=20=E2=80=94=20c?= =?UTF-8?q?lipboard,=20multi-monitor,=20quality,=20reconnect,=20H.264,=20d?= =?UTF-8?q?ocs?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Builds on the M6a-g MVP (d07e98e) with five user-facing features that take the in-browser remote-control client from "demo toy" to "workable for daily use", plus user-facing documentation. M6h — Text clipboard sync (both directions) - Host → browser via Clipboard{format: Text} → navigator.clipboard.writeText. - Browser → host via Cmd-V/Ctrl-V keydown intercept → navigator.clipboard .readText → Message{clipboard}, sent BEFORE the V keystroke so the host's paste hotkey lands on the freshly-synced text. - Handles both single-format Clipboard (older peers) and MultiClipboards wrapper (peers ≥ 1.3.0; gated by clipboard.rs:is_support_multi_clipboard). - Switched printable-with-modifier hotkeys (Ctrl-C etc.) from Unicode to Chr payload — host's process_unicode does key_sequence(char) which ignores modifiers, breaking copy/cut; process_chr respects them. - Firefox refuses navigator.clipboard.readText() by default — accepted as a known browser limitation, host → browser direction works regardless. M6i — Multi-monitor switching - HUD picker shown when peer_info.displays > 1. - On change: SwitchDisplay + CaptureDisplays{set:[idx]} two-message dance — required for clients ≥ 1.2.4 (we send "1.4.0"). Without the follow- up, switch_display_to leaves both video services subscribed and switching display 0 → 1 → 0 doesn't restore display 0. - Mouse coords offset by the active display's virtual-desktop origin (DisplayInfo.x, .y). Without this, clicks on display 2 landed on display 1 because both share canvas (0,0) but only display 1 has origin (0,0) in virtual-desktop space. M6j — Quality / FPS / mute controls - Image quality preset (Low/Balanced/Best) → Misc{option: {image_quality}}. - Custom FPS (15/30/60) → Misc{option: {custom_fps}}; host caps at 30 unless allow_more_fps is advertised. - Mute toggle additionally sends Misc{option: {disable_audio: Yes/No}} so the peer stops encoding audio while muted (saves CPU + bandwidth). M6k — Auto-reconnect on transient drops - session.recv() throw → reconnect with exponential backoff: 1s, 2s, 4s, 8s, 16s, 30s, 30s, capped at 30s, max 10 attempts. - Dim overlay sits on top of the canvas during retry; canvas keeps last-known frame for visual continuity. - Auth errors (password/signature) bail immediately — no point retrying. - User options (mute, image_quality, custom_fps, current display) re-applied to host on each successful reconnect, since host treats every session as fresh and resets to defaults. - Architecture: `session` is a let-binding mutated on reconnect; HUD button closures read it at click-time so they automatically retarget. Input modules (mouse/keyboard/clipboard) get a Proxy that forwards method calls to whatever session is current — avoids re-binding window/canvas listeners on each reconnect. M6l — H.264 video decode (Annex-B + SPS-derived codec string) - decode/bitstream.ts: iterate Annex-B NAL units, derive avc1.PPCCLL from the keyframe's inline SPS (host's hwcodec defaults to high profile; a hardcoded baseline string would make WebCodecs refuse the stream). - Defer H.264 decoder configure until first keyframe arrives. - VP9 codec string corrected from level 1.0 (vp09.00.10.08) to level 5.0 (vp09.00.50.08) — wrong level was probably forcing software decode in some browsers. - Default prefer flipped to VP8 (cheapest software encoder; H.264 path stays implemented for hosts with hwcodec/nvenc). M6m — docs/CONFIGURATION.md "Web client" section: routes, browser matrix, network requirements (relay reachability + reverse-proxy WS upgrade), feature status table, codec selection rationale, the recv/dec/draw HUD diagnostic, build commands. Bundle: 535 KB / ~75 KB gzipped. Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/CONFIGURATION.md | 113 +++++++ web_client/dist/bundle.css | 28 ++ web_client/dist/bundle.js | 6 +- web_client/dist/bundle.js.map | 8 +- web_client/src/decode/bitstream.ts | 67 ++++ web_client/src/decode/video.ts | 136 ++++++-- web_client/src/input/clipboard.ts | 99 ++++++ web_client/src/input/keyboard.ts | 28 +- web_client/src/input/mouse.ts | 28 +- web_client/src/main.ts | 479 +++++++++++++++++++++++----- web_client/src/transport/session.ts | 11 +- web_client/src/ui/style.css | 28 ++ 12 files changed, 893 insertions(+), 138 deletions(-) create mode 100644 web_client/src/decode/bitstream.ts create mode 100644 web_client/src/input/clipboard.ts diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md index 41e9700..fa7b456 100644 --- a/docs/CONFIGURATION.md +++ b/docs/CONFIGURATION.md @@ -317,6 +317,119 @@ for the dashboard with no separate session model. --- +## Web client + +Admins can remote-control any peer directly from the browser — no desktop +RustDesk install required. The Devices page row dropdown surfaces a +**Connect** button that opens `/admin/connect/` in a new tab. +The page is a TypeScript SPA bundled into hbbs (`include_bytes!` from +`web_client/dist/`), so there's no separate process or service to run. + +### Routes + +| Route | Auth | Purpose | +|---|---|---| +| `/admin/connect/:peer_id` | cookie + admin | Server-rendered HTML wrapper that injects per-request config (rendezvous host, relay host, server pk, peer id, admin display name) into a `