3 Commits

Author	SHA1	Message	Date
mike	9d53999eea	Implement password handling for unattended access	2026-05-08 11:34:07 +02:00
mike	4308a2f112	feat: M6 web client QoL — clipboard, multi-monitor, quality, reconnect, H.264, docs ... Builds on the M6a-g MVP (d07e98e) with five user-facing features that take the in-browser remote-control client from "demo toy" to "workable for daily use", plus user-facing documentation. M6h — Text clipboard sync (both directions) - Host → browser via Clipboard{format: Text} → navigator.clipboard.writeText. - Browser → host via Cmd-V/Ctrl-V keydown intercept → navigator.clipboard .readText → Message{clipboard}, sent BEFORE the V keystroke so the host's paste hotkey lands on the freshly-synced text. - Handles both single-format Clipboard (older peers) and MultiClipboards wrapper (peers ≥ 1.3.0; gated by clipboard.rs:is_support_multi_clipboard). - Switched printable-with-modifier hotkeys (Ctrl-C etc.) from Unicode to Chr payload — host's process_unicode does key_sequence(char) which ignores modifiers, breaking copy/cut; process_chr respects them. - Firefox refuses navigator.clipboard.readText() by default — accepted as a known browser limitation, host → browser direction works regardless. M6i — Multi-monitor switching - HUD picker shown when peer_info.displays > 1. - On change: SwitchDisplay + CaptureDisplays{set:[idx]} two-message dance — required for clients ≥ 1.2.4 (we send "1.4.0"). Without the follow- up, switch_display_to leaves both video services subscribed and switching display 0 → 1 → 0 doesn't restore display 0. - Mouse coords offset by the active display's virtual-desktop origin (DisplayInfo.x, .y). Without this, clicks on display 2 landed on display 1 because both share canvas (0,0) but only display 1 has origin (0,0) in virtual-desktop space. M6j — Quality / FPS / mute controls - Image quality preset (Low/Balanced/Best) → Misc{option: {image_quality}}. - Custom FPS (15/30/60) → Misc{option: {custom_fps}}; host caps at 30 unless allow_more_fps is advertised. - Mute toggle additionally sends Misc{option: {disable_audio: Yes/No}} so the peer stops encoding audio while muted (saves CPU + bandwidth). M6k — Auto-reconnect on transient drops - session.recv() throw → reconnect with exponential backoff: 1s, 2s, 4s, 8s, 16s, 30s, 30s, capped at 30s, max 10 attempts. - Dim overlay sits on top of the canvas during retry; canvas keeps last-known frame for visual continuity. - Auth errors (password/signature) bail immediately — no point retrying. - User options (mute, image_quality, custom_fps, current display) re-applied to host on each successful reconnect, since host treats every session as fresh and resets to defaults. - Architecture: `session` is a let-binding mutated on reconnect; HUD button closures read it at click-time so they automatically retarget. Input modules (mouse/keyboard/clipboard) get a Proxy that forwards method calls to whatever session is current — avoids re-binding window/canvas listeners on each reconnect. M6l — H.264 video decode (Annex-B + SPS-derived codec string) - decode/bitstream.ts: iterate Annex-B NAL units, derive avc1.PPCCLL from the keyframe's inline SPS (host's hwcodec defaults to high profile; a hardcoded baseline string would make WebCodecs refuse the stream). - Defer H.264 decoder configure until first keyframe arrives. - VP9 codec string corrected from level 1.0 (vp09.00.10.08) to level 5.0 (vp09.00.50.08) — wrong level was probably forcing software decode in some browsers. - Default prefer flipped to VP8 (cheapest software encoder; H.264 path stays implemented for hosts with hwcodec/nvenc). M6m — docs/CONFIGURATION.md "Web client" section: routes, browser matrix, network requirements (relay reachability + reverse-proxy WS upgrade), feature status table, codec selection rationale, the recv/dec/draw HUD diagnostic, build commands. Bundle: 535 KB / ~75 KB gzipped. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-03 17:43:23 +02:00
mike	d07e98e607	feat: M6 web client — view + control + audio in the dashboard ... Adds a TypeScript SPA embedded in hbbs that lets a logged-in admin click "Connect (web client)" on a Devices row and remote-control the peer from the browser, no desktop client install required. View, mouse/keyboard control, and host-audio playback all work end-to-end. Architecture ------------ Pure browser app, no server-side WS proxy: Browser ──ws://hbbs:21118── rendezvous (PunchHole + RequestRelay) │ └──ws://hbbr:21119──── relay (paired by uuid) │ └── peer (RustDesk desktop, any platform) Same wire path the desktop client takes via `Client::request_relay` and `Client::create_relay`. Browser-relay-only — no NAT punching, so we send nat_type=SYMMETRIC in PunchHoleRequest to make the peer skip the direct attempt and go straight to relay (initiate=true with a host-generated uuid that we then use for our own relay leg). The 5 wire steps: 1. PunchHoleRequest → harvests signed peer Ed25519 sign-pk + relay host + the peer's session uuid from RelayResponse 2. Verify the signed pk against /admin/connect's `id_ed25519.pub` 3. Open WS to hbbr:21119, send RequestRelay with that uuid 4. Read peer's Message{signed_id}, verify with peer_sign_pk, extract Curve25519 box pk; box-seal a fresh secretbox key under it; send Message{public_key} unencrypted 5. Secretbox-encrypted stream from here. Hash → LoginRequest → LoginResponse with PeerInfo. Mode = Legacy (Translate silently drops ControlKey/Unicode payloads on the host side). New Rust surface ---------------- - `admin_ui/connect.html` — SPA shell with `{{CUSTOM_CONFIG}}` placeholder - `src/api/admin/pages/connect.rs` — gates on AuthedUser, injects per-request config (rendezvous host, relay host, server pubkey, peer_id, admin name) into the `<script id="custom-config">` tag, serves bundle.{js,css} via include_bytes! - 3 routes added: GET /admin/connect/:peer_id and the two assets - Devices dropdown gains a sky-blue "Connect (web client)" link that opens in a new tab New TypeScript SPA (`web_client/`) ---------------------------------- Stack: pure DOM/TS, no React/Vue. Bundled by esbuild → `dist/bundle.{js,css}` which is committed (cargo build needs no Node toolchain). src/main.ts boot + password retry loop + receive dispatch src/crypto.ts tweetnacl wrapper (sign_open, box, secretbox) + @noble/hashes/sha2 (works on plain http; SubtleCrypto requires a secure context) src/proto/generated.{js,d.ts} pbjs static-module from libs/hbb_common/protos/{rendezvous,message}.proto src/transport/rendezvous.ts WS to hbbs; PunchHole + RequestRelay src/transport/relay.ts WS to hbbr; duplex frame transport src/transport/session.ts secure-handshake state machine + Hash/Login + 8-byte LE secretbox sequence counter (PRE-increment, send/recv independent — matches libs/hbb_common/src/tcp.rs:317-320) + preloginExtras stash for AudioFormat that arrives before LoginResponse src/decode/video.ts WebCodecs VideoDecoder (vp09.00.10.08 today; h264/h265/av1/vp8 codec strings ready for M6f) src/decode/audio.ts WebCodecs AudioDecoder (opus) → AudioContext; detects f32 vs f32-planar AudioData layout and deinterleaves when needed; gap-less scheduling via a sliding playhead src/ui/canvas.ts <canvas> with object-fit: contain letterbox; auto-resizes on resolution change; FPS counter src/input/mouse.ts MouseEvent → MouseEvent proto. Mask layout: (button << 3) | type (0=move,1=down,2=up, 3=wheel). Letterbox-aware viewport→peer coord mapping. Right-click suppresses the browser context menu; left-click does NOT preventDefault (would block focus) src/input/keyboard.ts Window-level keydown/keyup → KeyEvent proto in Legacy mode. Special keys → ControlKey enum; printable → unicode codepoint (down only, host's process_unicode does a single key_click). Browser shortcuts allowlisted (Cmd-T/N/W/R, Tab) so the user keeps tab control. Ctrl+Alt+Del HUD button (host-side `send_sas` is `#[cfg(windows)]`; no-op on Mac/Linux hosts but present for parity) Bundle size: 529 KB raw / ~74 KB gzipped. Tree-shaken protobufjs + tweetnacl + @noble/hashes only. Deployment notes ---------------- - WebCodecs and SubtleCrypto are gated to "secure context" origins — HTTPS, or http://localhost. Plain http://lan-ip won't work. Open via http://localhost during dev, or terminate TLS in front of hbbs (Caddy / nginx / Traefik) for production access. - `--relay-servers <host>` on hbbs must point at a host where TCP/WS 21119 is reachable from end-user browsers. Wire-format gotchas this commit nails (each one was a session of bisecting) -------------------------------------------------------------------------- - Hash.salt / Hash.challenge are proto `string` fields used as raw UTF-8 bytes in the SHA-256 chain. NOT base64-decoded. `pwd_hash = SHA256(pwd_text || salt_utf8)`, `resp = SHA256(pwd_hash || challenge_utf8)`. - Translate keyboard mode silently drops Unicode + ControlKey payloads on the host (input_service.rs:2022 has `// Do not handle unicode for now.`). Only Seq + Chr work in Translate. Use Legacy (mode=0) for everything. - Browser is forced to relay path by sending nat_type=SYMMETRIC. The peer generates its OWN uuid in handle_punch_hole's symmetric branch; use that uuid (carried back in RelayResponse) for the relay leg, not a fresh one. - Misc{audio_format} fires from the host's audio_service first-snapshot BETWEEN add_connection and login_response, so it lands on the wire before our session.recv() loop is set up. Session.open() captures pre-login messages into preloginExtras for the caller to replay. - protobufjs static-module sets unpopulated oneof fields to JS `null`, not `undefined`. A `if (msg.cursor_id !== undefined)` cursor branch swallowed every other message type including Misc; switched to loose `!= null` comparison. - WebCodecs AudioDecoder for opus emits `f32` (interleaved) AudioData — must deinterleave into separate AudioBuffer channels before playback. - VideoDecoder/AudioDecoder/SubtleCrypto are SecureContext-only; need http://localhost or https:// on the *page origin*, not the WS targets. - libsodium-wrappers ESM ships a broken relative import (`./libsodium.mjs` in a sibling package); switched to tweetnacl which has no such problem. - WebCrypto's SubtleCrypto.digest() doesn't accept SharedArrayBuffer-backed Uint8Arrays in newer TS lib types; doesn't matter — we use @noble/hashes for sha256 anyway since Subtle is secure-context-only. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-03 13:55:40 +02:00