feat: M6 web client QoL — clipboard, multi-monitor, quality, reconnect, H.264, docs
Builds on the M6a-g MVP (d07e98e) with five user-facing features that
take the in-browser remote-control client from "demo toy" to "workable
for daily use", plus user-facing documentation.
M6h — Text clipboard sync (both directions)
- Host → browser via Clipboard{format: Text} → navigator.clipboard.writeText.
- Browser → host via Cmd-V/Ctrl-V keydown intercept → navigator.clipboard
.readText → Message{clipboard}, sent BEFORE the V keystroke so the
host's paste hotkey lands on the freshly-synced text.
- Handles both single-format Clipboard (older peers) and MultiClipboards
wrapper (peers ≥ 1.3.0; gated by clipboard.rs:is_support_multi_clipboard).
- Switched printable-with-modifier hotkeys (Ctrl-C etc.) from Unicode to
Chr payload — host's process_unicode does key_sequence(char) which
ignores modifiers, breaking copy/cut; process_chr respects them.
- Firefox refuses navigator.clipboard.readText() by default — accepted as
a known browser limitation, host → browser direction works regardless.
M6i — Multi-monitor switching
- HUD picker shown when peer_info.displays > 1.
- On change: SwitchDisplay + CaptureDisplays{set:[idx]} two-message dance
— required for clients ≥ 1.2.4 (we send "1.4.0"). Without the follow-
up, switch_display_to leaves both video services subscribed and
switching display 0 → 1 → 0 doesn't restore display 0.
- Mouse coords offset by the active display's virtual-desktop origin
(DisplayInfo.x, .y). Without this, clicks on display 2 landed on
display 1 because both share canvas (0,0) but only display 1 has
origin (0,0) in virtual-desktop space.
M6j — Quality / FPS / mute controls
- Image quality preset (Low/Balanced/Best) → Misc{option: {image_quality}}.
- Custom FPS (15/30/60) → Misc{option: {custom_fps}}; host caps at 30
unless allow_more_fps is advertised.
- Mute toggle additionally sends Misc{option: {disable_audio: Yes/No}}
so the peer stops encoding audio while muted (saves CPU + bandwidth).
M6k — Auto-reconnect on transient drops
- session.recv() throw → reconnect with exponential backoff: 1s, 2s, 4s,
8s, 16s, 30s, 30s, capped at 30s, max 10 attempts.
- Dim overlay sits on top of the canvas during retry; canvas keeps
last-known frame for visual continuity.
- Auth errors (password/signature) bail immediately — no point retrying.
- User options (mute, image_quality, custom_fps, current display)
re-applied to host on each successful reconnect, since host treats
every session as fresh and resets to defaults.
- Architecture: `session` is a let-binding mutated on reconnect; HUD
button closures read it at click-time so they automatically retarget.
Input modules (mouse/keyboard/clipboard) get a Proxy that forwards
method calls to whatever session is current — avoids re-binding
window/canvas listeners on each reconnect.
M6l — H.264 video decode (Annex-B + SPS-derived codec string)
- decode/bitstream.ts: iterate Annex-B NAL units, derive avc1.PPCCLL
from the keyframe's inline SPS (host's hwcodec defaults to high
profile; a hardcoded baseline string would make WebCodecs refuse the
stream).
- Defer H.264 decoder configure until first keyframe arrives.
- VP9 codec string corrected from level 1.0 (vp09.00.10.08) to level
5.0 (vp09.00.50.08) — wrong level was probably forcing software
decode in some browsers.
- Default prefer flipped to VP8 (cheapest software encoder; H.264 path
stays implemented for hosts with hwcodec/nvenc).
M6m — docs/CONFIGURATION.md "Web client" section: routes, browser
matrix, network requirements (relay reachability + reverse-proxy WS
upgrade), feature status table, codec selection rationale, the
recv/dec/draw HUD diagnostic, build commands.
Bundle: 535 KB / ~75 KB gzipped.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Vendored
+28
@@ -142,6 +142,34 @@ html, body {
|
||||
}
|
||||
.hud-btn:hover { background: #475569; }
|
||||
|
||||
.hud-select {
|
||||
background: #334155;
|
||||
border: 0;
|
||||
color: #e2e8f0;
|
||||
padding: 2px 4px;
|
||||
border-radius: 3px;
|
||||
font-size: 11px;
|
||||
font-family: inherit;
|
||||
cursor: pointer;
|
||||
}
|
||||
.hud-select:hover { background: #475569; }
|
||||
|
||||
.reconnect-overlay {
|
||||
position: fixed;
|
||||
inset: 0;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
background: rgba(15, 23, 42, 0.65);
|
||||
backdrop-filter: blur(2px);
|
||||
-webkit-backdrop-filter: blur(2px);
|
||||
color: #e2e8f0;
|
||||
font-family: inherit;
|
||||
font-size: 15px;
|
||||
z-index: 20;
|
||||
pointer-events: none;
|
||||
}
|
||||
|
||||
.error {
|
||||
background: rgba(220, 38, 38, 0.15);
|
||||
border: 1px solid rgba(220, 38, 38, 0.4);
|
||||
|
||||
Vendored
+3
-3
File diff suppressed because one or more lines are too long
Vendored
+4
-4
File diff suppressed because one or more lines are too long
@@ -0,0 +1,67 @@
|
||||
// Annex-B helpers for H.264 / H.265 video framing.
|
||||
//
|
||||
// The host encodes H.264/H.265 via ffmpeg (hwcodec crate) and the default
|
||||
// muxer-less output is Annex-B: NAL units separated by 3- or 4-byte start
|
||||
// codes (`00 00 01` or `00 00 00 01`). SPS/PPS are inlined before each IDR.
|
||||
//
|
||||
// WebCodecs' VideoDecoder accepts Annex-B chunks directly when configured
|
||||
// without a `description` field — in that mode the browser scans for start
|
||||
// codes itself. We just need to:
|
||||
// 1. Find the first SPS NAL inside a keyframe to derive the avc1/hvc1
|
||||
// codec string the decoder requires.
|
||||
// 2. Pass each frame's bytes into the decoder unmodified.
|
||||
|
||||
const NAL_UNIT_TYPE_MASK_H264 = 0x1f;
|
||||
const SPS_NAL_TYPE_H264 = 7;
|
||||
|
||||
/** Iterate NAL units in an Annex-B stream. Yields `payload` excluding the
|
||||
* start code; first byte is the NAL header. */
|
||||
export function* iterAnnexBNalu(data: Uint8Array): Generator<Uint8Array> {
|
||||
// Locate every start code (00 00 01 or 00 00 00 01), then yield the
|
||||
// bytes between consecutive start codes.
|
||||
const starts: number[] = [];
|
||||
for (let i = 0; i + 2 < data.length; i++) {
|
||||
if (data[i] === 0 && data[i + 1] === 0) {
|
||||
if (data[i + 2] === 1) {
|
||||
starts.push(i + 3);
|
||||
i += 2;
|
||||
} else if (i + 3 < data.length && data[i + 2] === 0 && data[i + 3] === 1) {
|
||||
starts.push(i + 4);
|
||||
i += 3;
|
||||
}
|
||||
}
|
||||
}
|
||||
for (let k = 0; k < starts.length; k++) {
|
||||
const start = starts[k]!;
|
||||
const end = k + 1 < starts.length
|
||||
? starts[k + 1]! - (data[starts[k + 1]! - 4] === 0 ? 4 : 3)
|
||||
: data.length;
|
||||
if (end > start) {
|
||||
yield data.subarray(start, end);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Build an `avc1.PPCCLL` codec string from a keyframe's SPS NAL unit.
|
||||
*
|
||||
* SPS layout (after NAL header byte):
|
||||
* byte 0: profile_idc
|
||||
* byte 1: constraint_set_flags (low 6 bits) | reserved_zero_2bits
|
||||
* byte 2: level_idc
|
||||
*
|
||||
* Returns null if no SPS found in the buffer.
|
||||
*/
|
||||
export function deriveH264CodecString(keyframe: Uint8Array): string | null {
|
||||
for (const nalu of iterAnnexBNalu(keyframe)) {
|
||||
if (nalu.length < 4) continue;
|
||||
const naluType = nalu[0]! & NAL_UNIT_TYPE_MASK_H264;
|
||||
if (naluType !== SPS_NAL_TYPE_H264) continue;
|
||||
const profile = nalu[1]!;
|
||||
const constraints = nalu[2]!;
|
||||
const level = nalu[3]!;
|
||||
const hex = (n: number): string => n.toString(16).padStart(2, "0").toUpperCase();
|
||||
return `avc1.${hex(profile)}${hex(constraints)}${hex(level)}`;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
+106
-30
@@ -1,15 +1,19 @@
|
||||
// Video decode pipeline using WebCodecs VideoDecoder.
|
||||
//
|
||||
// M6d covers VP9 (the host's default codec — see the desktop client's
|
||||
// VpxEncoderConfig). VP9 frame data on the wire is the raw VP9 bitstream;
|
||||
// WebCodecs' VideoDecoder accepts each frame as an EncodedVideoChunk with
|
||||
// type "key" or "delta" derived from the EncodedVideoFrame.key flag.
|
||||
// Wire format per codec (all from the host's hwcodec/vpx encoders):
|
||||
// - VP9 / VP8 / AV1: raw bitstream per frame, no container framing.
|
||||
// - H.264 / H.265: Annex-B (start-code-prefixed NAL units) with inline
|
||||
// SPS/PPS at every IDR. WebCodecs accepts this mode
|
||||
// when `description` is omitted from the configure().
|
||||
//
|
||||
// The decoder runs asynchronously: we push chunks in via `decode()`, the
|
||||
// `output` callback fires later with a VideoFrame. We close each frame
|
||||
// immediately after drawing to release the GPU texture.
|
||||
// Codec string for H.264 must encode the actual profile/level the stream
|
||||
// uses (e.g. `avc1.640028` for high@4.0). Hardcoding `avc1.42E01E` (the
|
||||
// baseline default) means the browser refuses streams encoded by nvenc
|
||||
// or qsv at high profile. We parse the SPS on the first keyframe and
|
||||
// build the right string from profile_idc/constraint_set_flags/level_idc.
|
||||
|
||||
import { hbb } from "../proto/generated.js";
|
||||
import { deriveH264CodecString } from "./bitstream.js";
|
||||
|
||||
export type FrameSink = (frame: VideoFrame) => void;
|
||||
|
||||
@@ -17,6 +21,12 @@ export class VideoPipeline {
|
||||
private decoder: VideoDecoder | null = null;
|
||||
private currentCodec = "";
|
||||
private onFrame: FrameSink;
|
||||
// Diagnostic counters — log per-second so we can see whether a low FPS
|
||||
// is host-encode-bound (low frames-in) or browser-decode-bound (high
|
||||
// frames-in, low frames-out). Cleared each tick.
|
||||
private framesIn = 0;
|
||||
private framesOut = 0;
|
||||
private lastDiagAt = performance.now();
|
||||
|
||||
constructor(onFrame: FrameSink) {
|
||||
this.onFrame = onFrame;
|
||||
@@ -29,55 +39,116 @@ export class VideoPipeline {
|
||||
}
|
||||
}
|
||||
|
||||
/** Snapshot of receive/decode rates (frames per second over the last
|
||||
* second). Used by the HUD diag line. */
|
||||
diagStats(): { recvFps: number; decodeFps: number; codec: string } {
|
||||
const now = performance.now();
|
||||
const dt = (now - this.lastDiagAt) / 1000 || 1;
|
||||
const recvFps = Math.round(this.framesIn / dt);
|
||||
const decodeFps = Math.round(this.framesOut / dt);
|
||||
this.framesIn = 0;
|
||||
this.framesOut = 0;
|
||||
this.lastDiagAt = now;
|
||||
return { recvFps, decodeFps, codec: this.currentCodec };
|
||||
}
|
||||
|
||||
/** Push one EncodedVideoFrames container (oneof from a VideoFrame Message). */
|
||||
pushVideoFrame(vf: hbb.IVideoFrame): void {
|
||||
const frames = vf.vp9s?.frames || vf.vp8s?.frames || vf.av1s?.frames || vf.h264s?.frames || vf.h265s?.frames;
|
||||
const codec = this.detectCodec(vf);
|
||||
if (!frames || frames.length === 0 || !codec) {
|
||||
return;
|
||||
}
|
||||
if (codec !== this.currentCodec) {
|
||||
this.configureCodec(codec);
|
||||
}
|
||||
const family = this.detectFamily(vf);
|
||||
if (!family) return;
|
||||
const frames =
|
||||
vf.vp9s?.frames || vf.vp8s?.frames || vf.av1s?.frames ||
|
||||
vf.h264s?.frames || vf.h265s?.frames;
|
||||
if (!frames || frames.length === 0) return;
|
||||
|
||||
for (const f of frames) {
|
||||
this.decode(f, codec);
|
||||
this.handleFrame(family, f);
|
||||
}
|
||||
}
|
||||
|
||||
private detectCodec(vf: hbb.IVideoFrame): string {
|
||||
if (vf.vp9s) return "vp09.00.10.08"; // VP9 profile 0, 8-bit
|
||||
private detectFamily(vf: hbb.IVideoFrame): "vp9" | "vp8" | "av1" | "h264" | "h265" | "" {
|
||||
if (vf.vp9s) return "vp9";
|
||||
if (vf.vp8s) return "vp8";
|
||||
if (vf.av1s) return "av01.0.04M.08"; // AV1 main profile, level 4.0, 8-bit
|
||||
if (vf.h264s) return "avc1.42E01E"; // H.264 baseline (will need SPS-driven config in M6f)
|
||||
if (vf.h265s) return "hvc1.1.6.L93.B0";
|
||||
if (vf.av1s) return "av1";
|
||||
if (vf.h264s) return "h264";
|
||||
if (vf.h265s) return "h265";
|
||||
return "";
|
||||
}
|
||||
|
||||
private configureCodec(codec: string): void {
|
||||
/** Build the WebCodecs codec string for this frame's stream. For VP9
|
||||
* we use level 5.0 (covers up to 4K@30) so the browser's hardware
|
||||
* path engages — `vp09.00.10.08` (level 1.0) had been telling Chrome
|
||||
* the stream was tiny, which can force the software fallback. */
|
||||
private codecStringFor(family: string, frameData?: Uint8Array): string | null {
|
||||
switch (family) {
|
||||
case "vp9": return "vp09.00.50.08";
|
||||
case "vp8": return "vp8";
|
||||
case "av1": return "av01.0.04M.08";
|
||||
case "h264": return frameData ? deriveH264CodecString(frameData) : null;
|
||||
case "h265": return "hvc1.1.6.L93.B0"; // best-effort; SPS parsing for HEVC is more involved
|
||||
default: return null;
|
||||
}
|
||||
}
|
||||
|
||||
private handleFrame(
|
||||
family: "vp9" | "vp8" | "av1" | "h264" | "h265",
|
||||
f: hbb.IEncodedVideoFrame,
|
||||
): void {
|
||||
if (!f.data || f.data.length === 0) return;
|
||||
const data = f.data as Uint8Array;
|
||||
const isKey = !!f.key;
|
||||
this.framesIn++;
|
||||
|
||||
// For H.264 we MUST see an SPS before we can configure the decoder
|
||||
// (the codec string depends on its profile/level bytes). The host
|
||||
// sends SPS+PPS inline on every keyframe, so the first keyframe is
|
||||
// sufficient. Drop deltas that arrive before the first key.
|
||||
if (family === "h264" && !this.decoder) {
|
||||
if (!isKey) return;
|
||||
const codec = this.codecStringFor("h264", data);
|
||||
if (!codec) {
|
||||
console.warn("[rustdesk-web] H.264 keyframe missing SPS — dropping");
|
||||
return;
|
||||
}
|
||||
this.configureDecoder(codec);
|
||||
} else if (family !== "h264" && (!this.decoder || this.currentCodec !== this.codecStringFor(family))) {
|
||||
const codec = this.codecStringFor(family);
|
||||
if (!codec) return;
|
||||
this.configureDecoder(codec);
|
||||
}
|
||||
|
||||
this.decode(data, isKey, f.pts);
|
||||
}
|
||||
|
||||
private configureDecoder(codec: string): void {
|
||||
if (this.decoder) {
|
||||
this.decoder.close();
|
||||
try { this.decoder.close(); } catch { /* ignore */ }
|
||||
}
|
||||
this.currentCodec = codec;
|
||||
this.decoder = new VideoDecoder({
|
||||
output: (frame) => this.onFrame(frame),
|
||||
output: (frame) => {
|
||||
this.framesOut++;
|
||||
this.onFrame(frame);
|
||||
},
|
||||
error: (e) => {
|
||||
console.error("[rustdesk-web] VideoDecoder error:", e);
|
||||
},
|
||||
});
|
||||
// optimizeForLatency hints the decoder to skip frame reordering
|
||||
// buffers — important for screen sharing where each frame matters
|
||||
// and we don't have B-frames to reorder anyway.
|
||||
this.decoder.configure({
|
||||
codec,
|
||||
// optimizeForLatency hints the decoder to skip frame reordering buffers.
|
||||
optimizeForLatency: true,
|
||||
});
|
||||
}
|
||||
|
||||
private decode(f: hbb.IEncodedVideoFrame, _codec: string): void {
|
||||
if (!this.decoder || !f.data || f.data.length === 0) return;
|
||||
const data = f.data as Uint8Array;
|
||||
const pts = typeof f.pts === "number" ? f.pts : Number(f.pts || 0);
|
||||
private decode(data: Uint8Array, isKey: boolean, ptsField: number | Long | null | undefined): void {
|
||||
if (!this.decoder) return;
|
||||
const pts = typeof ptsField === "number" ? ptsField : Number(ptsField || 0);
|
||||
try {
|
||||
const chunk = new EncodedVideoChunk({
|
||||
type: f.key ? "key" : "delta",
|
||||
type: isKey ? "key" : "delta",
|
||||
timestamp: pts * 1000, // ms → microseconds
|
||||
data,
|
||||
});
|
||||
@@ -94,3 +165,8 @@ export class VideoPipeline {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// protobufjs static-module emits `pts: number | Long | null` for int64
|
||||
// fields. We don't import the Long shim, so use a structural type for the
|
||||
// signature.
|
||||
type Long = { low: number; high: number; unsigned: boolean; toNumber(): number };
|
||||
|
||||
@@ -0,0 +1,99 @@
|
||||
// Text clipboard sync, both directions.
|
||||
//
|
||||
// Host -> browser: incoming Message{clipboard: {format: Text, content}}
|
||||
// → navigator.clipboard.writeText (Async Clipboard API).
|
||||
// Browsers gate this on a "user gesture" — silently
|
||||
// fails the first time until the user has clicked
|
||||
// somewhere on the page. We swallow the rejection.
|
||||
//
|
||||
// Browser -> host: global `paste` event → read clipboardData → send
|
||||
// Message{clipboard: {format: Text, content: utf8 bytes,
|
||||
// compress: false}}.
|
||||
//
|
||||
// Image / RTF / HTML formats are deferred to a follow-up; only Text in v1.
|
||||
// The wire shape is documented at libs/hbb_common/protos/message.proto:
|
||||
// ClipboardFormat: Text=0 / Rtf=1 / Html=2 / Image*=21..23 / Special=31
|
||||
|
||||
import { hbb } from "../proto/generated.js";
|
||||
import type { Session } from "../transport/session.js";
|
||||
|
||||
/** Track the last value we wrote so we don't echo our own paste back to the host. */
|
||||
let lastWrittenText = "";
|
||||
|
||||
/** Receive an incoming Clipboard (single, format-typed) and apply it locally. */
|
||||
export async function applyIncoming(c: hbb.IClipboard): Promise<void> {
|
||||
if (c.format !== hbb.ClipboardFormat.Text) return;
|
||||
if (!c.content) return;
|
||||
const text = new TextDecoder().decode(c.content as Uint8Array);
|
||||
if (!text) return;
|
||||
try {
|
||||
await navigator.clipboard.writeText(text);
|
||||
lastWrittenText = text;
|
||||
} catch {
|
||||
// Browsers refuse writeText() without a recent user gesture. The
|
||||
// user can press any key once to "grant" gesture state. Until then
|
||||
// we silently drop. Visible failure UX is more annoying than silent.
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Receive an incoming MultiClipboards (newer wire format used by RustDesk
|
||||
* peers >= 1.3.0 against non-iOS controllers — see
|
||||
* /Users/sn0/Desktop/rustdesk/src/clipboard.rs:is_support_multi_clipboard).
|
||||
* Find the first Text-format entry and apply it.
|
||||
*/
|
||||
export async function applyIncomingMulti(mc: hbb.IMultiClipboards): Promise<void> {
|
||||
const list = mc.clipboards || [];
|
||||
for (const c of list) {
|
||||
if (c.format === hbb.ClipboardFormat.Text) {
|
||||
await applyIncoming(c);
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Read the browser's local clipboard text and send it to the host as
|
||||
* Message{clipboard: {format: Text, content: utf8 bytes}}. Used by the
|
||||
* keyboard handler on Cmd+V / Ctrl+V — `paste` events don't reliably fire
|
||||
* on `window` in Firefox without a focused input element, so we drive
|
||||
* clipboard sync from the keystroke instead.
|
||||
*/
|
||||
export async function pushClipboardText(session: Session): Promise<void> {
|
||||
let text = "";
|
||||
try {
|
||||
text = await navigator.clipboard.readText();
|
||||
} catch {
|
||||
// User hasn't granted clipboard permission, or no recent gesture.
|
||||
return;
|
||||
}
|
||||
if (!text) return;
|
||||
if (text === lastWrittenText) return; // host pushed this to us already; no echo
|
||||
const content = new TextEncoder().encode(text);
|
||||
await session.send(hbb.Message.create({
|
||||
clipboard: hbb.Clipboard.create({
|
||||
compress: false,
|
||||
format: hbb.ClipboardFormat.Text,
|
||||
content,
|
||||
}),
|
||||
})).catch(() => { /* relay closed */ });
|
||||
}
|
||||
|
||||
/** Defensive: also hook the standard `paste` event in case the browser
|
||||
* fires it (Chrome does, Firefox often doesn't). */
|
||||
export function bindClipboard(session: Session): () => void {
|
||||
const onPaste = (e: ClipboardEvent): void => {
|
||||
const text = e.clipboardData?.getData("text/plain");
|
||||
if (!text || text === lastWrittenText) return;
|
||||
const content = new TextEncoder().encode(text);
|
||||
session.send(hbb.Message.create({
|
||||
clipboard: hbb.Clipboard.create({
|
||||
compress: false,
|
||||
format: hbb.ClipboardFormat.Text,
|
||||
content,
|
||||
}),
|
||||
})).catch(() => { /* relay closed */ });
|
||||
};
|
||||
window.addEventListener("paste", onPaste);
|
||||
return () => window.removeEventListener("paste", onPaste);
|
||||
}
|
||||
@@ -20,6 +20,7 @@
|
||||
import { hbb } from "../proto/generated.js";
|
||||
import type { Session } from "../transport/session.js";
|
||||
import type { CanvasView } from "../ui/canvas.js";
|
||||
import { pushClipboardText } from "./clipboard.js";
|
||||
|
||||
/**
|
||||
* KeyboardEvent.code → ControlKey enum value. Covers the named keys; any
|
||||
@@ -106,15 +107,25 @@ export function bindKeyboard(_canvas: CanvasView, session: Session): () => void
|
||||
}
|
||||
|
||||
const ck = CODE_TO_CONTROL[e.code];
|
||||
const hotkeyModifier = e.ctrlKey || e.altKey || e.metaKey;
|
||||
let payload: hbb.IKeyEvent;
|
||||
if (ck !== undefined) {
|
||||
// Special key — fire on both keydown and keyup.
|
||||
payload = { control_key: ck, down };
|
||||
} else if (e.key.length === 1) {
|
||||
// Printable char. Host's process_unicode does a single key_click,
|
||||
// so we only send on keydown — keyup would double-type.
|
||||
if (!down) return;
|
||||
payload = { unicode: e.key.codePointAt(0)!, down: true };
|
||||
const codepoint = e.key.codePointAt(0)!;
|
||||
if (hotkeyModifier) {
|
||||
// Printable + Ctrl/Alt/Cmd → must use Chr (scancode-style) so the
|
||||
// host's process_chr respects the held modifiers. Unicode payloads
|
||||
// go through process_unicode which ignores modifiers (just types
|
||||
// the char), breaking Ctrl+C, Ctrl+V, etc. Fire on both edges.
|
||||
payload = { chr: codepoint, down };
|
||||
} else {
|
||||
// Plain typing — Unicode keydown only (host does a single
|
||||
// key_click; a keyup would double-type the char).
|
||||
if (!down) return;
|
||||
payload = { unicode: codepoint, down: true };
|
||||
}
|
||||
} else {
|
||||
// Unmapped key — silently drop. Rare in normal use.
|
||||
return;
|
||||
@@ -122,6 +133,15 @@ export function bindKeyboard(_canvas: CanvasView, session: Session): () => void
|
||||
payload.modifiers = modifierList(e);
|
||||
payload.mode = hbb.KeyboardMode.Legacy;
|
||||
|
||||
// Cmd+V / Ctrl+V → read the local clipboard and push it to the host
|
||||
// BEFORE the V keystroke so the host's paste hotkey lands on the
|
||||
// freshly-synced text. Non-blocking — the keystroke goes through
|
||||
// either way; if the clipboard read fails (permission, empty), the
|
||||
// host pastes whatever it had.
|
||||
if (down && hotkeyModifier && e.code === "KeyV") {
|
||||
pushClipboardText(session).catch(() => { /* swallowed */ });
|
||||
}
|
||||
|
||||
session.send(hbb.Message.create({ key_event: hbb.KeyEvent.create(payload) }))
|
||||
.catch(() => { /* relay closed */ });
|
||||
|
||||
|
||||
@@ -8,9 +8,13 @@
|
||||
// 0x08=back, 0x10=forward
|
||||
//
|
||||
// Coordinate translation:
|
||||
// The host expects coordinates in its own display pixel space. We map
|
||||
// browser canvas-rect coordinates → peer display coordinates by scaling
|
||||
// against the displayed CanvasView.size().
|
||||
// The host expects coordinates in *virtual desktop space* — i.e.
|
||||
// relative to the union of all monitors, not relative to the active
|
||||
// display. So we map browser canvas-rect coords → active-display
|
||||
// pixel coords (via canvas.size scaling), then add the active
|
||||
// display's origin (DisplayInfo.x / .y). Without the offset, clicks
|
||||
// on display 2 land on display 1 because both share canvas (0,0)
|
||||
// but only display 1's origin is (0,0) in virtual desktop space.
|
||||
|
||||
import { hbb } from "../proto/generated.js";
|
||||
import type { Session } from "../transport/session.js";
|
||||
@@ -38,7 +42,13 @@ function buttonBitForEvent(button: number): number {
|
||||
}
|
||||
}
|
||||
|
||||
export function bindMouse(canvas: CanvasView, session: Session): () => void {
|
||||
export function bindMouse(
|
||||
canvas: CanvasView,
|
||||
session: Session,
|
||||
/** Returns the current display's virtual-desktop origin (DisplayInfo.x, .y).
|
||||
* Polled per-event so display switches take effect immediately. */
|
||||
getOrigin: () => { x: number; y: number },
|
||||
): () => void {
|
||||
const el = canvas.el();
|
||||
|
||||
const sendMouse = (mask: number, x: number, y: number, ev: { altKey: boolean; ctrlKey: boolean; shiftKey: boolean; metaKey: boolean }): void => {
|
||||
@@ -57,7 +67,8 @@ export function bindMouse(canvas: CanvasView, session: Session): () => void {
|
||||
})).catch(() => { /* relay closed */ });
|
||||
};
|
||||
|
||||
// Translate viewport pixel coordinates → peer display pixels.
|
||||
// Translate viewport pixel coordinates → virtual-desktop pixels
|
||||
// (active display origin + position within display).
|
||||
const mapCoords = (clientX: number, clientY: number): { x: number; y: number } | null => {
|
||||
const sz = canvas.size();
|
||||
if (!sz.width || !sz.height) return null;
|
||||
@@ -73,7 +84,12 @@ export function bindMouse(canvas: CanvasView, session: Session): () => void {
|
||||
const px = (clientX - offsetX) / scale;
|
||||
const py = (clientY - offsetY) / scale;
|
||||
if (px < 0 || py < 0 || px > sz.width || py > sz.height) return null;
|
||||
return { x: px, y: py };
|
||||
// Offset by the active display's virtual-desktop origin. For a
|
||||
// single-display host this is (0,0). For multi-display, display 2
|
||||
// typically starts at x = display1.width (or wherever Windows put
|
||||
// it in Display Settings).
|
||||
const origin = getOrigin();
|
||||
return { x: px + origin.x, y: py + origin.y };
|
||||
};
|
||||
|
||||
// Track currently-pressed buttons so we can include them in move events.
|
||||
|
||||
+391
-88
@@ -20,6 +20,11 @@ import { AudioPipeline } from "./decode/audio.js";
|
||||
import { CanvasView } from "./ui/canvas.js";
|
||||
import { bindMouse } from "./input/mouse.js";
|
||||
import { bindKeyboard, sendCtrlAltDel } from "./input/keyboard.js";
|
||||
import {
|
||||
applyIncoming as applyClipboard,
|
||||
applyIncomingMulti as applyClipboardMulti,
|
||||
bindClipboard,
|
||||
} from "./input/clipboard.js";
|
||||
|
||||
interface CustomConfig {
|
||||
api_server: string;
|
||||
@@ -105,32 +110,54 @@ async function main(): Promise<void> {
|
||||
while (true) {
|
||||
const password = await askPassword(cfg, lastErr);
|
||||
try {
|
||||
await connectAndRun(cfg, serverPk, password);
|
||||
return; // success — connectAndRun rendered the success UI
|
||||
const ready = await connectOnce(cfg, serverPk, password, /*statusUI=*/true);
|
||||
// First connect succeeded — hand off to the session loop, which
|
||||
// owns reconnect.
|
||||
await runSession(cfg, serverPk, password, ready);
|
||||
return;
|
||||
} catch (e) {
|
||||
const msg = String(e);
|
||||
if (msg.includes("Wrong Password") || msg.includes("Bad password") || msg.toLowerCase().includes("password")) {
|
||||
lastErr = "Wrong password — try again.";
|
||||
continue;
|
||||
}
|
||||
// Anything else: surface and stop. The connectAndRun has likely
|
||||
// already rendered a specific error page, but provide a fallback.
|
||||
console.error("[rustdesk-web] fatal:", e);
|
||||
status(`<div class="error"><h1>Connection failed</h1><pre>${escapeHtml(msg)}</pre></div>`);
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async function connectAndRun(cfg: CustomConfig, serverPk: Uint8Array, password: string): Promise<void> {
|
||||
// ============ STEP 1 — PunchHoleRequest with nat_type=SYMMETRIC ============
|
||||
status(`<h1>Connecting to ${escapeHtml(cfg.peer_id)}…</h1><p class="muted">1/3: rendezvous</p>`);
|
||||
interface ConnectResult {
|
||||
session: Session;
|
||||
peerInfo: hbb.IPeerInfo;
|
||||
preloginExtras: hbb.Message[];
|
||||
}
|
||||
|
||||
/**
|
||||
* Single connect attempt: rendezvous → verify peer pk → relay WS →
|
||||
* secure handshake → login. Returns a live, logged-in Session.
|
||||
*
|
||||
* Used both for the initial connect (with statusUI=true to show
|
||||
* step-by-step progress) and for each reconnect attempt (statusUI=false
|
||||
* — the reconnect overlay is its own UI element, the placeholder is
|
||||
* already gone by then).
|
||||
*/
|
||||
async function connectOnce(
|
||||
cfg: CustomConfig,
|
||||
serverPk: Uint8Array,
|
||||
password: string,
|
||||
statusUI: boolean,
|
||||
): Promise<ConnectResult> {
|
||||
const setStatus = (html: string): void => { if (statusUI) status(html); };
|
||||
|
||||
setStatus(`<h1>Connecting to ${escapeHtml(cfg.peer_id)}…</h1><p class="muted">1/3: rendezvous</p>`);
|
||||
const wsRendezvous = rendezvousUrl(cfg.rendezvous_server);
|
||||
const ph = await punchHole(wsRendezvous, cfg.peer_id, cfg.key, CLIENT_VERSION);
|
||||
if (ph.signedIdPk.length === 0) {
|
||||
throw new Error("Server didn't sign the peer key — update the host's RustDesk to 1.4+.");
|
||||
}
|
||||
|
||||
// ============ STEP 2 — verify peer's Ed25519 SIGN pk ============
|
||||
let peerSignPk: Uint8Array;
|
||||
let verifiedId: string;
|
||||
try {
|
||||
@@ -154,8 +181,7 @@ async function connectAndRun(cfg: CustomConfig, serverPk: Uint8Array, password:
|
||||
console.warn("[rustdesk-web] no peer uuid in RelayResponse — fallback (relay pairing will likely fail)");
|
||||
}
|
||||
|
||||
// ============ STEP 3 — open relay WS ============
|
||||
status(`<h1>Connecting to ${escapeHtml(cfg.peer_id)}…</h1><p class="muted">2/3: relay handshake</p>`);
|
||||
setStatus(`<h1>Connecting to ${escapeHtml(cfg.peer_id)}…</h1><p class="muted">2/3: relay handshake</p>`);
|
||||
const wsRelay = relayUrl(ph.relayServer);
|
||||
const relay = await Relay.connect({
|
||||
wsUrl: wsRelay,
|
||||
@@ -164,11 +190,9 @@ async function connectAndRun(cfg: CustomConfig, serverPk: Uint8Array, password:
|
||||
licenceKey: cfg.key,
|
||||
});
|
||||
|
||||
// ============ STEP 4 — secure handshake + login ============
|
||||
status(`<h1>Connecting to ${escapeHtml(cfg.peer_id)}…</h1><p class="muted">3/3: secure handshake + login</p>`);
|
||||
let ready;
|
||||
setStatus(`<h1>Connecting to ${escapeHtml(cfg.peer_id)}…</h1><p class="muted">3/3: secure handshake + login</p>`);
|
||||
try {
|
||||
ready = await Session.open({
|
||||
const ready = await Session.open({
|
||||
relay,
|
||||
peerId: cfg.peer_id,
|
||||
peerSignPk,
|
||||
@@ -177,22 +201,21 @@ async function connectAndRun(cfg: CustomConfig, serverPk: Uint8Array, password:
|
||||
clientVersion: CLIENT_VERSION,
|
||||
sessionId: Math.floor(Math.random() * Number.MAX_SAFE_INTEGER),
|
||||
});
|
||||
(window as unknown as { __rdw: unknown }).__rdw = { relay, session: ready.session, peerInfo: ready.peerInfo, cfg };
|
||||
return ready;
|
||||
} catch (e) {
|
||||
relay.close();
|
||||
throw e;
|
||||
}
|
||||
|
||||
(window as unknown as { __rdw: unknown }).__rdw = { relay, session: ready.session, peerInfo: ready.peerInfo, cfg };
|
||||
|
||||
// ============ Post-login: render canvas + start video receive loop ============
|
||||
await runSession(cfg, ready.session, ready.peerInfo, ready.preloginExtras);
|
||||
}
|
||||
|
||||
const sleep = (ms: number): Promise<void> => new Promise((r) => setTimeout(r, ms));
|
||||
|
||||
async function runSession(
|
||||
cfg: CustomConfig,
|
||||
session: Session,
|
||||
peerInfo: hbb.IPeerInfo,
|
||||
preloginExtras: hbb.Message[],
|
||||
serverPk: Uint8Array,
|
||||
password: string,
|
||||
initial: ConnectResult,
|
||||
): Promise<void> {
|
||||
const root = document.getElementById("root");
|
||||
if (!root) return;
|
||||
@@ -205,18 +228,64 @@ async function runSession(
|
||||
const pipeline = new VideoPipeline((frame) => canvas.draw(frame));
|
||||
const audio = new AudioPipeline();
|
||||
|
||||
// Wire input.
|
||||
const detachMouse = bindMouse(canvas, session);
|
||||
const detachKb = bindKeyboard(canvas, session);
|
||||
// Mutable session — closures below read the current value at call
|
||||
// time (let-binding capture, not value capture), so reassigning on
|
||||
// reconnect makes every HUD button automatically forward to the new
|
||||
// session without re-binding handlers.
|
||||
let session = initial.session;
|
||||
let peerInfo = initial.peerInfo;
|
||||
let preloginExtras = initial.preloginExtras;
|
||||
|
||||
// Track the active display. Mouse coords need the display's virtual-
|
||||
// desktop origin (DisplayInfo.{x, y}) added to them so clicks on
|
||||
// display 2 don't land on display 1.
|
||||
let displays = peerInfo.displays || [];
|
||||
let currentDisplay = peerInfo.current_display || 0;
|
||||
if (currentDisplay < 0 || currentDisplay >= displays.length) currentDisplay = 0;
|
||||
const getOrigin = (): { x: number; y: number } => {
|
||||
const d = displays[currentDisplay];
|
||||
return { x: d?.x || 0, y: d?.y || 0 };
|
||||
};
|
||||
|
||||
// User-set option state. The host doesn't remember these across
|
||||
// sessions, so we re-send them on every reconnect.
|
||||
let userQuality: hbb.ImageQuality | null = null;
|
||||
let userFps: number | null = null;
|
||||
|
||||
// Wire input. Handlers close over the `session` let, so reassigning
|
||||
// it on reconnect transparently retargets them.
|
||||
let detachMouse = bindMouse(canvas, getSessionProxy(), getOrigin);
|
||||
let detachKb = bindKeyboard(canvas, getSessionProxy());
|
||||
let detachClip = bindClipboard(getSessionProxy());
|
||||
function getSessionProxy(): Session {
|
||||
// The input modules expect a Session; give them a Proxy that
|
||||
// forwards method calls to whatever session is current at call
|
||||
// time. Avoids re-binding window/canvas listeners on each reconnect.
|
||||
return new Proxy({} as Session, {
|
||||
get(_t, prop) {
|
||||
const target = session as unknown as Record<string | symbol, unknown>;
|
||||
const value = target[prop];
|
||||
if (typeof value === "function") {
|
||||
return (value as (...args: unknown[]) => unknown).bind(session);
|
||||
}
|
||||
return value;
|
||||
},
|
||||
});
|
||||
}
|
||||
// Focus the canvas so keystrokes start flowing immediately.
|
||||
canvas.el().focus();
|
||||
|
||||
// Wipe the initial "connecting…" text so it doesn't sit forever
|
||||
// *next to* the controls we're about to append.
|
||||
hudEl.textContent = "";
|
||||
|
||||
// FPS / dims line (left of HUD).
|
||||
const fpsLine = document.createElement("span");
|
||||
fpsLine.className = "hud-fps";
|
||||
hudEl.appendChild(fpsLine);
|
||||
|
||||
// Mute toggle.
|
||||
// Mute toggle. Also tells the peer to stop encoding audio while muted
|
||||
// (saves their CPU + relay bandwidth — re-enabled on unmute).
|
||||
const muteBtn = document.createElement("button");
|
||||
muteBtn.className = "hud-btn";
|
||||
muteBtn.textContent = "🔇 Mute";
|
||||
@@ -224,12 +293,120 @@ async function runSession(
|
||||
const muted = !audio.isMuted();
|
||||
audio.setMuted(muted);
|
||||
muteBtn.textContent = muted ? "🔈 Unmute" : "🔇 Mute";
|
||||
// BoolOption: NotSet=0, No=1, Yes=2 — explicit Yes/No; NotSet would
|
||||
// leave the host's existing setting alone.
|
||||
session.send(hbb.Message.create({
|
||||
misc: hbb.Misc.create({
|
||||
option: hbb.OptionMessage.create({
|
||||
disable_audio: muted
|
||||
? hbb.OptionMessage.BoolOption.Yes
|
||||
: hbb.OptionMessage.BoolOption.No,
|
||||
}),
|
||||
}),
|
||||
})).catch(() => { /* relay closed */ });
|
||||
// First user gesture also unblocks the AudioContext on browsers that
|
||||
// require a click before audio plays.
|
||||
audio.resume().catch(() => { /* ignore */ });
|
||||
});
|
||||
hudEl.appendChild(muteBtn);
|
||||
|
||||
// Image quality picker. Wire format per src/server/video_qos.rs:218-238:
|
||||
// - Preset enum (image_quality field): Low=2, Balanced=3, Best=4
|
||||
// - Custom (custom_image_quality field): bitrate-percent << 8 — host
|
||||
// does `(q >> 8 & 0xFFF) * 2 / 100` to recover the float ratio.
|
||||
// We expose the three presets only; the desktop client also has a
|
||||
// free slider that maps to the Custom branch, but presets cover the
|
||||
// common cases (Low for slow links, Best for screen sharing demos).
|
||||
const qualitySel = document.createElement("select");
|
||||
qualitySel.className = "hud-select";
|
||||
qualitySel.title = "Image quality";
|
||||
for (const [label, value] of [
|
||||
["Balanced", hbb.ImageQuality.Balanced],
|
||||
["Best", hbb.ImageQuality.Best],
|
||||
["Low", hbb.ImageQuality.Low],
|
||||
] as const) {
|
||||
const opt = document.createElement("option");
|
||||
opt.value = String(value);
|
||||
opt.textContent = label;
|
||||
qualitySel.appendChild(opt);
|
||||
}
|
||||
qualitySel.addEventListener("change", () => {
|
||||
const q = parseInt(qualitySel.value, 10) as hbb.ImageQuality;
|
||||
userQuality = q;
|
||||
session.send(hbb.Message.create({
|
||||
misc: hbb.Misc.create({
|
||||
option: hbb.OptionMessage.create({ image_quality: q }),
|
||||
}),
|
||||
})).catch(() => { /* relay closed */ });
|
||||
canvas.el().focus();
|
||||
});
|
||||
hudEl.appendChild(qualitySel);
|
||||
|
||||
// Custom FPS picker. Host clamps via VIDEO_QOS.user_custom_fps —
|
||||
// any value > 0 takes effect immediately, but the host caps at 30
|
||||
// for clients that don't advertise allow_more_fps (we don't, so 30
|
||||
// is the practical max even if we send 60).
|
||||
const fpsSel = document.createElement("select");
|
||||
fpsSel.className = "hud-select";
|
||||
fpsSel.title = "Target FPS";
|
||||
for (const fps of [15, 30, 60]) {
|
||||
const opt = document.createElement("option");
|
||||
opt.value = String(fps);
|
||||
opt.textContent = `${fps} fps`;
|
||||
if (fps === 30) opt.selected = true;
|
||||
fpsSel.appendChild(opt);
|
||||
}
|
||||
fpsSel.addEventListener("change", () => {
|
||||
const fps = parseInt(fpsSel.value, 10);
|
||||
userFps = fps;
|
||||
session.send(hbb.Message.create({
|
||||
misc: hbb.Misc.create({
|
||||
option: hbb.OptionMessage.create({ custom_fps: fps }),
|
||||
}),
|
||||
})).catch(() => { /* relay closed */ });
|
||||
canvas.el().focus();
|
||||
});
|
||||
hudEl.appendChild(fpsSel);
|
||||
|
||||
// Display picker — only shown when the host has more than one screen.
|
||||
if (displays.length > 1) {
|
||||
const sel = document.createElement("select");
|
||||
sel.className = "hud-select";
|
||||
displays.forEach((d, i) => {
|
||||
const opt = document.createElement("option");
|
||||
opt.value = String(i);
|
||||
opt.textContent = `Display ${i + 1} · ${d.width}×${d.height}`;
|
||||
if (i === currentDisplay) opt.selected = true;
|
||||
sel.appendChild(opt);
|
||||
});
|
||||
sel.addEventListener("change", () => {
|
||||
const idx = parseInt(sel.value, 10);
|
||||
// Two-message dance — required for clients >= 1.2.4 (we send
|
||||
// version "1.4.0"). The host's switch_display_to skips
|
||||
// unsubscribing the previous service for these clients and waits
|
||||
// for an explicit CaptureDisplays{set:[idx]} to clean up. Without
|
||||
// that follow-up, switching display 0 → 1 → 0 doesn't actually
|
||||
// restore display 0 because both video services stay subscribed
|
||||
// and the host's display_idx tracker can desync. See
|
||||
// src/server/connection.rs:3679-3699 (switch_display_to) and
|
||||
// src/ui_session_interface.rs:849-869 (mirrors this dance in the
|
||||
// desktop client).
|
||||
session.send(hbb.Message.create({
|
||||
misc: hbb.Misc.create({
|
||||
switch_display: hbb.SwitchDisplay.create({ display: idx }),
|
||||
}),
|
||||
})).catch(() => { /* relay closed */ });
|
||||
session.send(hbb.Message.create({
|
||||
misc: hbb.Misc.create({
|
||||
capture_displays: hbb.CaptureDisplays.create({ set: [idx] }),
|
||||
}),
|
||||
})).catch(() => { /* relay closed */ });
|
||||
currentDisplay = idx;
|
||||
canvas.el().focus();
|
||||
});
|
||||
hudEl.appendChild(sel);
|
||||
}
|
||||
|
||||
// Ctrl+Alt+Del (Windows hosts only — server-side `#[cfg(windows)]`).
|
||||
const cadBtn = document.createElement("button");
|
||||
cadBtn.textContent = "Ctrl+Alt+Del";
|
||||
@@ -243,79 +420,205 @@ async function runSession(
|
||||
setInterval(() => {
|
||||
const sz = canvas.size();
|
||||
const dims = sz.width ? `${sz.width}×${sz.height}` : "—";
|
||||
fpsLine.textContent = ` ${escapeHtml(peerInfo.hostname || cfg.peer_id)} · ${dims} · ${canvas.fps()} fps `;
|
||||
const d = pipeline.diagStats();
|
||||
// recv = wire frames/sec from the host, decode = browser-decoded
|
||||
// frames/sec, draw = canvas paints/sec. Three numbers narrow down
|
||||
// where any FPS gap is sitting.
|
||||
fpsLine.textContent = ` ${escapeHtml(peerInfo.hostname || cfg.peer_id)} · ${dims} · recv ${d.recvFps} / dec ${d.decodeFps} / draw ${canvas.fps()} fps · ${d.codec || "—"} `;
|
||||
}, 1000);
|
||||
void detachMouse; void detachKb; // kept-alive references for future cleanup
|
||||
void detachMouse; void detachKb; void detachClip; // kept-alive refs for future cleanup
|
||||
|
||||
// Replay any pre-login messages (e.g. Misc{audio_format} that arrives
|
||||
// before LoginResponse — the host's audio_service first-snapshot fires
|
||||
// between add_connection and login_response) into the post-login
|
||||
// dispatcher so they get handled.
|
||||
for (const m of preloginExtras) {
|
||||
if (m.misc?.audio_format) {
|
||||
try {
|
||||
audio.configure(m.misc.audio_format);
|
||||
} catch (e) {
|
||||
console.warn("[rustdesk-web] audio init failed:", e);
|
||||
}
|
||||
}
|
||||
}
|
||||
// Reconnect overlay — a centred banner that appears on top of the
|
||||
// canvas while we're trying to re-establish the session. The canvas
|
||||
// keeps its last-known frame underneath so the user has visual
|
||||
// continuity across blips.
|
||||
const overlay = document.createElement("div");
|
||||
overlay.className = "reconnect-overlay";
|
||||
overlay.style.display = "none";
|
||||
sessionEl.appendChild(overlay);
|
||||
const showOverlay = (text: string): void => {
|
||||
overlay.textContent = text;
|
||||
overlay.style.display = "flex";
|
||||
};
|
||||
const hideOverlay = (): void => {
|
||||
overlay.style.display = "none";
|
||||
};
|
||||
|
||||
// Long-lived receive loop. Branches by Message oneof.
|
||||
// Outer loop: each iteration is one connected session. After the
|
||||
// first iteration's recv loop exits, we attempt reconnect with
|
||||
// backoff and either continue here (success) or render a fatal
|
||||
// overlay and return (give up).
|
||||
let attempt = 0;
|
||||
const MAX_ATTEMPTS = 10;
|
||||
while (true) {
|
||||
let msg: hbb.Message;
|
||||
try {
|
||||
msg = await session.recv();
|
||||
} catch (e) {
|
||||
console.error("[rustdesk-web] session recv failed:", e);
|
||||
hudEl.textContent = `disconnected: ${String(e).slice(0, 80)}`;
|
||||
break;
|
||||
}
|
||||
|
||||
if (msg.video_frame) {
|
||||
pipeline.pushVideoFrame(msg.video_frame);
|
||||
continue;
|
||||
}
|
||||
if (msg.test_delay) {
|
||||
// Echo back so the peer can compute round-trip latency.
|
||||
const td = msg.test_delay;
|
||||
session.send(hbb.Message.create({
|
||||
test_delay: hbb.TestDelay.create({
|
||||
time: td.time,
|
||||
from_client: true,
|
||||
last_delay: td.last_delay,
|
||||
target_bitrate: td.target_bitrate,
|
||||
}),
|
||||
})).catch(() => { /* relay closing — drop */ });
|
||||
continue;
|
||||
}
|
||||
if (msg.audio_frame) {
|
||||
const data = (msg.audio_frame.data as Uint8Array) || new Uint8Array(0);
|
||||
audio.pushFrame(data);
|
||||
continue;
|
||||
}
|
||||
if (msg.cursor_data || msg.cursor_position || msg.cursor_id != null) {
|
||||
// Cursor sprite + position — deferred to a QoL milestone. The loose
|
||||
// `!= null` here is load-bearing: protobufjs static-module sets
|
||||
// unpopulated oneof fields to `null` (not `undefined`), so a strict
|
||||
// `!== undefined` check would swallow every other Message type and
|
||||
// keep e.g. Misc/audio_format from ever reaching its handler.
|
||||
continue;
|
||||
}
|
||||
if (msg.misc) {
|
||||
if (msg.misc.audio_format) {
|
||||
// Replay any pre-login messages from the just-completed login.
|
||||
// The most important one is Misc{audio_format} — the host's
|
||||
// audio_service first-snapshot fires between add_connection and
|
||||
// login_response, so it arrives BEFORE LoginResponse and we
|
||||
// stash it during login.
|
||||
for (const m of preloginExtras) {
|
||||
if (m.misc?.audio_format) {
|
||||
try {
|
||||
audio.configure(msg.misc.audio_format);
|
||||
audio.configure(m.misc.audio_format);
|
||||
// After (re)configure the gain defaults to 1; re-apply mute
|
||||
// if the user had toggled it before the disconnect.
|
||||
if (audio.isMuted()) audio.setMuted(true);
|
||||
} catch (e) {
|
||||
console.warn("[rustdesk-web] audio init failed:", e);
|
||||
}
|
||||
}
|
||||
continue;
|
||||
}
|
||||
}
|
||||
preloginExtras = [];
|
||||
|
||||
pipeline.close();
|
||||
audio.close();
|
||||
// On reconnect, re-apply user-set options. The host treats each
|
||||
// session as fresh and resets these to defaults.
|
||||
if (attempt > 0) {
|
||||
if (audio.isMuted()) {
|
||||
session.send(hbb.Message.create({
|
||||
misc: hbb.Misc.create({
|
||||
option: hbb.OptionMessage.create({
|
||||
disable_audio: hbb.OptionMessage.BoolOption.Yes,
|
||||
}),
|
||||
}),
|
||||
})).catch(() => { /* swallow */ });
|
||||
}
|
||||
if (userQuality != null) {
|
||||
session.send(hbb.Message.create({
|
||||
misc: hbb.Misc.create({
|
||||
option: hbb.OptionMessage.create({ image_quality: userQuality }),
|
||||
}),
|
||||
})).catch(() => { /* swallow */ });
|
||||
}
|
||||
if (userFps != null) {
|
||||
session.send(hbb.Message.create({
|
||||
misc: hbb.Misc.create({
|
||||
option: hbb.OptionMessage.create({ custom_fps: userFps }),
|
||||
}),
|
||||
})).catch(() => { /* swallow */ });
|
||||
}
|
||||
if (currentDisplay > 0) {
|
||||
session.send(hbb.Message.create({
|
||||
misc: hbb.Misc.create({
|
||||
switch_display: hbb.SwitchDisplay.create({ display: currentDisplay }),
|
||||
}),
|
||||
})).catch(() => { /* swallow */ });
|
||||
session.send(hbb.Message.create({
|
||||
misc: hbb.Misc.create({
|
||||
capture_displays: hbb.CaptureDisplays.create({ set: [currentDisplay] }),
|
||||
}),
|
||||
})).catch(() => { /* swallow */ });
|
||||
}
|
||||
}
|
||||
|
||||
// Inner receive loop. Branches by Message oneof. Returns when the
|
||||
// session breaks (relay close, decrypt fail, etc.).
|
||||
let disconnectReason = "";
|
||||
try {
|
||||
while (true) {
|
||||
const msg = await session.recv();
|
||||
if (msg.video_frame) {
|
||||
pipeline.pushVideoFrame(msg.video_frame);
|
||||
continue;
|
||||
}
|
||||
if (msg.test_delay) {
|
||||
const td = msg.test_delay;
|
||||
session.send(hbb.Message.create({
|
||||
test_delay: hbb.TestDelay.create({
|
||||
time: td.time,
|
||||
from_client: true,
|
||||
last_delay: td.last_delay,
|
||||
target_bitrate: td.target_bitrate,
|
||||
}),
|
||||
})).catch(() => { /* relay closing — drop */ });
|
||||
continue;
|
||||
}
|
||||
if (msg.audio_frame) {
|
||||
const data = (msg.audio_frame.data as Uint8Array) || new Uint8Array(0);
|
||||
audio.pushFrame(data);
|
||||
continue;
|
||||
}
|
||||
if (msg.cursor_data || msg.cursor_position || msg.cursor_id != null) {
|
||||
// Cursor sprite + position — deferred to a QoL milestone. The
|
||||
// loose `!= null` here is load-bearing: protobufjs static-
|
||||
// module sets unpopulated oneof fields to `null` (not
|
||||
// `undefined`), so a strict `!== undefined` check would
|
||||
// swallow every other Message type.
|
||||
continue;
|
||||
}
|
||||
if (msg.clipboard) {
|
||||
applyClipboard(msg.clipboard).catch(() => { /* swallowed */ });
|
||||
continue;
|
||||
}
|
||||
if (msg.multi_clipboards) {
|
||||
applyClipboardMulti(msg.multi_clipboards).catch(() => { /* swallowed */ });
|
||||
continue;
|
||||
}
|
||||
if (msg.misc) {
|
||||
if (msg.misc.audio_format) {
|
||||
try {
|
||||
audio.configure(msg.misc.audio_format);
|
||||
if (audio.isMuted()) audio.setMuted(true);
|
||||
} catch (e) {
|
||||
console.warn("[rustdesk-web] audio init failed:", e);
|
||||
}
|
||||
}
|
||||
continue;
|
||||
}
|
||||
}
|
||||
} catch (e) {
|
||||
disconnectReason = String(e);
|
||||
console.warn("[rustdesk-web] session disconnected:", disconnectReason);
|
||||
}
|
||||
|
||||
// Disconnected. Reset the codec pipeline so the new session's
|
||||
// first keyframe reconfigures from scratch (codec or resolution
|
||||
// may have changed if the peer was restarted).
|
||||
pipeline.close();
|
||||
|
||||
// Try to reconnect with exponential backoff. The first attempt
|
||||
// is immediate; subsequent attempts back off 1s, 2s, 4s, ...
|
||||
// capped at 30s.
|
||||
let reconnected: ConnectResult | null = null;
|
||||
for (attempt = 1; attempt <= MAX_ATTEMPTS; attempt++) {
|
||||
showOverlay(`Reconnecting (${attempt}/${MAX_ATTEMPTS})…`);
|
||||
try {
|
||||
reconnected = await connectOnce(cfg, serverPk, password, /*statusUI=*/false);
|
||||
break;
|
||||
} catch (e) {
|
||||
const msg = String(e);
|
||||
// Auth errors won't fix on retry — the peer changed something
|
||||
// (password rotated, key changed). Bail with a clear message.
|
||||
if (msg.toLowerCase().includes("password") ||
|
||||
msg.toLowerCase().includes("signature verify")) {
|
||||
showOverlay(`Cannot reconnect: ${msg.slice(0, 100)}`);
|
||||
audio.close();
|
||||
return;
|
||||
}
|
||||
if (attempt === MAX_ATTEMPTS) break;
|
||||
// 1s, 2s, 4s, 8s, 16s, 30s, 30s, ...
|
||||
const delay = Math.min(1000 * 2 ** (attempt - 1), 30_000);
|
||||
showOverlay(`Reconnecting (${attempt}/${MAX_ATTEMPTS})… retry in ${Math.round(delay / 1000)}s`);
|
||||
await sleep(delay);
|
||||
}
|
||||
}
|
||||
|
||||
if (!reconnected) {
|
||||
showOverlay(`Disconnected. ${escapeHtml(disconnectReason).slice(0, 120)}`);
|
||||
audio.close();
|
||||
return;
|
||||
}
|
||||
|
||||
// Reconnected — swap in the new session and loop back. The let-
|
||||
// binding capture means HUD click handlers and the input-handler
|
||||
// proxy automatically retarget without rebinding.
|
||||
session = reconnected.session;
|
||||
peerInfo = reconnected.peerInfo;
|
||||
preloginExtras = reconnected.preloginExtras;
|
||||
displays = peerInfo.displays || [];
|
||||
if (currentDisplay >= displays.length) currentDisplay = 0;
|
||||
hideOverlay();
|
||||
canvas.el().focus();
|
||||
}
|
||||
}
|
||||
|
||||
main().catch((err) => {
|
||||
|
||||
@@ -157,9 +157,14 @@ export class Session {
|
||||
supported_decoding: hbb.SupportedDecoding.create({
|
||||
ability_vp9: 1,
|
||||
ability_h264: 1,
|
||||
// VP8 / AV1 / H.265 left at 0 for v1; codec branches land in
|
||||
// M6f and the QoL milestones.
|
||||
prefer: hbb.SupportedDecoding.PreferCodec.VP9,
|
||||
ability_vp8: 1,
|
||||
// AV1 / H.265 left at 0 for v1.
|
||||
// Prefer VP8 — it's the cheapest software encoder of the
|
||||
// lot. Hosts without hwcodec/nvenc can keep up at 15-30 fps
|
||||
// for screen sharing, where VP9 caps at single digits and
|
||||
// H.264 falls back to "auto" (= VP9 in practice). VP8 is
|
||||
// universally supported in WebCodecs.
|
||||
prefer: hbb.SupportedDecoding.PreferCodec.VP8,
|
||||
}),
|
||||
// Explicitly opt INTO audio. The host's update_options() at
|
||||
// server/connection.rs:3974 only re-subscribes audio_service
|
||||
|
||||
@@ -142,6 +142,34 @@ html, body {
|
||||
}
|
||||
.hud-btn:hover { background: #475569; }
|
||||
|
||||
.hud-select {
|
||||
background: #334155;
|
||||
border: 0;
|
||||
color: #e2e8f0;
|
||||
padding: 2px 4px;
|
||||
border-radius: 3px;
|
||||
font-size: 11px;
|
||||
font-family: inherit;
|
||||
cursor: pointer;
|
||||
}
|
||||
.hud-select:hover { background: #475569; }
|
||||
|
||||
.reconnect-overlay {
|
||||
position: fixed;
|
||||
inset: 0;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
background: rgba(15, 23, 42, 0.65);
|
||||
backdrop-filter: blur(2px);
|
||||
-webkit-backdrop-filter: blur(2px);
|
||||
color: #e2e8f0;
|
||||
font-family: inherit;
|
||||
font-size: 15px;
|
||||
z-index: 20;
|
||||
pointer-events: none;
|
||||
}
|
||||
|
||||
.error {
|
||||
background: rgba(220, 38, 38, 0.15);
|
||||
border: 1px solid rgba(220, 38, 38, 0.4);
|
||||
|
||||
Reference in New Issue
Block a user