mike
d07e98e607
Adds a TypeScript SPA embedded in hbbs that lets a logged-in admin click
"Connect (web client)" on a Devices row and remote-control the peer from
the browser, no desktop client install required. View, mouse/keyboard
control, and host-audio playback all work end-to-end.
Architecture
------------
Pure browser app, no server-side WS proxy:
Browser ──ws://hbbs:21118── rendezvous (PunchHole + RequestRelay)
│
└──ws://hbbr:21119──── relay (paired by uuid)
│
└── peer (RustDesk desktop, any platform)
Same wire path the desktop client takes via `Client::request_relay` and
`Client::create_relay`. Browser-relay-only — no NAT punching, so we send
nat_type=SYMMETRIC in PunchHoleRequest to make the peer skip the direct
attempt and go straight to relay (initiate=true with a host-generated uuid
that we then use for our own relay leg).
The 5 wire steps:
1. PunchHoleRequest → harvests signed peer Ed25519 sign-pk + relay host
+ the peer's session uuid from RelayResponse
2. Verify the signed pk against /admin/connect's `id_ed25519.pub`
3. Open WS to hbbr:21119, send RequestRelay with that uuid
4. Read peer's Message{signed_id}, verify with peer_sign_pk, extract
Curve25519 box pk; box-seal a fresh secretbox key under it; send
Message{public_key} unencrypted
5. Secretbox-encrypted stream from here. Hash → LoginRequest →
LoginResponse with PeerInfo. Mode = Legacy (Translate silently drops
ControlKey/Unicode payloads on the host side).
New Rust surface
----------------
- `admin_ui/connect.html` — SPA shell with `{{CUSTOM_CONFIG}}` placeholder
- `src/api/admin/pages/connect.rs` — gates on AuthedUser, injects per-request
config (rendezvous host, relay host, server pubkey, peer_id, admin name)
into the `<script id="custom-config">` tag, serves bundle.{js,css} via
include_bytes!
- 3 routes added: GET /admin/connect/:peer_id and the two assets
- Devices dropdown gains a sky-blue "Connect (web client)" link that opens
in a new tab
New TypeScript SPA (`web_client/`)
----------------------------------
Stack: pure DOM/TS, no React/Vue. Bundled by esbuild → `dist/bundle.{js,css}`
which is committed (cargo build needs no Node toolchain).
src/main.ts boot + password retry loop + receive dispatch
src/crypto.ts tweetnacl wrapper (sign_open, box, secretbox)
+ @noble/hashes/sha2 (works on plain http;
SubtleCrypto requires a secure context)
src/proto/generated.{js,d.ts} pbjs static-module from
libs/hbb_common/protos/{rendezvous,message}.proto
src/transport/rendezvous.ts WS to hbbs; PunchHole + RequestRelay
src/transport/relay.ts WS to hbbr; duplex frame transport
src/transport/session.ts secure-handshake state machine + Hash/Login
+ 8-byte LE secretbox sequence counter
(PRE-increment, send/recv independent —
matches libs/hbb_common/src/tcp.rs:317-320)
+ preloginExtras stash for AudioFormat that
arrives before LoginResponse
src/decode/video.ts WebCodecs VideoDecoder (vp09.00.10.08 today;
h264/h265/av1/vp8 codec strings ready for M6f)
src/decode/audio.ts WebCodecs AudioDecoder (opus) → AudioContext;
detects f32 vs f32-planar AudioData layout
and deinterleaves when needed; gap-less
scheduling via a sliding playhead
src/ui/canvas.ts <canvas> with object-fit: contain letterbox;
auto-resizes on resolution change; FPS counter
src/input/mouse.ts MouseEvent → MouseEvent proto. Mask layout:
(button << 3) | type (0=move,1=down,2=up,
3=wheel). Letterbox-aware viewport→peer
coord mapping. Right-click suppresses the
browser context menu; left-click does NOT
preventDefault (would block focus)
src/input/keyboard.ts Window-level keydown/keyup → KeyEvent proto
in Legacy mode. Special keys → ControlKey
enum; printable → unicode codepoint (down
only, host's process_unicode does a single
key_click). Browser shortcuts allowlisted
(Cmd-T/N/W/R, Tab) so the user keeps tab
control. Ctrl+Alt+Del HUD button (host-side
`send_sas` is `#[cfg(windows)]`; no-op on
Mac/Linux hosts but present for parity)
Bundle size: 529 KB raw / ~74 KB gzipped. Tree-shaken protobufjs +
tweetnacl + @noble/hashes only.
Deployment notes
----------------
- WebCodecs and SubtleCrypto are gated to "secure context" origins —
HTTPS, or http://localhost. Plain http://lan-ip won't work. Open via
http://localhost during dev, or terminate TLS in front of hbbs (Caddy
/ nginx / Traefik) for production access.
- `--relay-servers <host>` on hbbs must point at a host where TCP/WS
21119 is reachable from end-user browsers.
Wire-format gotchas this commit nails (each one was a session of bisecting)
--------------------------------------------------------------------------
- Hash.salt / Hash.challenge are proto `string` fields used as raw UTF-8
bytes in the SHA-256 chain. NOT base64-decoded. `pwd_hash =
SHA256(pwd_text || salt_utf8)`, `resp = SHA256(pwd_hash || challenge_utf8)`.
- Translate keyboard mode silently drops Unicode + ControlKey payloads on
the host (input_service.rs:2022 has `// Do not handle unicode for now.`).
Only Seq + Chr work in Translate. Use Legacy (mode=0) for everything.
- Browser is forced to relay path by sending nat_type=SYMMETRIC. The peer
generates its OWN uuid in handle_punch_hole's symmetric branch; use that
uuid (carried back in RelayResponse) for the relay leg, not a fresh one.
- Misc{audio_format} fires from the host's audio_service first-snapshot
BETWEEN add_connection and login_response, so it lands on the wire
before our session.recv() loop is set up. Session.open() captures
pre-login messages into preloginExtras for the caller to replay.
- protobufjs static-module sets unpopulated oneof fields to JS `null`,
not `undefined`. A `if (msg.cursor_id !== undefined)` cursor branch
swallowed every other message type including Misc; switched to loose
`!= null` comparison.
- WebCodecs AudioDecoder for opus emits `f32` (interleaved) AudioData —
must deinterleave into separate AudioBuffer channels before playback.
- VideoDecoder/AudioDecoder/SubtleCrypto are SecureContext-only; need
http://localhost or https:// on the *page origin*, not the WS targets.
- libsodium-wrappers ESM ships a broken relative import (`./libsodium.mjs`
in a sibling package); switched to tweetnacl which has no such problem.
- WebCrypto's SubtleCrypto.digest() doesn't accept SharedArrayBuffer-backed
Uint8Arrays in newer TS lib types; doesn't matter — we use @noble/hashes
for sha256 anyway since Subtle is secure-context-only.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
56 lines
1.9 KiB
Markdown
56 lines
1.9 KiB
Markdown
# RustDesk web client
|
|
|
|
Browser-based RustDesk client embedded in `rustdesk-server`. Surfaced from the
|
|
admin dashboard as a "Connect" button on the Devices page.
|
|
|
|
## Architecture (one-liner)
|
|
|
|
Plain TypeScript SPA → talks WebSocket directly to `hbbs:21118` (rendezvous)
|
|
and `hbbr:21119` (relay) → `protobufjs` for wire format → `libsodium-wrappers`
|
|
for crypto → WebCodecs for video/audio decode → `<canvas>` for display.
|
|
|
|
No frameworks. ~1 MB minified.
|
|
|
|
## Building
|
|
|
|
```sh
|
|
./build.sh # bundles to dist/bundle.{js,css}
|
|
git add dist/
|
|
```
|
|
|
|
`dist/` is committed so `cargo build -p hbbs` doesn't need Node. Anyone
|
|
touching code under `src/` should re-run `./build.sh` and commit the new
|
|
`dist/` files in the same commit.
|
|
|
|
## Regenerating proto bindings
|
|
|
|
Rare — only when `libs/hbb_common` bumps and proto fields change:
|
|
|
|
```sh
|
|
npm run protogen
|
|
./build.sh
|
|
git add src/proto/generated.* dist/
|
|
```
|
|
|
|
## Layout
|
|
|
|
```
|
|
src/
|
|
main.ts boot: read #custom-config, init transport
|
|
crypto.ts libsodium wrapper
|
|
proto/ generated protobufjs static modules (committed)
|
|
transport/ rendezvous WS, relay WS, secure handshake state machine
|
|
decode/ video (WebCodecs VideoDecoder), audio (AudioDecoder)
|
|
input/ mouse/keyboard capture → protobuf MouseEvent/KeyEvent
|
|
ui/ canvas + toolbar + style.css
|
|
audit.ts POST /api/audit/conn with admin cookie
|
|
dist/
|
|
bundle.js + .css committed esbuild output
|
|
```
|
|
|
|
## Wire-protocol references
|
|
|
|
- `/Users/sn0/Desktop/rustdesk-server/libs/hbb_common/protos/{rendezvous,message}.proto`
|
|
- `/Users/sn0/Desktop/rustdesk/src/client.rs` — desktop-client connect/secure/login state machine
|
|
- `/Users/sn0/Desktop/rustdesk-server/libs/hbb_common/src/tcp.rs:296-344` — secretbox nonce derivation (8-byte LE counter, separate per direction)
|