feat(call): native screen-capture pipeline + faster picker

Picker speed (Phase 1+2):
- screen_sources.rs split into list_screen_sources (metadata only,
  returns in ~10ms) + capture_screen_source_thumbnail (single source,
  by id). ScreenSourcePicker now shows names + placeholders instantly
  and streams thumbnails in as each capture lands. Total wall-clock
  is bounded by the slowest source instead of the serial sum.
- enumerate_screen_sources kept as a dead_code fallback so any
  rollout regression can switch the frontend back without code loss.

Native capture (Phase 3):
- New src-tauri/src/screen_capture.rs. start_screen_capture spawns a
  Rust thread per share that grabs frames via xcap, downscales to the
  user's quality preset, JPEG-encodes at Q72, and streams each frame
  through a Tauri Channel<FramePayload>. stop_screen_capture signals
  the stop flag and joins the worker.
- Worker re-resolves the xcap handle inside the thread because
  xcap::Window holds a !Send HWND — passing the source id string
  across the thread boundary sidesteps that.
- New lib/screenCapture.ts: decodes each frame into an ImageBitmap,
  draws to an offscreen canvas, exposes canvas.captureStream() as the
  MediaStream LiveKit publishes. Latest-wins frame queue drops stale
  frames when the JS side falls behind the Rust producer. 3s first-
  frame timeout so a silently-failing source (locked screen, DRM
  window) surfaces as a clean NativeCaptureUnavailable and we fall
  back to getDisplayMedia.
- CallContext.startScreenShare takes the native path first when the
  picker provided a sourceId and system audio wasn't requested. The
  old chromeMediaSourceId attempt and final setScreenShareEnabled
  fallback stay in place for the audio case + non-Tauri runtimes.
- stopScreenShare kills the native handle first, then unpublishes any
  manually-published ScreenShare/ScreenShareAudio tracks, then falls
  back to setScreenShareEnabled(false). disconnectRoom also stops
  the handle so we don't leak Rust threads across calls.

Scope note: native path is video-only. System-audio capture needs
WASAPI-loopback (Windows) or ScreenCaptureKit-audio (macOS); until
those are wired, requesting audio in the picker falls through to
the getDisplayMedia path and shows the OS picker for that one case.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
byGalax
2026-04-22 21:42:52 +02:00
parent 8f9b823d69
commit c3e0c47d32
7 changed files with 752 additions and 40 deletions
+9
View File
@@ -1,4 +1,5 @@
mod crypto;
mod screen_capture;
mod screen_sources;
#[cfg(feature = "rust-livekit")]
@@ -93,6 +94,10 @@ pub fn run() {
crypto::crypto_box_seal_open,
crypto::crypto_pwhash,
screen_sources::enumerate_screen_sources,
screen_sources::list_screen_sources,
screen_sources::capture_screen_source_thumbnail,
screen_capture::start_screen_capture,
screen_capture::stop_screen_capture,
])
.plugin(tauri_plugin_notification::init());
@@ -110,6 +115,10 @@ pub fn run() {
crypto::crypto_box_seal_open,
crypto::crypto_pwhash,
screen_sources::enumerate_screen_sources,
screen_sources::list_screen_sources,
screen_sources::capture_screen_source_thumbnail,
screen_capture::start_screen_capture,
screen_capture::stop_screen_capture,
livekit_bridge::livekit_connect,
livekit_bridge::livekit_disconnect,
livekit_bridge::livekit_send_data,