feat(call): native screen-capture pipeline + faster picker
Picker speed (Phase 1+2): - screen_sources.rs split into list_screen_sources (metadata only, returns in ~10ms) + capture_screen_source_thumbnail (single source, by id). ScreenSourcePicker now shows names + placeholders instantly and streams thumbnails in as each capture lands. Total wall-clock is bounded by the slowest source instead of the serial sum. - enumerate_screen_sources kept as a dead_code fallback so any rollout regression can switch the frontend back without code loss. Native capture (Phase 3): - New src-tauri/src/screen_capture.rs. start_screen_capture spawns a Rust thread per share that grabs frames via xcap, downscales to the user's quality preset, JPEG-encodes at Q72, and streams each frame through a Tauri Channel<FramePayload>. stop_screen_capture signals the stop flag and joins the worker. - Worker re-resolves the xcap handle inside the thread because xcap::Window holds a !Send HWND — passing the source id string across the thread boundary sidesteps that. - New lib/screenCapture.ts: decodes each frame into an ImageBitmap, draws to an offscreen canvas, exposes canvas.captureStream() as the MediaStream LiveKit publishes. Latest-wins frame queue drops stale frames when the JS side falls behind the Rust producer. 3s first- frame timeout so a silently-failing source (locked screen, DRM window) surfaces as a clean NativeCaptureUnavailable and we fall back to getDisplayMedia. - CallContext.startScreenShare takes the native path first when the picker provided a sourceId and system audio wasn't requested. The old chromeMediaSourceId attempt and final setScreenShareEnabled fallback stay in place for the audio case + non-Tauri runtimes. - stopScreenShare kills the native handle first, then unpublishes any manually-published ScreenShare/ScreenShareAudio tracks, then falls back to setScreenShareEnabled(false). disconnectRoom also stops the handle so we don't leak Rust threads across calls. Scope note: native path is video-only. System-audio capture needs WASAPI-loopback (Windows) or ScreenCaptureKit-audio (macOS); until those are wired, requesting audio in the picker falls through to the getDisplayMedia path and shows the OS picker for that one case. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -1,4 +1,5 @@
|
||||
mod crypto;
|
||||
mod screen_capture;
|
||||
mod screen_sources;
|
||||
|
||||
#[cfg(feature = "rust-livekit")]
|
||||
@@ -93,6 +94,10 @@ pub fn run() {
|
||||
crypto::crypto_box_seal_open,
|
||||
crypto::crypto_pwhash,
|
||||
screen_sources::enumerate_screen_sources,
|
||||
screen_sources::list_screen_sources,
|
||||
screen_sources::capture_screen_source_thumbnail,
|
||||
screen_capture::start_screen_capture,
|
||||
screen_capture::stop_screen_capture,
|
||||
])
|
||||
.plugin(tauri_plugin_notification::init());
|
||||
|
||||
@@ -110,6 +115,10 @@ pub fn run() {
|
||||
crypto::crypto_box_seal_open,
|
||||
crypto::crypto_pwhash,
|
||||
screen_sources::enumerate_screen_sources,
|
||||
screen_sources::list_screen_sources,
|
||||
screen_sources::capture_screen_source_thumbnail,
|
||||
screen_capture::start_screen_capture,
|
||||
screen_capture::stop_screen_capture,
|
||||
livekit_bridge::livekit_connect,
|
||||
livekit_bridge::livekit_disconnect,
|
||||
livekit_bridge::livekit_send_data,
|
||||
|
||||
Reference in New Issue
Block a user