feat(call): native screen-capture pipeline + faster picker

Picker speed (Phase 1+2):
- screen_sources.rs split into list_screen_sources (metadata only,
  returns in ~10ms) + capture_screen_source_thumbnail (single source,
  by id). ScreenSourcePicker now shows names + placeholders instantly
  and streams thumbnails in as each capture lands. Total wall-clock
  is bounded by the slowest source instead of the serial sum.
- enumerate_screen_sources kept as a dead_code fallback so any
  rollout regression can switch the frontend back without code loss.

Native capture (Phase 3):
- New src-tauri/src/screen_capture.rs. start_screen_capture spawns a
  Rust thread per share that grabs frames via xcap, downscales to the
  user's quality preset, JPEG-encodes at Q72, and streams each frame
  through a Tauri Channel<FramePayload>. stop_screen_capture signals
  the stop flag and joins the worker.
- Worker re-resolves the xcap handle inside the thread because
  xcap::Window holds a !Send HWND — passing the source id string
  across the thread boundary sidesteps that.
- New lib/screenCapture.ts: decodes each frame into an ImageBitmap,
  draws to an offscreen canvas, exposes canvas.captureStream() as the
  MediaStream LiveKit publishes. Latest-wins frame queue drops stale
  frames when the JS side falls behind the Rust producer. 3s first-
  frame timeout so a silently-failing source (locked screen, DRM
  window) surfaces as a clean NativeCaptureUnavailable and we fall
  back to getDisplayMedia.
- CallContext.startScreenShare takes the native path first when the
  picker provided a sourceId and system audio wasn't requested. The
  old chromeMediaSourceId attempt and final setScreenShareEnabled
  fallback stay in place for the audio case + non-Tauri runtimes.
- stopScreenShare kills the native handle first, then unpublishes any
  manually-published ScreenShare/ScreenShareAudio tracks, then falls
  back to setScreenShareEnabled(false). disconnectRoom also stops
  the handle so we don't leak Rust threads across calls.

Scope note: native path is video-only. System-audio capture needs
WASAPI-loopback (Windows) or ScreenCaptureKit-audio (macOS); until
those are wired, requesting audio in the picker falls through to
the getDisplayMedia path and shows the OS picker for that one case.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
byGalax
2026-04-22 21:42:52 +02:00
parent 8f9b823d69
commit c3e0c47d32
7 changed files with 752 additions and 40 deletions
@@ -10,7 +10,8 @@ import {
updateScreenShareSettings,
} from '../lib/screenShareSettings';
import {
enumerateScreenSources,
captureScreenSourceThumbnail,
listScreenSources,
type ScreenSource,
thumbnailDataUrl,
} from '../lib/screenSources';
@@ -47,9 +48,11 @@ export function ScreenSourcePicker({ open, onClose, onStart }: Props) {
const [busy, setBusy] = useState(false);
const [error, setError] = useState<string | null>(null);
// Re-enumerate every time the picker opens so closed windows + new ones
// stay accurate. A previous stale list would surface sources the user
// can't actually share anymore.
// Two-phase load: (1) fast list returns names + placeholders so the grid
// paints instantly, (2) fire one thumbnail capture per source in
// parallel. Thumbnails fill in as each capture completes — Tauri's
// command thread pool runs them concurrently so wall-clock is bounded
// by the slowest source, not the sum.
useEffect(() => {
if (!open) {
setSources(null);
@@ -59,9 +62,27 @@ export function ScreenSourcePicker({ open, onClose, onStart }: Props) {
}
let cancelled = false;
void (async () => {
const list = await enumerateScreenSources();
const list = await listScreenSources();
if (cancelled) return;
setSources(list);
// Fan out thumbnail captures. No Promise.all — we want each result
// to render as it lands, not wait for the full batch. The id-based
// setState patch means the slowest source can still be in flight
// while the user already picked one of the fast ones.
for (const src of list) {
void (async () => {
const png = await captureScreenSourceThumbnail(src.id);
if (cancelled || png === null) return;
setSources((prev) => {
if (!prev) return prev;
const idx = prev.findIndex((s) => s.id === src.id);
if (idx === -1) return prev;
const next = prev.slice();
next[idx] = { ...prev[idx]!, thumbnailPng: png };
return next;
});
})();
}
})();
return () => {
cancelled = true;