Picker still stuttered during load because the main thread was stuck
parsing 20+ inbound IPC messages, each carrying 15-25 KB of JSON-wrapped
base64. Two changes compound to fix this:
1. Binary IPC. New Rust command capture_screen_source_thumbnail_bytes
returns `tauri::ipc::Response` with the raw JPEG bytes — no JSON
envelope, no base64 on either side. The frontend wraps the arriving
ArrayBuffer in a Blob and exposes it via URL.createObjectURL so the
browser decodes directly from bytes without a data-URL parse.
Empirically drops per-arrival main-thread work from ~10-15 ms to
~1-2 ms.
2. rAF-batched thumbnail state updates. Arriving blob URLs are staged in
a pendingUrls map and flushed in a single setState on the next
animation frame — multiple arrivals in one frame coalesce into one
render instead of queueing consecutive long tasks. Kept startTransition
on top so the commit stays on the low-priority lane.
Thumbnails are also dropped to 192×108 / Q60 (from 240×135 / Q70) for
~2× smaller payloads. Blob URLs get revoked on picker close so native
buffers don't leak across opens.
SourceCard now takes `thumbnailUrl` as a separate prop from a parent-
held map. Keeps source object references stable so React.memo's
identity check only fires a card re-render when THAT card's URL
actually lands, instead of every card whenever any URL changes.
Next session: WASAPI loopback for system-audio capture in native share.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previous pass moved to JPEG + startTransition but the grid still froze.
Root causes that survived:
1. React.memo was broken — the parent re-created the inline
`onClick={() => onSelect(src.id)}` arrow on every render, so memo's
reference check always triggered a fresh render on every card even
though nothing visible had changed. Fixed by passing `onSelect` as a
stable prop and constructing the click handler inside the memoized
child.
2. 20 data-URL `<img>` sources getting decoded more or less at once gave
the compositor enough work to make scroll feel laggy. `decoding="async"`
punts decode to the browser's image thread; `loading="lazy"` skips it
entirely for cards outside the viewport.
3. Concurrency at 4 was still high enough for Windows GDI BitBlt /
PrintWindow to contend for the desktop compositor — the whole Tauri
window stuttered because the OS-level screen capture was saturating
the graphics pipeline. Dropped to 2 concurrent captures; total load
takes a touch longer but the picker stays interactive throughout.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The picker still felt frozen while thumbnails were streaming in because
each result was both (a) large — PNG @ 320×180 landed at 60-150 KB
base64 — and (b) triggering a high-priority React re-render of the whole
grid. Three fixes together restore interactivity:
- Thumbnails encoded as JPEG @ Q70 at 240×135 instead of PNG @ 320×180.
Drops the typical payload from ~100 KB to ~20 KB, so IPC JSON-parsing
on arrival is 5× faster.
- SourceCard wrapped in React.memo so only the card whose thumbnail just
landed re-renders. Previously one new thumbnail caused all ~20 cards
to re-evaluate their props.
- setSources updates run inside startTransition so scroll / click events
stay on the high-priority lane while the grid backfills.
Also: when the user enables "Sound mit übertragen" AND has a source
picked, the picker now surfaces an inline amber note explaining that
the OS picker will appear for the audio capture path. Matches the
existing console info log but is visible pre-click so users don't
experience it as a bug.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Volume crash:
- setParticipantVolume / setScreenShareVolume propagated values up to 2.0
(200%) to the per-track GainNode, but also called applyToAttachedElements
which set the raw HTMLAudioElement.volume — that property is hard-clamped
to [0, 1] and throws IndexSizeError above 1. Clip the element-path apply
at 1.0. WebAudio GainNode keeps doing the actual amplification.
Picker freeze:
- Firing ~20 captureScreenSourceThumbnail invokes in parallel caused
perceptible input freezes while each ~100KB base64 result arrived and
triggered a setState. Bounded the worker pool to 4 concurrent captures
with a queue — overall wall-clock is nearly identical and the grid stays
scrollable / clickable throughout the load.
Native-path diagnostics:
- Previous logs only fired on non-NativeCaptureUnavailable errors, so
users couldn't tell whether the native path was skipped (audio toggle
on, no sourceId) or attempted-and-failed. Added explicit info logs for
each skip reason plus an always-on warn with the underlying error when
the try block throws. Makes the next debug pass on screenshare much
quicker.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Picker speed (Phase 1+2):
- screen_sources.rs split into list_screen_sources (metadata only,
returns in ~10ms) + capture_screen_source_thumbnail (single source,
by id). ScreenSourcePicker now shows names + placeholders instantly
and streams thumbnails in as each capture lands. Total wall-clock
is bounded by the slowest source instead of the serial sum.
- enumerate_screen_sources kept as a dead_code fallback so any
rollout regression can switch the frontend back without code loss.
Native capture (Phase 3):
- New src-tauri/src/screen_capture.rs. start_screen_capture spawns a
Rust thread per share that grabs frames via xcap, downscales to the
user's quality preset, JPEG-encodes at Q72, and streams each frame
through a Tauri Channel<FramePayload>. stop_screen_capture signals
the stop flag and joins the worker.
- Worker re-resolves the xcap handle inside the thread because
xcap::Window holds a !Send HWND — passing the source id string
across the thread boundary sidesteps that.
- New lib/screenCapture.ts: decodes each frame into an ImageBitmap,
draws to an offscreen canvas, exposes canvas.captureStream() as the
MediaStream LiveKit publishes. Latest-wins frame queue drops stale
frames when the JS side falls behind the Rust producer. 3s first-
frame timeout so a silently-failing source (locked screen, DRM
window) surfaces as a clean NativeCaptureUnavailable and we fall
back to getDisplayMedia.
- CallContext.startScreenShare takes the native path first when the
picker provided a sourceId and system audio wasn't requested. The
old chromeMediaSourceId attempt and final setScreenShareEnabled
fallback stay in place for the audio case + non-Tauri runtimes.
- stopScreenShare kills the native handle first, then unpublishes any
manually-published ScreenShare/ScreenShareAudio tracks, then falls
back to setScreenShareEnabled(false). disconnectRoom also stops
the handle so we don't leak Rust threads across calls.
Scope note: native path is video-only. System-audio capture needs
WASAPI-loopback (Windows) or ScreenCaptureKit-audio (macOS); until
those are wired, requesting audio in the picker falls through to
the getDisplayMedia path and shows the OS picker for that one case.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
HTMLMediaElement.volume caps at 1.0, so boosting a quiet peer past
100% needs an explicit GainNode in the output chain. New
remoteAudioPipelines module owns one AudioContext + GainNode per
remote audio track; attachTrack / detachTrack now create and tear
down the pipeline alongside the LiveKit element.
Once a track is on the WebAudio path its direct output is diverted
(createMediaElementSource semantics), so audio.muted / volume can't
drive output anymore. Deafen, watch-state, manual screen-share mute
and per-user volume are collapsed into one effective-gain formula
that gets recomputed on every state flip — the effect subscribes to
both participantVolumes and screenShareVolumes for live slider drags.
Slider ranges updated to 0–200% across:
- ParticipantVolumeMenu (per-user right-click menu)
- ParticipantsPopover (in-call participant list)
- ScreenShareContextMenu (per-share right-click)
Values above 100% render the percentage in amber as a soft hint that
clipping is possible. Clamp in both volume stores extended to [0, 2]
so persisted values survive.
setAudioOutputDevice now additionally routes via
AudioContext.setSinkId (Chrome 115+) for the WebAudio graph; the
HTMLAudioElement.setSinkId fallback stays for older runtimes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previously the music icon always rendered in the in-call bar, opening a
popover with a "Keine Sounds gespeichert" empty-state. Matches Discord's
pattern better to just drop the button entirely until the user has
something to play — otherwise it reads as a broken control. Live-
subscribes to soundboardStorage so adding/removing sounds mid-call flips
the button in or out without reopening the call. Also closes the popover
automatically if the user clears their last sound while it's open.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Rust side:
- New src-tauri/src/screen_sources.rs with an enumerate_screen_sources
command. Uses the xcap crate for cross-platform screen + window
enumeration and capture; PNG thumbnails are letterbox-scaled to fit
320x180 and returned as base64.
- Source ids emit Chromium's internal desktopCapturer format
("screen:<id>:0", "window:<hwnd>:0") so JS can try passing them
straight into chromeMediaSourceId.
- Registered in both invoke_handler branches in lib.rs.
Frontend:
- New lib/screenSources.ts — Tauri command wrapper + thumbnailDataUrl
helper for the picker UI.
- New components/ScreenSourcePicker.tsx — Discord-style grid: sources
grouped under "Bildschirme" / "Fenster", large thumbnail cards with
selection state, quality preset + system-audio toggle in the footer.
"Teilen" button is enabled either way; without a selection it says
"Ohne Auswahl weiter" and falls through to the OS picker.
- Replaces the old form-style ScreenShareDialog entirely (removed).
CallContext wiring:
- startScreenShare now accepts an optional sourceId. When set, it
captures that exact source via getUserMedia's legacy
chromeMediaSourceId constraint and publishes the resulting tracks
manually (video as ScreenShare, audio as ScreenShareAudio). Falls
back to setScreenShareEnabled if WebView2 rejects the constraint,
so users always get a working share even if the direct path fails.
- Track 'ended' listeners unpublish the pub when the OS revokes
capture (close of shared window, OS "stop sharing" banner).
InCallPanel:
- Left-click on the share button now opens the picker instead of
starting with last-saved settings; right-click opens it too. The
picker itself is the 1-click UX.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Watch-gate lifted into CallContext. watchingShareUserIds /
dismissedShareUserIds / screenShareAudioMutedIds as session-only state,
cleared on CallState.idle and on TrackUnsubscribed for each sharer.
Survives layout changes (grid <-> focus <-> fullscreen) without
resetting which the old local-state viewer dropped on remount.
- ScreenShareAudio tracks tagged via data-track-source="screenshare" at
attach-time; initial muted follows watching + manual mute mirrors so
audio never plays before the user clicks "Bildschirm anschauen". Deafen
still wins at the top of the priority chain.
- New screenShareVolumes store (session-only, keyed by participantId).
attachTrack pulls the initial volume from this store for screenshare
audio elements so the context-menu slider takes effect immediately.
- Screen shares are no longer auto-promoted to focus. They render as
equal-size grid tiles like everyone else; user clicks to focus. The
"Bildschirm anschauen" overlay replaces auto-play as the opt-in.
- Dismissed sharer-ids filter out of buildTiles, so "Zuschauen beenden"
really hides the tile until the sharer stops + restarts.
- New ScreenShareContextMenu (portal, Esc / outside-click to close):
volume slider + audio mute toggle when the share has audio + a
destructive "Zuschauen beenden" row. Wired via a dispatcher in
InCallPanel that picks between participant-volume and share-menu
based on tile.kind.
- Fullscreen cinema gets a "Hide participant strip" toggle (top-right,
session-only) so focused content reaches the full viewport when the
bottom thumbnail row would otherwise steal 160px. Fades with the
auto-hide controls; only surfaces when there's a focus + peers to hide.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Default flipped to off so voice doesn't get coloured by browser NS on
first run. Users turn it on explicitly under Settings → Sprache, where
the toggle already lived before the in-call button shipped.
The in-call SparklesIcon button and all the associated wiring
(onToggleNoiseSuppression, noiseSuppression prop, local subscription
in InCallPanel) is removed. The hot-swap UX is preserved: a new
subscribeAudioSettings watcher in CallContext detects noiseSuppression
flips during an active call and re-runs setupMicPipeline so the change
takes effect without rejoining.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- New voiceHotkeys.ts storage module. Stores per-action bindings with
modifier flags (Ctrl/Shift/Alt) so Discord-style chords like
Ctrl+Shift+M work. Defaults match Discord — Ctrl+Shift+M mute,
Ctrl+Shift+D deafen — but ship disabled to avoid surprise collisions.
- globalShortcut.ts grows registerGlobalShortcutPress /
unregisterGlobalShortcut helpers that accept pre-formatted Tauri
accelerator strings, since voiceHotkey chords can't be expressed by
the existing codeToShortcut path (PTT-only single-key).
- CallContext registers the chords OS-wide while a call is active
(connected | reconnecting) so the hotkeys work from any focused
window. A window-keydown fallback handles the non-Tauri / denied
registration case. Both unregister on call end.
- SettingsPage adds VoiceHotkeyControls (mute + deafen variants)
with a chord-capture button that waits past modifier-only presses
and binds to the first real key. Labels render as "Ctrl+Shift+M"
consistently with the in-call PTT hint.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- PiP widget now shows a live mm:ss / hh:mm:ss duration instead of the
generic "tippe zum Öffnen" while a call is in progress.
- New ParticipantsPopover — portal-mounted, fixed bottom-right, lists
everyone in the call with avatar, speaking ring, mute/deafen badges
and a per-peer volume slider. Wired to CallControls via the users
button (data-participants-trigger skips the outside-click dismiss
while toggling).
- Non-terminal MicErrorBanner: getUserMedia failures inside joinRoom
used to be silently swallowed by a console.error; they now set a
categorized message (NotAllowedError / NotFoundError / NotReadableError)
on CallContext.micError, render as a rose banner in both docked and
fullscreen modes, and offer a Retry button that calls the extracted
setupMicPipeline without rejoining the room.
- Screen-share toggle is now 1-click using the last-saved preset +
displaySurface. Right-click on the share button still opens the
quality dialog for users who want to adjust before starting.
- Noise-suppression toggle in the control bar (SparklesIcon). Flipping
it updates audioSettings and hot-swaps the mic track via
setAudioInputDevice so the new constraint takes effect without a
rejoin. Mirrors Discord's Krisp button placement.
- Fullscreen auto-speaker now tracks "most recently started speaking"
instead of "exactly one currently speaking", so two people briefly
overlapping doesn't kick the focus back to grid. Tracked in a
prevSpeakers ref against each activeSpeakers diff.
- Fullscreen controls auto-hide after 5s of mouse idle; mousemove /
touchstart bring them back. Pinned visible while any popover
(soundboard / volume-menu / participants / mic-error banner) is open
so users can interact without the chrome fading mid-click.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A — Call core:
- Deafen now implies mute + remembers pre-deafen mic state so un-deafen
restores it (Discord-parity). Peers still see the headphones-off +
mic-off badges in sync via the existing data-channel broadcast.
- Self-join sound fires on the local peer's r.connect() too, not just
on remote ParticipantConnected, so the user gets the "I'm in" cue.
- New CallState.reconnecting holds the UI steady when LiveKit drops the
signaling socket and retries; duration keeps ticking, status label
switches to "Verbinde neu…". Full teardown only on terminal
Disconnected (after LK gives up).
- joinActiveCall falls back to connected after 5s if no peer arrived —
avoids hanging in "Verbinde…" when peers left the room mid-rejoin.
B — Ringtone:
- Oscillator base gain up (incoming 0.22 -> 0.4, outgoing 0.14 -> 0.22)
so the default pattern survives laptop speakers + background music.
- New ringtoneVolume slider in Settings, default 0.9, live-applies to
both the oscillator fallback and the custom-file <audio> element.
C — Participant tile:
- Split the speaking indicator: video tiles get the emerald border +
inset glow; audio tiles rely on the existing avatar pulse. No more
double-chrome when someone talks in grid/focus view.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
v1 used to wrap the installer in a .nsis.zip and sign that wrapper.
v2 signs the .exe directly, so the updater url points at the .exe and
the .sig file sits next to it. Script was still looking for the
legacy .nsis.zip path and aborting after a successful build.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Node execSync runs via cmd.exe on Windows, which preserves the `--`
separator pnpm injects between the script name and forwarded args.
Tauri CLI then forwards that `--` to cargo, which rejects `--bundles`
with "unexpected argument". Invoking `pnpm exec tauri build --bundles nsis`
skips the script indirection so no `--` is emitted.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Switches the Tauri updater endpoint from GitHub Releases to a static
host. New Ed25519 pubkey (old private key was lost); existing 0.10.x
installs need one manual reinstall to pick up the new updater identity.
Release flow is now pnpm release <version> <notes> which bumps,
builds + signs locally, scps artifacts to the server, commits, tags.
GitHub workflow stays as workflow_dispatch backup (Windows only).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
