Strip the sourceId-gated chromeMediaSource + xcap native capture paths
from startScreenShare and collapse to a single setScreenShareEnabled
call. Neither of the bypassed paths produced smooth frames in WebView2:
chromeMediaSource: 'desktop' is an extension-only Chromium constraint
and throws outside extension origins, and the xcap JPEG-over-IPC
fallback couldn't sustain 30fps at 1080p on a single main-thread.
setScreenShareEnabled goes through Chromium's native getDisplayMedia
capture, which is the only path that gets HW-accelerated frames into
the WebRTC encoder from WebView2.
Audio continues via the WASAPI loopback module — getDisplayMedia can't
grab system sound in WebView2 without desktop-capture entitlements
Chromium reserves for extensions. The audio track's teardown chains to
the ScreenShare video track's 'ended' event so the Windows stop-share
overlay kills both sides in lockstep.
ScreenSourcePicker is now a quality + audio chooser only; the
thumbnail grid disappears because custom source IDs don't round-trip
through WebView2, and a custom picker in front of the OS picker just
means the user picks twice.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The fallback capture path was silently capping any 60fps preset to
30 because the hardcoded clamp never got updated when the 60fps presets
landed. Also syncs Cargo.lock that drifted against 0.11.0 metadata.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Release script bumped Cargo.toml but cargo only refreshes the lock on
the next build. Aligning them so the lock doesn't drift across tags.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Hooks the custom screen-share picker up to a native WASAPI loopback
capture so "Mit System-Sound" no longer falls back to the OS picker on
Windows. Rust side opens the default render endpoint, channels 48 kHz
f32 stereo to an AudioWorklet, which feeds a MediaStreamDestination for
LiveKit to publish as ScreenShareAudio. Ring buffer sized for latency
(80 ms target, drop-to-target on overflow) and the AudioContext is
resumed eagerly so initial burstiness can't pile up.
Adds a temporary attachTrack:audio diagnostic log to confirm source
tagging matches between old and new clients.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previous commit had the command typed as Result<tauri::ipc::Response,
String>. Turns out that forces Tauri to JSON-serialise the variant
wrapper around the Response body — the frontend gets a JSON object
instead of the raw ArrayBuffer, the runtime check for byteLength fails,
and every thumbnail comes back as null.
Changed the return type to `tauri::ipc::Response` directly. Bad source
ids and capture failures now funnel into an empty byte buffer; the JS
side still detects "no thumbnail" via `byteLength === 0` so the
contract stays the same.
Frontend also widens the invoke-result typing to ArrayBuffer |
Uint8Array | number[] so an older WebView2 that happens to deserialise
as an array still works, and normalises into a plain ArrayBuffer
before constructing the Blob to sidestep a TS SharedArrayBuffer
incompatibility.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Picker still stuttered during load because the main thread was stuck
parsing 20+ inbound IPC messages, each carrying 15-25 KB of JSON-wrapped
base64. Two changes compound to fix this:
1. Binary IPC. New Rust command capture_screen_source_thumbnail_bytes
returns `tauri::ipc::Response` with the raw JPEG bytes — no JSON
envelope, no base64 on either side. The frontend wraps the arriving
ArrayBuffer in a Blob and exposes it via URL.createObjectURL so the
browser decodes directly from bytes without a data-URL parse.
Empirically drops per-arrival main-thread work from ~10-15 ms to
~1-2 ms.
2. rAF-batched thumbnail state updates. Arriving blob URLs are staged in
a pendingUrls map and flushed in a single setState on the next
animation frame — multiple arrivals in one frame coalesce into one
render instead of queueing consecutive long tasks. Kept startTransition
on top so the commit stays on the low-priority lane.
Thumbnails are also dropped to 192×108 / Q60 (from 240×135 / Q70) for
~2× smaller payloads. Blob URLs get revoked on picker close so native
buffers don't leak across opens.
SourceCard now takes `thumbnailUrl` as a separate prop from a parent-
held map. Keeps source object references stable so React.memo's
identity check only fires a card re-render when THAT card's URL
actually lands, instead of every card whenever any URL changes.
Next session: WASAPI loopback for system-audio capture in native share.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The picker still felt frozen while thumbnails were streaming in because
each result was both (a) large — PNG @ 320×180 landed at 60-150 KB
base64 — and (b) triggering a high-priority React re-render of the whole
grid. Three fixes together restore interactivity:
- Thumbnails encoded as JPEG @ Q70 at 240×135 instead of PNG @ 320×180.
Drops the typical payload from ~100 KB to ~20 KB, so IPC JSON-parsing
on arrival is 5× faster.
- SourceCard wrapped in React.memo so only the card whose thumbnail just
landed re-renders. Previously one new thumbnail caused all ~20 cards
to re-evaluate their props.
- setSources updates run inside startTransition so scroll / click events
stay on the high-priority lane while the grid backfills.
Also: when the user enables "Sound mit übertragen" AND has a source
picked, the picker now surfaces an inline amber note explaining that
the OS picker will appear for the audio capture path. Matches the
existing console info log but is visible pre-click so users don't
experience it as a bug.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Picker speed (Phase 1+2):
- screen_sources.rs split into list_screen_sources (metadata only,
returns in ~10ms) + capture_screen_source_thumbnail (single source,
by id). ScreenSourcePicker now shows names + placeholders instantly
and streams thumbnails in as each capture lands. Total wall-clock
is bounded by the slowest source instead of the serial sum.
- enumerate_screen_sources kept as a dead_code fallback so any
rollout regression can switch the frontend back without code loss.
Native capture (Phase 3):
- New src-tauri/src/screen_capture.rs. start_screen_capture spawns a
Rust thread per share that grabs frames via xcap, downscales to the
user's quality preset, JPEG-encodes at Q72, and streams each frame
through a Tauri Channel<FramePayload>. stop_screen_capture signals
the stop flag and joins the worker.
- Worker re-resolves the xcap handle inside the thread because
xcap::Window holds a !Send HWND — passing the source id string
across the thread boundary sidesteps that.
- New lib/screenCapture.ts: decodes each frame into an ImageBitmap,
draws to an offscreen canvas, exposes canvas.captureStream() as the
MediaStream LiveKit publishes. Latest-wins frame queue drops stale
frames when the JS side falls behind the Rust producer. 3s first-
frame timeout so a silently-failing source (locked screen, DRM
window) surfaces as a clean NativeCaptureUnavailable and we fall
back to getDisplayMedia.
- CallContext.startScreenShare takes the native path first when the
picker provided a sourceId and system audio wasn't requested. The
old chromeMediaSourceId attempt and final setScreenShareEnabled
fallback stay in place for the audio case + non-Tauri runtimes.
- stopScreenShare kills the native handle first, then unpublishes any
manually-published ScreenShare/ScreenShareAudio tracks, then falls
back to setScreenShareEnabled(false). disconnectRoom also stops
the handle so we don't leak Rust threads across calls.
Scope note: native path is video-only. System-audio capture needs
WASAPI-loopback (Windows) or ScreenCaptureKit-audio (macOS); until
those are wired, requesting audio in the picker falls through to
the getDisplayMedia path and shows the OS picker for that one case.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Rust side:
- New src-tauri/src/screen_sources.rs with an enumerate_screen_sources
command. Uses the xcap crate for cross-platform screen + window
enumeration and capture; PNG thumbnails are letterbox-scaled to fit
320x180 and returned as base64.
- Source ids emit Chromium's internal desktopCapturer format
("screen:<id>:0", "window:<hwnd>:0") so JS can try passing them
straight into chromeMediaSourceId.
- Registered in both invoke_handler branches in lib.rs.
Frontend:
- New lib/screenSources.ts — Tauri command wrapper + thumbnailDataUrl
helper for the picker UI.
- New components/ScreenSourcePicker.tsx — Discord-style grid: sources
grouped under "Bildschirme" / "Fenster", large thumbnail cards with
selection state, quality preset + system-audio toggle in the footer.
"Teilen" button is enabled either way; without a selection it says
"Ohne Auswahl weiter" and falls through to the OS picker.
- Replaces the old form-style ScreenShareDialog entirely (removed).
CallContext wiring:
- startScreenShare now accepts an optional sourceId. When set, it
captures that exact source via getUserMedia's legacy
chromeMediaSourceId constraint and publishes the resulting tracks
manually (video as ScreenShare, audio as ScreenShareAudio). Falls
back to setScreenShareEnabled if WebView2 rejects the constraint,
so users always get a working share even if the direct path fails.
- Track 'ended' listeners unpublish the pub when the OS revokes
capture (close of shared window, OS "stop sharing" banner).
InCallPanel:
- Left-click on the share button now opens the picker instead of
starting with last-saved settings; right-click opens it too. The
picker itself is the 1-click UX.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Switches the Tauri updater endpoint from GitHub Releases to a static
host. New Ed25519 pubkey (old private key was lost); existing 0.10.x
installs need one manual reinstall to pick up the new updater identity.
Release flow is now pnpm release <version> <notes> which bumps,
builds + signs locally, scps artifacts to the server, commits, tags.
GitHub workflow stays as workflow_dispatch backup (Windows only).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Red-dot overlay drawn as raw RGBA (no extra resource bundled).
Shown on Windows when unread count > 0, cleared when 0.
macOS keeps numeric Dock badge; Linux has no cross-DE badge API.
Bumps version 0.10.0 -> 0.10.1.
Phase B.1 of the native-livekit migration. Ships the command surface and
event bridge behind a cargo feature so the default build stays unaffected
while the JS-SDK path keeps running in production.
Rust side
- livekit 0.7 (default tokio runtime, rustls-tls-native-roots) pulled as
an optional dependency; tokio also optional under the same feature
- Feature `rust-livekit` gates everything — off by default; on via
`cargo build --features rust-livekit`
- src-tauri/src/livekit_bridge.rs: LivekitState mutex, connect /
disconnect / send_data commands, event pump for room_state,
participant_joined, participant_left, data_received
- Mic / camera / screen share commands stubbed with explicit
"not implemented" errors so JS callers fail loudly rather than
silently no-op
JS side
- src/lib/nativeLiveKit.ts exposes a NativeRoom class with the same event
/ method shape the CallContext will need, plus a VITE_USE_RUST_LIVEKIT
flag so the adapter can be swapped once the bridge reaches parity
- isRustLivekitAvailable() gates access at both env + runtime layers
Build impact
- Baseline build unchanged (1s incremental, no new deps pulled)
- Feature build initial: ~10min (libwebrtc download + link)
- Feature build incremental: ~1s
- Binary size with feature: +12-20MB vs baseline
Open questions (documented for the next phase)
- Video-frame rendering bridge remains an upstream gap; livekit-rust
exposes NativeVideoFrame but no stable path to expose that as a
MediaStreamTrack inside the WebView
- Audio-only rust path is realistic near-term; full-rust needs either
upstream video-bridge or a native-overlay render window
Phase A of the crypto/livekit rust-native migration.
Rust side
- dryoc crate (pure-rust libsodium-compat, no C toolchain)
- Tauri commands: crypto_random_bytes, crypto_secretbox_encrypt/decrypt,
crypto_box_keypair, crypto_box_encrypt/decrypt, crypto_box_seal/open,
crypto_pwhash — all bit-compatible with libsodium-wrappers-sumo
- Commands registered via invoke_handler in lib.rs
- All IPC payloads base64-encoded to survive serde_json
JS side
- lib/nativeCryptoOps.ts exposes pwhashArgon2id + randomBytesAsync
plus optional secretbox accelerators for future call-site migration
- Native-first, WASM fallback on error or when VITE_USE_NATIVE_CRYPTO is
false / in browser preview
- Argon2id call-sites migrated: secureFileStore.deriveKey and
deviceBackup.deriveKey (covers vault unlock + backup/recovery flows)
Impact
- Vault unlock: ~1200ms → ~200ms (measured locally, Argon2id moderate)
- Per-message AEAD left on WASM-worker path: IPC overhead ~40µs would
dominate any native speedup below ~100µs/op
- WASM stays installed as graceful fallback so browser-preview builds
keep working and a native failure self-heals at runtime
System tray (desktop)
- tauri tray-icon feature + tray with menu (Öffnen/Ausblenden/Beenden)
- Left-click toggles main window; right-click shows menu
- JS emits tray-unread-update event, Rust mirrors into tooltip +
macOS dock badge via set_badge_label
- ConversationsContext wires totalUnread → tray
Window state persistence
- tauri-plugin-window-state (desktop-only target guard)
- Auto-restore size/position/maximized between restarts
Local SQLite message cache
- tauri-plugin-sql hydration of conversation view on mount
- persistMessages after each refresh, deleteCachedMessage on realtime
DELETE, pruneCache keeps latest 1000 per conversation
- Stores plaintext only (same trust boundary as stronghold device
key; cache never leaves the device, E2EE w.r.t. server unchanged)
Web Worker for decryption
- workers/decrypt.worker.ts runs crypto_secretbox_open_easy + utf-8
decode off the main thread with its own libsodium instance
- lib/decryptWorker.ts is a request/response wrapper with inline
fallback when Worker spawn fails
- shared decryptMessages accepts aeadBatchDelegate so key lookup
stays on the main thread while the AEAD loop offloads
Build fix
- Enable tauri tray-icon feature
- Import Listener + Manager traits, clone tray handle for the
event listener, conditional icon attach
File variety (video/pdf/generic), user status with custom message,
auto online/offline, DND gate for ring + notifications, drag-drop +
paste upload, @mentions in groups, link previews, emoji picker,
crash recovery.
Edit message decrypt:
- handleUpdate in useConversationMessages now refetches the canonical row
via REST after a realtime UPDATE instead of trusting the realtime
payload's bytea encoding. Same pattern as handleInsert — base64 vs
`\x…` hex serialisation varies across supabase/postgrest versions and
was silently producing undecryptable ciphertext for edited messages
on the receiver side
Windows WebView2 background throttling:
- ConversationsContext, useFriendships and useConversationMessages now
listen for visibilitychange / focus / online events and trigger both a
fresh REST refresh and a best-effort channel.subscribe() on wake.
WebView2 aggressively throttles background WebSockets and was dropping
realtime events entirely while the window was minimised, so new
messages and friend acceptances only surfaced after a manual reload
Bump tauri version 0.7.0 -> 0.7.1
Audio device selection:
- audioSettings: persisted inputDeviceId + outputDeviceId
- CallContext: uses stored input deviceId on mic enable, new
setAudioInputDevice / setAudioOutputDevice actions that hot-swap
without reconnect. Output swap applies HTMLMediaElement.setSinkId
to every attached remote-audio element (LiveKit's switchActiveDevice
only tracks elements it attached itself)
- SettingsPage: new "Mikrofon" + "Ausgabegerät" selects with
enumerateDevices, devicechange listener, permission-probe button.
setSinkId-unsupported fallback is messaged but non-blocking
Fullscreen:
- FullscreenCall was absolute inset-0 z-40 which trapped it inside the
<main> pane — sidebar + chat-list stayed visible. Switched to
fixed inset-0 z-[60] so the call overlays the whole window
Discord-style
- ScreenShareViewer fullscreen: CSS-only toggle (native Fullscreen API
unreliable under Tauri WKWebView), portalled to document.body when
active so no ancestor stacking context can clip it. Esc exits
ActiveCallBanner:
- cleanup effect returned early when presence was entirely empty,
leaving the "1 im Raum" fallback stuck after both peers left. Now
schedules dismissLastCall as soon as othersIn.length === 0, with a
3s grace window to absorb presence re-sync flicker
Bump tauri version 0.6.0 -> 0.7.0
Messages:
- Reply-to: hover action, composer chip with cancel, quote bubble inside
the replying message with tap-to-jump + amber highlight ring
- Search: header search button toggles in-conversation search bar with
prev/next + match counter, auto-jump to active match
- Forward: multi-select conversation picker. Attachments are now carried
over: download + decrypt source, re-encrypt under each target conv-key,
re-upload with fresh per-attachment keys, insert new attachment rows
Conversations:
- Archive + mute per member. New migration 20260420000001 adds `archived`
+ `muted_until` on conversation_members. Shared helpers:
setConversationArchived / setConversationMutedUntil / isConversationMuted
- ChatsPage: archive toggle in header with unread badge for archived
bucket, split active/archived lists, muted indicator (BellOff icon,
dimmed unread badge)
- ConversationRowMenu via createPortal (escapes sidebar overflow clip),
forwardRef-based MenuItem so submenu positioning refs survive React 18
- ConversationsContext: suppresses notification sound + OS notif when
target conversation is muted
- Refresh on `profiles UPDATE` realtime so peer avatar / displayName
changes flow to conversation.members without manual refresh
Resilience:
- ErrorBoundary (Discord-style): centred spinner + escalating copy, no
manual reload button. Backoff retry schedule [2s, 4s, 8s, 15s, 30s].
Uses a keyed Fragment (not a div wrapper) so flex h-full chains survive
- App wrapped root + per-route RouteBoundary, conversation-level boundary
- AuthContext: flip `ready` immediately on cached session read; validate
getUser in background so a stalled/offline Supabase doesn't freeze the
app on the loading spinner
Crypto:
- Swap libsodium-wrappers -> libsodium-wrappers-sumo (compact build was
missing crypto_pwhash so Argon2id vault KDF threw, falling back to
plaintext localStorage on every launch)
- Shim d.ts for sumo types (sumo is API superset, no official types ship)
- vite optimizeDeps includes sumo with the "require" condition
- secureFileStore: exists(dir) check before mkdir; surface genuine
permission errors instead of silent catch
Tauri:
- fs scope adds `$APPLOCALDATA` direct entry (not just /**) so the app
data directory itself can be mkdir'd on first launch
Chat layout:
- Skip call_event messages when computing avatar run boundaries so a
regular bubble followed by a call event from the same sender still
shows its avatar
Visual:
- Light/dark theme via ThemeContext + CSS vars
- New design tokens (surface/fg/accent/line/etc.) across all pages
- Reusable Avatar component (img + letter fallback) wired into UserBar,
ConversationHeader, ChatsPage, FriendsPage, GroupInfoPanel, IncomingCallPanel
Calls:
- Split call UI: IncomingCallPanel, InCallPanel, CallControls,
CallParticipantTile, ScreenShareViewer
- Active speaker hook (useActiveSpeakers)
- Fix: ActiveCallBanner stayed hidden after hangup while peers in room.
- useCallPresence: bind presence callbacks only when we own subscribe
(Supabase forbids .on() after .subscribe() on shared dedup'd channels)
- useCallPresence: never removeChannel — channel is shared with CallContext
so tearing it down on ConversationHeader unmount killed live tracking
- ActiveCallBanner: lastCallConversationId fallback so banner shows
instantly after hangup, auto-dismiss when room confirmed empty
- Drop unused useAnyActiveCall
Bump tauri version 0.5.0 -> 0.6.0
byGalax
byGalax