Audit of write-actions revealed that send (and edit via realtime UPDATE) are
already optimistic via local state insertion in `useConversationMessages`,
and friend nicknames are pure-local localStorage. Five user-write actions
were waiting on the ~100-200 ms server roundtrip + realtime echo:
* Toggle mute (`setConversationMutedUntil`)
* Toggle mentions-only (`setConversationMentionsOnly`)
* Toggle archive (`setConversationArchived`)
* Pin / unpin message (`pinMessage` / `unpinMessage`)
* Revoke device (`revokeDevice` RPC)
All five now flip local state synchronously and roll back on failure. The
existing realtime subscriptions reconcile canonically (no-op when the
optimistic patch already matches the server row), so this is purely a UX
latency improvement — no protocol or persistence changes.
Reactions (`toggleReaction` / `voteExclusive`) were intentionally skipped
this round: rollback semantics for the exclusive-vote path with multiple
sequential awaits are messy enough to warrant a dedicated pass.
PIN-unlock used to freeze the renderer for ~1-2 s on mid-hardware while
the moderate-preset Argon2id KDF + sealed-key secretbox open ran on the
main thread. Push that work into a Vite-bundled ESM Web Worker so the
unlock screen stays responsive.
The worker (apps/desktop/src/workers/crypto.worker.ts) bundles its own
libsodium-wrappers-sumo instance and registers a fresh CryptoBackend
inside the worker realm. Client wrapper (apps/desktop/src/lib/cryptoWorker.ts)
spawns a one-shot worker per unlock — workers are cheap, PIN-unlock is
once-per-session, and one-shot avoids the request-id bookkeeping that the
existing decrypt.worker needs for high-volume per-message decrypts.
Falls back to inline main-thread openUserKey when the Worker constructor
is unavailable (vitest's jsdom) or when worker spawn / round-trip fails
(strict CSP). All 14 desktop + 71 shared tests still pass — the existing
loadOrUnlockUserKey test exercises the inline-fallback branch.
Private key bytes are transferred (zero-copy) back to the main thread,
detaching the worker-side ArrayBuffer view on transfer.
Build emits crypto.worker-<hash>.js (~2.5 MB, mostly libsodium WASM glue
duplicated from the main bundle). Acceptable trade-off for the unblocked
UI; a future change could lazy-load libsodium on the main thread to drop
the duplication.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds opt-in (default OFF) auto-lock: after X minutes of no user input
the app calls signOut() (full memory wipe + PIN re-entry on next open).
Settings dropdown (Aus / 5 / 15 / 30 / 60 min) lives in SecurityCenter
below the existing wipe-on-close toggle. The idle timer is mounted in
AppShell via useIdleAutoLock; activity events are throttled to 1 Hz to
avoid timer thrash on rapid mouse movement. The localStorage key is
added to PRESERVE_LOCAL_STORAGE so a wipe never silently disables the
feature.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Google closed Tenor v2 to new API clients in Jan 2026, so the only
people who could use the picker were those with a pre-existing Google
Cloud Console key. Swapped to GIPHY's Developer API (still open, free
keys at https://developers.giphy.com/dashboard/).
- Env var renamed VITE_TENOR_API_KEY → VITE_GIPHY_API_KEY
- Endpoint, response mapping, error sentinel updated
- File still named tenor.ts for import-path stability — renaming
later if it bothers anyone
- Public GifResult interface unchanged so the picker UI didn't need
edits beyond the error-message switch
The 0.18.1 fix relied on an existing-device + present-stronghold-key match.
That fails for users who:
- had multiple device registrations and only retain the latest device's
private key in the local vault
- had a vault wipe / fresh OS install at some point
- have device rows that vanished server-side but keys still locally
Migration now scans conversation_keys for distinct un-migrated
recipient_device_ids visible to the user (RLS-filtered) and probes the
stronghold for each, regardless of whether the server still lists that
device. Result struct surfaces attempted/migrated/noKey/decryptFail/rpcFail
counters; SecurityCenter shows them via a new "Migration erneut ausführen"
button so users can self-diagnose without DevTools.
Also adds [crypto-migration] console.info breadcrumbs at every decision
point so a single F12 shows what happened.
PostgREST translates .eq('col', null) to `col = NULL` which is always false
in SQL. The migration silently returned zero rows -> setupNewUserIdentity
fired but re-wrapped nothing -> users could set a PIN but every send threw
'Awaiting key'. Switching to .is('col', null) emits `col IS NULL` and the
migration finally finds its work.
Also makes the migration trigger idempotent and re-fires it on:
- every successful loadOrUnlockUserKey
- AuthContext startup when the user-key is already cached
so users stuck on 0.18.0 auto-recover the moment they install 0.18.1.
PinInput: focused + active-slot now show a brand-coloured ring, glow, and
a blinking caret so users see where the next keystroke lands.
devices rows no longer carry public_key for crypto purposes. The whole
per-device key API surface (provisionNewDevice, loadDevicePrivateKey,
saveDevicePrivateKey, forgetDevicePrivateKey, restoreDeviceFromServerRecord)
is removed; registerDevice now records {name, platform} only. SQL drops the
NOT NULL on devices.public_key so future telemetry rows can omit it.
Note: SQL not applied locally - push via pnpm prod:migrate when ready.
When a conversation opens, the local client checks every accepted member
for a recipient_user_id bundle on the active key version. Members without
one get a best-effort wrap from the local conv-key handle. This closes
the legacy migration gap where peer B couldn't read because no one had
yet wrapped the new per-user conv-key for them.
Drops the manual backup-string flow; replaces it with PIN change,
recovery-code regeneration, and identity reset (all sealed via the new
user_keys table).
Replaces the per-device DeviceRecord lookup with a per-user discriminated
union (loading | needs-setup | needs-unlock | unlocked). Heartbeat block
deleted (telemetry no longer device-bound); webPush keyed by install-id.
Renames DecryptParams.ownDeviceId to ownUserId so decryptMessages actually
looks up bundles by user. Sweeps remaining OwnDeviceCtx and
loadDevicePrivateKey consumers in the desktop app to use cachedUserKey
from userIdentity. Files scheduled for deletion in later tasks
(BackupExportDialog, DeviceRestore, BackupRestoreDialog, BackupPromptBanner,
deviceBackup, DeviceRegistration) are left untouched.
App productName becomes Netralax (driving exe name and window title);
existing installs keep their %APPDATA%\ChatApp profile via an explicit
app.setPath('userData', appData/ChatApp) so no Login/Sounds/Secret store
data is lost.
The Windows taskbar overlay now renders a red bubble with the actual
unread count (Discord parity) instead of just a static red dot. Renderer
paints a 64×64 PNG via canvas — full-bleed red circle, white bold count
with a "99+" cap, no outer ring — and passes the data URL through the
existing setTrayUnread IPC. Main decodes via nativeImage and applies it
as the BrowserWindow overlay icon. Falls back to the static dot if the
renderer canvas pipeline is unavailable.
Also: app.setName('Netralax') + setAppUserModelId('cloud.netralax.desktop')
for Windows taskbar grouping and notification source attribution, and
release.mjs now reads productName dynamically from package.json so the
artifact lookup stays correct after the rename.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
After 0.16.2 some users saw the approval banner stack up to 6+ entries
on first launch — every old device they ever registered (Tauri-era,
test installs, dev builds) showed up because the only "already legit"
filter was `created_at <= ownDevice.created_at`. That fails when own
device is restored from Backup (older than every other entry) or when
the user accumulated installs around the migration window.
Add a semantic check: if a device already has at least one row in
`conversation_keys` (recipient_device_id), it has been wrapped before
and is by definition not awaiting approval. Treat as approved silently.
Bulk query against the candidate IDs, no N+1.
Plus UX: when more than one request is pending, render a sticky header
with a count and "Alle ablehnen" button so users with stale piles can
clear them in one click.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Disable the previous auto-share of conversation keys to newly-registered
devices: a stolen password / new device registered by an attacker no
longer automatically grants history access. Backup-Restore (which
restores the old device-id) still opens existing wraps as before.
Phase 1 of the approval replacement:
- New `lib/deviceApproval.ts`: realtime listener for `devices` INSERT,
surfaces a pending list, persists approve/deny decisions in
`chatapp.approvedDeviceIds` / `chatapp.dismissedDeviceIds`. Filters the
initial fetch by created_at > own-device's created_at so a freshly
installed client doesn't try to "approve" pre-existing devices.
- New `components/DeviceApprovalBanner.tsx`: bottom-right Discord-style
banner per pending request with Genehmigen / Ablehnen actions; reuses
`wrapForOneDevice` from conversationKeySync to fan out conv-keys.
- AppShell mounts both the listener and the banner.
Plus dev userData isolation in main.ts: when running unpackaged, append
`-Dev` to the userData path so `pnpm dev` runs side-by-side with the
installed packaged build instead of colliding on the single-instance
lock. Window title also distinguished as "ChatApp (Dev)".
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The real cost in the "frame by frame" stutter wasn't the custom capture
path — it was LiveKit re-encoding via VP9 software with L3T3_KEY SVC
(three spatial × three temporal layers, all CPU). Switching the
per-publish codec to H.264 lets Chromium's hardware encoder take over
on Windows and sidesteps the SVC mode entirely (H.264 has no SVC).
Also pushes `contentHint = 'detail'` on the track — setScreenShareEnabled
does this internally, the manual publishTrack paths had been missing it,
which changes how the encoder allocates its frame budget for static UI
content.
Auto preset default framerate 60 → 30. 60fps desktop share burns three
full-res encodes per frame at sizes up to 4K; 30 is what getDisplayMedia
practically delivers anyway.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Hooks the custom screen-share picker up to a native WASAPI loopback
capture so "Mit System-Sound" no longer falls back to the OS picker on
Windows. Rust side opens the default render endpoint, channels 48 kHz
f32 stereo to an AudioWorklet, which feeds a MediaStreamDestination for
LiveKit to publish as ScreenShareAudio. Ring buffer sized for latency
(80 ms target, drop-to-target on overflow) and the AudioContext is
resumed eagerly so initial burstiness can't pile up.
Adds a temporary attachTrack:audio diagnostic log to confirm source
tagging matches between old and new clients.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Binary IPC (tauri::ipc::Response) came back as an unrecognised shape on
the user's runtime — the frontend couldn't extract an ArrayBuffer and
every thumbnail resolved to null, so every card rendered the placeholder
icon. Added:
- Widened the invoke typing to ArrayBuffer | Uint8Array | number[] so
all three known Tauri/WebView2 deserialisation shapes parse.
- A one-time console.warn when the Response body lands as an unknown
object shape, so the real wire format can be diagnosed if this ever
trips again.
- An automatic tier-2 fallback: if the binary path produced 0 usable
bytes, re-invoke the legacy base64 command and decode client-side.
Slower on the JS thread than binary IPC but known to work across all
Tauri 2.x runtimes.
Net behaviour: thumbnails render again. If the binary path works on a
given build, we get the fast path; otherwise the base64 fallback keeps
the picker usable.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previous commit had the command typed as Result<tauri::ipc::Response,
String>. Turns out that forces Tauri to JSON-serialise the variant
wrapper around the Response body — the frontend gets a JSON object
instead of the raw ArrayBuffer, the runtime check for byteLength fails,
and every thumbnail comes back as null.
Changed the return type to `tauri::ipc::Response` directly. Bad source
ids and capture failures now funnel into an empty byte buffer; the JS
side still detects "no thumbnail" via `byteLength === 0` so the
contract stays the same.
Frontend also widens the invoke-result typing to ArrayBuffer |
Uint8Array | number[] so an older WebView2 that happens to deserialise
as an array still works, and normalises into a plain ArrayBuffer
before constructing the Blob to sidestep a TS SharedArrayBuffer
incompatibility.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Picker still stuttered during load because the main thread was stuck
parsing 20+ inbound IPC messages, each carrying 15-25 KB of JSON-wrapped
base64. Two changes compound to fix this:
1. Binary IPC. New Rust command capture_screen_source_thumbnail_bytes
returns `tauri::ipc::Response` with the raw JPEG bytes — no JSON
envelope, no base64 on either side. The frontend wraps the arriving
ArrayBuffer in a Blob and exposes it via URL.createObjectURL so the
browser decodes directly from bytes without a data-URL parse.
Empirically drops per-arrival main-thread work from ~10-15 ms to
~1-2 ms.
2. rAF-batched thumbnail state updates. Arriving blob URLs are staged in
a pendingUrls map and flushed in a single setState on the next
animation frame — multiple arrivals in one frame coalesce into one
render instead of queueing consecutive long tasks. Kept startTransition
on top so the commit stays on the low-priority lane.
Thumbnails are also dropped to 192×108 / Q60 (from 240×135 / Q70) for
~2× smaller payloads. Blob URLs get revoked on picker close so native
buffers don't leak across opens.
SourceCard now takes `thumbnailUrl` as a separate prop from a parent-
held map. Keeps source object references stable so React.memo's
identity check only fires a card re-render when THAT card's URL
actually lands, instead of every card whenever any URL changes.
Next session: WASAPI loopback for system-audio capture in native share.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The picker still felt frozen while thumbnails were streaming in because
each result was both (a) large — PNG @ 320×180 landed at 60-150 KB
base64 — and (b) triggering a high-priority React re-render of the whole
grid. Three fixes together restore interactivity:
- Thumbnails encoded as JPEG @ Q70 at 240×135 instead of PNG @ 320×180.
Drops the typical payload from ~100 KB to ~20 KB, so IPC JSON-parsing
on arrival is 5× faster.
- SourceCard wrapped in React.memo so only the card whose thumbnail just
landed re-renders. Previously one new thumbnail caused all ~20 cards
to re-evaluate their props.
- setSources updates run inside startTransition so scroll / click events
stay on the high-priority lane while the grid backfills.
Also: when the user enables "Sound mit übertragen" AND has a source
picked, the picker now surfaces an inline amber note explaining that
the OS picker will appear for the audio capture path. Matches the
existing console info log but is visible pre-click so users don't
experience it as a bug.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Volume crash:
- setParticipantVolume / setScreenShareVolume propagated values up to 2.0
(200%) to the per-track GainNode, but also called applyToAttachedElements
which set the raw HTMLAudioElement.volume — that property is hard-clamped
to [0, 1] and throws IndexSizeError above 1. Clip the element-path apply
at 1.0. WebAudio GainNode keeps doing the actual amplification.
Picker freeze:
- Firing ~20 captureScreenSourceThumbnail invokes in parallel caused
perceptible input freezes while each ~100KB base64 result arrived and
triggered a setState. Bounded the worker pool to 4 concurrent captures
with a queue — overall wall-clock is nearly identical and the grid stays
scrollable / clickable throughout the load.
Native-path diagnostics:
- Previous logs only fired on non-NativeCaptureUnavailable errors, so
users couldn't tell whether the native path was skipped (audio toggle
on, no sourceId) or attempted-and-failed. Added explicit info logs for
each skip reason plus an always-on warn with the underlying error when
the try block throws. Makes the next debug pass on screenshare much
quicker.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Picker speed (Phase 1+2):
- screen_sources.rs split into list_screen_sources (metadata only,
returns in ~10ms) + capture_screen_source_thumbnail (single source,
by id). ScreenSourcePicker now shows names + placeholders instantly
and streams thumbnails in as each capture lands. Total wall-clock
is bounded by the slowest source instead of the serial sum.
- enumerate_screen_sources kept as a dead_code fallback so any
rollout regression can switch the frontend back without code loss.
Native capture (Phase 3):
- New src-tauri/src/screen_capture.rs. start_screen_capture spawns a
Rust thread per share that grabs frames via xcap, downscales to the
user's quality preset, JPEG-encodes at Q72, and streams each frame
through a Tauri Channel<FramePayload>. stop_screen_capture signals
the stop flag and joins the worker.
- Worker re-resolves the xcap handle inside the thread because
xcap::Window holds a !Send HWND — passing the source id string
across the thread boundary sidesteps that.
- New lib/screenCapture.ts: decodes each frame into an ImageBitmap,
draws to an offscreen canvas, exposes canvas.captureStream() as the
MediaStream LiveKit publishes. Latest-wins frame queue drops stale
frames when the JS side falls behind the Rust producer. 3s first-
frame timeout so a silently-failing source (locked screen, DRM
window) surfaces as a clean NativeCaptureUnavailable and we fall
back to getDisplayMedia.
- CallContext.startScreenShare takes the native path first when the
picker provided a sourceId and system audio wasn't requested. The
old chromeMediaSourceId attempt and final setScreenShareEnabled
fallback stay in place for the audio case + non-Tauri runtimes.
- stopScreenShare kills the native handle first, then unpublishes any
manually-published ScreenShare/ScreenShareAudio tracks, then falls
back to setScreenShareEnabled(false). disconnectRoom also stops
the handle so we don't leak Rust threads across calls.
Scope note: native path is video-only. System-audio capture needs
WASAPI-loopback (Windows) or ScreenCaptureKit-audio (macOS); until
those are wired, requesting audio in the picker falls through to
the getDisplayMedia path and shows the OS picker for that one case.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
HTMLMediaElement.volume caps at 1.0, so boosting a quiet peer past
100% needs an explicit GainNode in the output chain. New
remoteAudioPipelines module owns one AudioContext + GainNode per
remote audio track; attachTrack / detachTrack now create and tear
down the pipeline alongside the LiveKit element.
Once a track is on the WebAudio path its direct output is diverted
(createMediaElementSource semantics), so audio.muted / volume can't
drive output anymore. Deafen, watch-state, manual screen-share mute
and per-user volume are collapsed into one effective-gain formula
that gets recomputed on every state flip — the effect subscribes to
both participantVolumes and screenShareVolumes for live slider drags.
Slider ranges updated to 0–200% across:
- ParticipantVolumeMenu (per-user right-click menu)
- ParticipantsPopover (in-call participant list)
- ScreenShareContextMenu (per-share right-click)
Values above 100% render the percentage in amber as a soft hint that
clipping is possible. Clamp in both volume stores extended to [0, 2]
so persisted values survive.
setAudioOutputDevice now additionally routes via
AudioContext.setSinkId (Chrome 115+) for the WebAudio graph; the
HTMLAudioElement.setSinkId fallback stays for older runtimes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Rust side:
- New src-tauri/src/screen_sources.rs with an enumerate_screen_sources
command. Uses the xcap crate for cross-platform screen + window
enumeration and capture; PNG thumbnails are letterbox-scaled to fit
320x180 and returned as base64.
- Source ids emit Chromium's internal desktopCapturer format
("screen:<id>:0", "window:<hwnd>:0") so JS can try passing them
straight into chromeMediaSourceId.
- Registered in both invoke_handler branches in lib.rs.
Frontend:
- New lib/screenSources.ts — Tauri command wrapper + thumbnailDataUrl
helper for the picker UI.
- New components/ScreenSourcePicker.tsx — Discord-style grid: sources
grouped under "Bildschirme" / "Fenster", large thumbnail cards with
selection state, quality preset + system-audio toggle in the footer.
"Teilen" button is enabled either way; without a selection it says
"Ohne Auswahl weiter" and falls through to the OS picker.
- Replaces the old form-style ScreenShareDialog entirely (removed).
CallContext wiring:
- startScreenShare now accepts an optional sourceId. When set, it
captures that exact source via getUserMedia's legacy
chromeMediaSourceId constraint and publishes the resulting tracks
manually (video as ScreenShare, audio as ScreenShareAudio). Falls
back to setScreenShareEnabled if WebView2 rejects the constraint,
so users always get a working share even if the direct path fails.
- Track 'ended' listeners unpublish the pub when the OS revokes
capture (close of shared window, OS "stop sharing" banner).
InCallPanel:
- Left-click on the share button now opens the picker instead of
starting with last-saved settings; right-click opens it too. The
picker itself is the 1-click UX.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Watch-gate lifted into CallContext. watchingShareUserIds /
dismissedShareUserIds / screenShareAudioMutedIds as session-only state,
cleared on CallState.idle and on TrackUnsubscribed for each sharer.
Survives layout changes (grid <-> focus <-> fullscreen) without
resetting which the old local-state viewer dropped on remount.
- ScreenShareAudio tracks tagged via data-track-source="screenshare" at
attach-time; initial muted follows watching + manual mute mirrors so
audio never plays before the user clicks "Bildschirm anschauen". Deafen
still wins at the top of the priority chain.
- New screenShareVolumes store (session-only, keyed by participantId).
attachTrack pulls the initial volume from this store for screenshare
audio elements so the context-menu slider takes effect immediately.
- Screen shares are no longer auto-promoted to focus. They render as
equal-size grid tiles like everyone else; user clicks to focus. The
"Bildschirm anschauen" overlay replaces auto-play as the opt-in.
- Dismissed sharer-ids filter out of buildTiles, so "Zuschauen beenden"
really hides the tile until the sharer stops + restarts.
- New ScreenShareContextMenu (portal, Esc / outside-click to close):
volume slider + audio mute toggle when the share has audio + a
destructive "Zuschauen beenden" row. Wired via a dispatcher in
InCallPanel that picks between participant-volume and share-menu
based on tile.kind.
- Fullscreen cinema gets a "Hide participant strip" toggle (top-right,
session-only) so focused content reaches the full viewport when the
bottom thumbnail row would otherwise steal 160px. Fades with the
auto-hide controls; only surfaces when there's a focus + peers to hide.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Default flipped to off so voice doesn't get coloured by browser NS on
first run. Users turn it on explicitly under Settings → Sprache, where
the toggle already lived before the in-call button shipped.
The in-call SparklesIcon button and all the associated wiring
(onToggleNoiseSuppression, noiseSuppression prop, local subscription
in InCallPanel) is removed. The hot-swap UX is preserved: a new
subscribeAudioSettings watcher in CallContext detects noiseSuppression
flips during an active call and re-runs setupMicPipeline so the change
takes effect without rejoining.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
byGalax