byGalax
1681 lines
56 KiB
Markdown
1681 lines
56 KiB
Markdown
# Phase 8 — Feature Batch Implementation Plan
|
||
|
||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||
|
||
**Goal:** Ship six independent feature improvements in one batch:
|
||
- **A1** Composer-drafts cross-restart per chat
|
||
- **A2** Voice-message playback-speed control
|
||
- **B1** View-once "Halten zum Sehen" + screenshot-block (Windows + macOS)
|
||
- **C1** Multi-user whiteboard live cursors
|
||
- **C2** Screen-share annotation overlay (shared between viewers)
|
||
- **D1** Soundboard hotkeys always-on (also fire outside calls, play locally)
|
||
- **D2** Animated avatars (GIF/APNG/animated-WebP upload bypasses the canvas re-encode)
|
||
|
||
**Architecture:** Each task is self-contained — different files, different domains, no shared state. They can be implemented and committed in any order.
|
||
|
||
**Tech Stack:** Electron 33, React 18, Supabase realtime (broadcast for cursors and annotations), better-sqlite3 (drafts), Web Audio API (voice speed, soundboard local play), Vitest.
|
||
|
||
---
|
||
|
||
## Pre-Flight: What's already there
|
||
|
||
The research phase before writing this plan confirmed:
|
||
|
||
- **Voice-waveform + scrub is already done** (`AttachmentAudio.tsx:68-128`) — only playback-speed is missing. Out of scope here.
|
||
- **Soundboard hotkeys are already wired** (`CallContext.tsx:2262`, `soundboardHotkeys.ts`) but gated on `state.kind === 'connected'`. Task D1 lifts that gate.
|
||
- **Multi-user whiteboard realtime works** (`useWhiteboardStrokes.ts:48-94` subscribes to `whiteboard_strokes` INSERT). Task C1 adds presence-channel live cursors on top; in-progress stroke broadcast is explicitly **out of scope** (deferred to a phase 9 if needed).
|
||
- **Animated-avatar display already works natively** in `<Avatar>` because `<img>` renders animated GIF/APNG/WebP. The only blocker is `avatarUpload.ts:9-44` which re-encodes via `canvas.toBlob` and strips animation. Task D2 bypasses that for animated MIME types.
|
||
- **View-once bug root cause** (`ViewOnceImage.tsx:55-63`): `setRevealedAt(res.viewedAt)` fires before `setFullscreen(true)`, React re-renders with `burned=true` and returns the tombstone early — the dialog never paints. Task B1 reorders + switches to a hold-to-view pattern.
|
||
|
||
---
|
||
|
||
# Group A — Messaging UX
|
||
|
||
## Task A1: Composer drafts cross-restart per chat
|
||
|
||
**Goal:** When the user types in a chat then switches away (or restarts the app), the typed text + reply target are restored on return. Discord-style.
|
||
|
||
**Files:**
|
||
- Create: `apps/desktop/src/lib/composerDraftStore.ts`
|
||
- Create: `apps/desktop/src/lib/composerDraftStore.test.ts`
|
||
- Modify: `apps/desktop/src/lib/messageCache.ts` (add a new table to the existing SQLite init)
|
||
- Modify: `apps/desktop/src/pages/ConversationPage.tsx` (initial `useState` for `text`, `replyTo`; write effect on change; clear on send)
|
||
- Modify: `apps/desktop/src/App.tsx` (call `hydrateDrafts()` once on app boot)
|
||
|
||
**Scope:**
|
||
- Persist: `text` (string) + `replyToId` (string | null) per `conversationId`. Survives app restart.
|
||
- Don't persist: pending file attachments (kept in-RAM only — Files don't serialize cleanly, session-scope is the pragmatic balance).
|
||
- Clear behaviour: cleared on successful send; preserved on cancel/escape; preserved when switching chats.
|
||
|
||
### Steps
|
||
|
||
- [ ] **Step 1: Add a `composer_drafts` table to the SQLite bootstrap**
|
||
|
||
In `apps/desktop/src/lib/messageCache.ts` inside `getHandle()` add to the schema bootstrap (after the existing `CREATE TABLE IF NOT EXISTS messages` block, around line 39):
|
||
|
||
```ts
|
||
await execute(
|
||
handle,
|
||
`CREATE TABLE IF NOT EXISTS composer_drafts (
|
||
conversation_id TEXT PRIMARY KEY,
|
||
text TEXT NOT NULL,
|
||
reply_to_id TEXT,
|
||
updated_at TEXT NOT NULL
|
||
);`,
|
||
);
|
||
```
|
||
|
||
- [ ] **Step 2: Write the failing test for the draft store**
|
||
|
||
Create `apps/desktop/src/lib/composerDraftStore.test.ts`:
|
||
|
||
```ts
|
||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||
|
||
import {
|
||
__resetForTests,
|
||
clearDraft,
|
||
getDraftSync,
|
||
hasDraft,
|
||
hydrateDrafts,
|
||
setDraft,
|
||
} from './composerDraftStore';
|
||
|
||
const sqlExecuteMock = vi.fn().mockResolvedValue(undefined);
|
||
const sqlSelectMock = vi.fn().mockResolvedValue([]);
|
||
const sqlLoadMock = vi.fn().mockResolvedValue('mock-handle');
|
||
|
||
vi.stubGlobal('window', {
|
||
electronAPI: {
|
||
sqlLoad: sqlLoadMock,
|
||
sqlExecute: sqlExecuteMock,
|
||
sqlSelect: sqlSelectMock,
|
||
},
|
||
});
|
||
|
||
describe('composerDraftStore', () => {
|
||
beforeEach(() => {
|
||
sqlExecuteMock.mockClear();
|
||
sqlSelectMock.mockClear();
|
||
sqlLoadMock.mockClear();
|
||
__resetForTests();
|
||
});
|
||
|
||
afterEach(() => {
|
||
__resetForTests();
|
||
});
|
||
|
||
it('returns null for an unknown conversation', () => {
|
||
expect(getDraftSync('unknown')).toBeNull();
|
||
expect(hasDraft('unknown')).toBe(false);
|
||
});
|
||
|
||
it('stores and returns a draft synchronously after set', () => {
|
||
setDraft('a', { text: 'hi', replyToId: null });
|
||
const draft = getDraftSync('a');
|
||
expect(draft).not.toBeNull();
|
||
expect(draft?.text).toBe('hi');
|
||
expect(draft?.replyToId).toBeNull();
|
||
expect(hasDraft('a')).toBe(true);
|
||
});
|
||
|
||
it('isolates drafts per conversation', () => {
|
||
setDraft('a', { text: 'one', replyToId: null });
|
||
setDraft('b', { text: 'two', replyToId: 'msg-9' });
|
||
expect(getDraftSync('a')?.text).toBe('one');
|
||
expect(getDraftSync('b')?.replyToId).toBe('msg-9');
|
||
});
|
||
|
||
it('clearDraft removes the draft from memory', () => {
|
||
setDraft('a', { text: 'one', replyToId: null });
|
||
clearDraft('a');
|
||
expect(getDraftSync('a')).toBeNull();
|
||
expect(hasDraft('a')).toBe(false);
|
||
});
|
||
|
||
it('treats an empty-string text + null reply as "no draft"', () => {
|
||
setDraft('a', { text: '', replyToId: null });
|
||
expect(getDraftSync('a')).toBeNull();
|
||
expect(hasDraft('a')).toBe(false);
|
||
});
|
||
|
||
it('hydrateDrafts populates the in-memory map from SQLite rows', async () => {
|
||
sqlSelectMock.mockResolvedValueOnce([
|
||
{ conversation_id: 'a', text: 'persisted', reply_to_id: 'msg-1', updated_at: '2026-05-17T00:00:00Z' },
|
||
]);
|
||
await hydrateDrafts();
|
||
expect(getDraftSync('a')?.text).toBe('persisted');
|
||
expect(getDraftSync('a')?.replyToId).toBe('msg-1');
|
||
});
|
||
});
|
||
```
|
||
|
||
- [ ] **Step 3: Implement the store**
|
||
|
||
Create `apps/desktop/src/lib/composerDraftStore.ts`:
|
||
|
||
```ts
|
||
// Composer-draft persistence. Two-tier semantics:
|
||
// * In-memory `Map<convId, Draft>` for instant synchronous reads on
|
||
// mount (mirrors the `messageMemoryCache` pattern from Phase 7).
|
||
// * SQLite (`composer_drafts` table, schema in `messageCache.ts`) for
|
||
// cross-restart persistence. Writes are debounced and fire-and-forget
|
||
// — losing the last 400ms of typing on a hard crash is acceptable;
|
||
// blocking the keystroke handler is not.
|
||
//
|
||
// Attachments are intentionally NOT serialized:
|
||
// * Files don't round-trip through SQLite cleanly (binary blobs blow
|
||
// up the cache size).
|
||
// * `replyToId` IS persisted; the consuming page looks up the actual
|
||
// message by id at render time.
|
||
|
||
import { isTauriRuntime } from './globalShortcut';
|
||
|
||
const DB_NAME = 'chatapp-cache';
|
||
const WRITE_DEBOUNCE_MS = 400;
|
||
|
||
interface Draft {
|
||
text: string;
|
||
replyToId: string | null;
|
||
}
|
||
|
||
interface DraftRow {
|
||
conversation_id: string;
|
||
text: string;
|
||
reply_to_id: string | null;
|
||
updated_at: string;
|
||
}
|
||
|
||
const drafts = new Map<string, Draft>();
|
||
const pendingWrites = new Map<string, ReturnType<typeof setTimeout>>();
|
||
let handlePromise: Promise<string | null> | null = null;
|
||
|
||
async function getHandle(): Promise<string | null> {
|
||
if (handlePromise) return handlePromise;
|
||
if (!isTauriRuntime()) {
|
||
handlePromise = Promise.resolve(null);
|
||
return handlePromise;
|
||
}
|
||
handlePromise = (async () => {
|
||
try {
|
||
return await window.electronAPI.sqlLoad({ name: DB_NAME });
|
||
} catch (err: unknown) {
|
||
console.warn('composerDraftStore: sqlLoad failed', err);
|
||
return null;
|
||
}
|
||
})();
|
||
return handlePromise;
|
||
}
|
||
|
||
export function getDraftSync(conversationId: string): Draft | null {
|
||
const stored = drafts.get(conversationId);
|
||
if (!stored) return null;
|
||
return { text: stored.text, replyToId: stored.replyToId };
|
||
}
|
||
|
||
export function hasDraft(conversationId: string): boolean {
|
||
return drafts.has(conversationId);
|
||
}
|
||
|
||
export function setDraft(conversationId: string, draft: Draft): void {
|
||
if (draft.text.length === 0 && draft.replyToId === null) {
|
||
if (drafts.has(conversationId)) {
|
||
drafts.delete(conversationId);
|
||
scheduleWrite(conversationId);
|
||
}
|
||
return;
|
||
}
|
||
drafts.set(conversationId, { text: draft.text, replyToId: draft.replyToId });
|
||
scheduleWrite(conversationId);
|
||
}
|
||
|
||
export function clearDraft(conversationId: string): void {
|
||
if (!drafts.has(conversationId)) return;
|
||
drafts.delete(conversationId);
|
||
scheduleWrite(conversationId);
|
||
}
|
||
|
||
function scheduleWrite(conversationId: string): void {
|
||
const existing = pendingWrites.get(conversationId);
|
||
if (existing) clearTimeout(existing);
|
||
const timer = setTimeout(() => {
|
||
pendingWrites.delete(conversationId);
|
||
void flushOne(conversationId);
|
||
}, WRITE_DEBOUNCE_MS);
|
||
pendingWrites.set(conversationId, timer);
|
||
}
|
||
|
||
async function flushOne(conversationId: string): Promise<void> {
|
||
const handle = await getHandle();
|
||
if (!handle) return;
|
||
const draft = drafts.get(conversationId);
|
||
try {
|
||
if (draft) {
|
||
await window.electronAPI.sqlExecute({
|
||
handle,
|
||
query:
|
||
`INSERT INTO composer_drafts (conversation_id, text, reply_to_id, updated_at)
|
||
VALUES ($1, $2, $3, $4)
|
||
ON CONFLICT(conversation_id) DO UPDATE SET
|
||
text = excluded.text,
|
||
reply_to_id = excluded.reply_to_id,
|
||
updated_at = excluded.updated_at`,
|
||
bindings: [conversationId, draft.text, draft.replyToId, new Date().toISOString()],
|
||
});
|
||
} else {
|
||
await window.electronAPI.sqlExecute({
|
||
handle,
|
||
query: 'DELETE FROM composer_drafts WHERE conversation_id = $1',
|
||
bindings: [conversationId],
|
||
});
|
||
}
|
||
} catch (err: unknown) {
|
||
console.warn('composerDraftStore: flush failed', err);
|
||
}
|
||
}
|
||
|
||
export async function hydrateDrafts(): Promise<void> {
|
||
const handle = await getHandle();
|
||
if (!handle) return;
|
||
try {
|
||
const rows = (await window.electronAPI.sqlSelect({
|
||
handle,
|
||
query: 'SELECT conversation_id, text, reply_to_id, updated_at FROM composer_drafts',
|
||
bindings: [],
|
||
})) as unknown as DraftRow[];
|
||
for (const r of rows) {
|
||
if (!r.conversation_id || (r.text.length === 0 && r.reply_to_id === null)) continue;
|
||
drafts.set(r.conversation_id, { text: r.text, replyToId: r.reply_to_id });
|
||
}
|
||
} catch (err: unknown) {
|
||
console.warn('composerDraftStore: hydrate failed', err);
|
||
}
|
||
}
|
||
|
||
export function __resetForTests(): void {
|
||
for (const t of pendingWrites.values()) clearTimeout(t);
|
||
pendingWrites.clear();
|
||
drafts.clear();
|
||
handlePromise = null;
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 4: Run tests to confirm 6/6 pass**
|
||
|
||
Run: `pnpm --filter @chat-app/desktop test -- composerDraftStore`
|
||
|
||
- [ ] **Step 5: Hydrate drafts on app boot**
|
||
|
||
In `apps/desktop/src/App.tsx`, inside the `App()` component (right at the top), add the boot effect. Add the import near the others:
|
||
|
||
```ts
|
||
import { hydrateDrafts } from './lib/composerDraftStore';
|
||
```
|
||
|
||
Inside `App()` before the `return`:
|
||
|
||
```ts
|
||
useEffect(() => {
|
||
void hydrateDrafts();
|
||
}, []);
|
||
```
|
||
|
||
Add `useEffect` to the React import on line 1.
|
||
|
||
- [ ] **Step 6: Wire the store into ConversationPage**
|
||
|
||
In `apps/desktop/src/pages/ConversationPage.tsx`, add to the imports near the other local-lib imports (around line 67):
|
||
|
||
```ts
|
||
import { clearDraft, getDraftSync, setDraft } from '../lib/composerDraftStore';
|
||
```
|
||
|
||
Replace the `text` state declaration (line 203):
|
||
|
||
```ts
|
||
const [text, setText] = useState('');
|
||
```
|
||
|
||
with:
|
||
|
||
```ts
|
||
const [text, setText] = useState<string>(() => {
|
||
if (!id) return '';
|
||
return getDraftSync(id)?.text ?? '';
|
||
});
|
||
```
|
||
|
||
Below the existing `replyTo` state (around line 229), add a resolver effect that maps a saved `replyToId` back to a `DecryptedMessage` once messages have loaded:
|
||
|
||
```ts
|
||
useEffect(() => {
|
||
if (!id) return;
|
||
const draft = getDraftSync(id);
|
||
const savedReplyToId = draft?.replyToId ?? null;
|
||
if (!savedReplyToId) return;
|
||
if (replyTo?.id === savedReplyToId) return;
|
||
const match = messages.find((m) => m.id === savedReplyToId);
|
||
if (match) setReplyTo(match);
|
||
}, [id, messages, replyTo?.id]);
|
||
```
|
||
|
||
Add a write effect that mirrors `text` + `replyTo` changes to the store:
|
||
|
||
```ts
|
||
useEffect(() => {
|
||
if (!id) return;
|
||
setDraft(id, { text, replyToId: replyTo?.id ?? null });
|
||
}, [id, text, replyTo?.id]);
|
||
```
|
||
|
||
In the `handleSend` function (around line 748), after the successful send right after `setText('')`, add:
|
||
|
||
```ts
|
||
if (id) clearDraft(id);
|
||
```
|
||
|
||
- [ ] **Step 7: Typecheck + tests**
|
||
|
||
```bash
|
||
pnpm --filter @chat-app/desktop typecheck
|
||
pnpm --filter @chat-app/desktop test
|
||
```
|
||
|
||
Both must pass.
|
||
|
||
- [ ] **Step 8: Commit**
|
||
|
||
```bash
|
||
git add apps/desktop/src/lib/composerDraftStore.ts apps/desktop/src/lib/composerDraftStore.test.ts apps/desktop/src/lib/messageCache.ts apps/desktop/src/pages/ConversationPage.tsx apps/desktop/src/App.tsx
|
||
git commit -m "feat(composer): persist text + reply target per chat across restarts"
|
||
```
|
||
|
||
---
|
||
|
||
## Task A2: Voice-message playback speed
|
||
|
||
**Goal:** Add a 1× / 1.5× / 2× toggle to the voice-message player. Remember last-used speed per user.
|
||
|
||
**Files:**
|
||
- Modify: `apps/desktop/src/components/AttachmentAudio.tsx` (add speed state + chip UI + applies to `audioRef.current.playbackRate`)
|
||
- Create: `apps/desktop/src/lib/voiceSpeedSettings.ts` — small localStorage wrapper for the default
|
||
|
||
### Steps
|
||
|
||
- [ ] **Step 1: Create the settings helper**
|
||
|
||
Create `apps/desktop/src/lib/voiceSpeedSettings.ts`:
|
||
|
||
```ts
|
||
// Persists the preferred voice-message playback rate across sessions.
|
||
// localStorage is fine here — non-sensitive, single source of truth per
|
||
// device, no cross-device sync needed.
|
||
|
||
const KEY = 'chatapp:voice-speed';
|
||
const ALLOWED = [1, 1.5, 2] as const;
|
||
export type VoiceSpeed = (typeof ALLOWED)[number];
|
||
|
||
export function getVoiceSpeed(): VoiceSpeed {
|
||
try {
|
||
const raw = window.localStorage.getItem(KEY);
|
||
if (!raw) return 1;
|
||
const parsed = Number(raw);
|
||
if (ALLOWED.includes(parsed as VoiceSpeed)) return parsed as VoiceSpeed;
|
||
} catch {
|
||
/* localStorage unavailable */
|
||
}
|
||
return 1;
|
||
}
|
||
|
||
export function setVoiceSpeed(speed: VoiceSpeed): void {
|
||
try {
|
||
window.localStorage.setItem(KEY, String(speed));
|
||
} catch {
|
||
/* localStorage unavailable — best effort */
|
||
}
|
||
}
|
||
|
||
export const VOICE_SPEEDS = ALLOWED;
|
||
```
|
||
|
||
- [ ] **Step 2: Wire into AttachmentAudio**
|
||
|
||
In `apps/desktop/src/components/AttachmentAudio.tsx`, add to imports:
|
||
|
||
```ts
|
||
import { getVoiceSpeed, setVoiceSpeed, VOICE_SPEEDS, type VoiceSpeed } from '../lib/voiceSpeedSettings';
|
||
```
|
||
|
||
Add a state hook near the other `useState` declarations (around line 25):
|
||
|
||
```ts
|
||
const [speed, setSpeed] = useState<VoiceSpeed>(() => getVoiceSpeed());
|
||
```
|
||
|
||
Add an effect that pushes the speed onto the audio element whenever either changes (right below the existing `arrayBuf` effect around line 106):
|
||
|
||
```ts
|
||
useEffect(() => {
|
||
const el = audioRef.current;
|
||
if (!el) return;
|
||
el.playbackRate = speed;
|
||
}, [speed, blobUrl]);
|
||
```
|
||
|
||
Add a speed-chip group inside the player's flex row, right after the play/pause button closes (after the closing `</button>` around line 156, before the `<div className="flex min-w-0 flex-1 ...">` block):
|
||
|
||
```tsx
|
||
<div className="flex shrink-0 items-center gap-0.5 rounded-md bg-surface-3 p-0.5 text-[10px] font-semibold text-fg-muted">
|
||
{VOICE_SPEEDS.map((s) => {
|
||
const active = s === speed;
|
||
return (
|
||
<button
|
||
key={s}
|
||
type="button"
|
||
onClick={() => {
|
||
setSpeed(s);
|
||
setVoiceSpeed(s);
|
||
}}
|
||
className={
|
||
'flex h-6 w-7 cursor-pointer items-center justify-center rounded transition ' +
|
||
(active ? 'bg-accent text-accent-fg' : 'hover:bg-surface hover:text-fg')
|
||
}
|
||
aria-pressed={active}
|
||
title={'Wiedergabegeschwindigkeit ' + s + '×'}
|
||
>
|
||
{s}×
|
||
</button>
|
||
);
|
||
})}
|
||
</div>
|
||
```
|
||
|
||
- [ ] **Step 3: Manual smoke**
|
||
|
||
`pnpm desktop:dev`. Send / open a voice message. Click 1.5×, then 2×, then back to 1×. Pitch should adjust accordingly. Refresh the app — the chip group should land on the last-used speed.
|
||
|
||
- [ ] **Step 4: Typecheck + tests**
|
||
|
||
```bash
|
||
pnpm --filter @chat-app/desktop typecheck
|
||
pnpm --filter @chat-app/desktop test
|
||
```
|
||
|
||
- [ ] **Step 5: Commit**
|
||
|
||
```bash
|
||
git add apps/desktop/src/lib/voiceSpeedSettings.ts apps/desktop/src/components/AttachmentAudio.tsx
|
||
git commit -m "feat(voice): playback-speed toggle (1x/1.5x/2x) with per-user default"
|
||
```
|
||
|
||
---
|
||
|
||
# Group B — View-Once Hardening
|
||
|
||
## Task B1: Fix the instant-close bug + hold-to-view + screenshot block
|
||
|
||
**Goal:**
|
||
1. Fix the bug where clicking a view-once attachment closes it before the user sees anything.
|
||
2. Switch to a "hold to view" pattern: image visible only while the mouse is held down.
|
||
3. Block desktop-OS screenshots / screen recording during the reveal window via Electron's `setContentProtection(true)`.
|
||
|
||
**Files:**
|
||
- Modify: `apps/desktop/src/components/ViewOnceImage.tsx` — rewrite the reveal flow
|
||
- Modify: `apps/desktop/electron/main.ts` — add IPC handler for content-protection toggle
|
||
- Modify: `apps/desktop/electron/ipc-types.ts` — declare the new channel
|
||
- Modify: `apps/desktop/electron/preload.ts` — expose to renderer
|
||
- Modify: `apps/desktop/electron/preload-types.d.ts` — type the renderer API
|
||
|
||
### Root cause
|
||
|
||
`ViewOnceImage.tsx:55-63`:
|
||
|
||
```ts
|
||
const handleOpen = async (): Promise<void> => {
|
||
try {
|
||
const res = await markAttachmentViewed(supabase, attachmentId);
|
||
if (res.viewedAt) setRevealedAt(res.viewedAt); // ← sets burned = true
|
||
} catch (err) {
|
||
console.warn('mark-viewed failed', err);
|
||
}
|
||
setFullscreen(true); // ← never paints; component returns tombstone above
|
||
};
|
||
```
|
||
|
||
Once `revealedAt` is set, the early return `if (burned && !isSender) return tombstone` fires on the next render and the fullscreen dialog never paints. The new flow defers `markAttachmentViewed` until the user releases the hold gesture, so the tombstone only swaps in AFTER the image has actually been viewed.
|
||
|
||
### Steps
|
||
|
||
- [ ] **Step 1: Add the IPC channel constants and types**
|
||
|
||
Read the current `apps/desktop/electron/ipc-types.ts` first to confirm the pattern (existing `CHANNELS` shape, exported types). Then add to the `CHANNELS` constant:
|
||
|
||
```ts
|
||
WINDOW_SET_CONTENT_PROTECTION: 'window:set-content-protection',
|
||
```
|
||
|
||
Export a new typed args interface:
|
||
|
||
```ts
|
||
export interface WindowSetContentProtectionArgs {
|
||
enabled: boolean;
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 2: Wire the main-process handler**
|
||
|
||
In `apps/desktop/electron/main.ts`, find the main-window setup (grep for `BrowserWindow` and `ipcMain.handle` to find the right scope). Inside the same scope that registers other ipcMain handlers, add:
|
||
|
||
```ts
|
||
ipcMain.handle(
|
||
CHANNELS.WINDOW_SET_CONTENT_PROTECTION,
|
||
(_evt, args: WindowSetContentProtectionArgs) => {
|
||
// Electron's setContentProtection covers Windows (WDA_MONITOR) and macOS
|
||
// (NSWindowSharingNone) in one call. No-op on Linux X11. Wrapped in
|
||
// try/catch because the window can already be destroyed by the time
|
||
// this fires during a teardown.
|
||
try {
|
||
mainWindow.setContentProtection(args.enabled);
|
||
} catch (err) {
|
||
console.warn('setContentProtection failed', err);
|
||
}
|
||
},
|
||
);
|
||
```
|
||
|
||
Replace `mainWindow` with the actual variable name used in `main.ts` — read the file first to confirm.
|
||
|
||
- [ ] **Step 3: Expose to the renderer via preload**
|
||
|
||
In `apps/desktop/electron/preload.ts`, add to the `electronAPI` object (mirror the existing pattern):
|
||
|
||
```ts
|
||
setContentProtection: (enabled: boolean) =>
|
||
ipcRenderer.invoke(CHANNELS.WINDOW_SET_CONTENT_PROTECTION, { enabled }),
|
||
```
|
||
|
||
In `apps/desktop/electron/preload-types.d.ts`, add the new method to the `ElectronAPI` interface:
|
||
|
||
```ts
|
||
setContentProtection: (enabled: boolean) => Promise<void>;
|
||
```
|
||
|
||
- [ ] **Step 4: Rewrite ViewOnceImage with hold-to-view**
|
||
|
||
Replace the entire `ViewOnceImage` component body in `apps/desktop/src/components/ViewOnceImage.tsx`. Change the React import on line 1 to:
|
||
|
||
```ts
|
||
import { useEffect, useRef, useState } from 'react';
|
||
```
|
||
|
||
Replace lines 17-87 (the component body) with:
|
||
|
||
```tsx
|
||
// Three states:
|
||
// 1. viewedAt is null AND user is recipient → blurred lock card. Press-and-
|
||
// hold reveals the image fullscreen; release closes it AND fires the
|
||
// mark-viewed RPC.
|
||
// 2. viewedAt is set → tombstone "Angesehen am …".
|
||
// 3. user is sender → normal image, tombstone update appears once recipient
|
||
// burns it.
|
||
//
|
||
// While revealed, the renderer window enables content-protection
|
||
// (`win.setContentProtection(true)`) so OS-level screen capture (OBS, Win/Cmd
|
||
// snipping tools, screen recorders) sees a black/empty surface. Re-enabled
|
||
// on release / unmount.
|
||
export function ViewOnceImage({ attachmentId, viewedAt, isSender, src }: Props) {
|
||
const [revealedAt, setRevealedAt] = useState<string | null>(viewedAt);
|
||
const [revealing, setRevealing] = useState(false);
|
||
const burnedRef = useRef(false);
|
||
|
||
const burned = revealedAt !== null;
|
||
|
||
// Tear down screen-capture protection if the component unmounts mid-reveal.
|
||
useEffect(() => {
|
||
return () => {
|
||
if (revealing) {
|
||
void window.electronAPI?.setContentProtection?.(false).catch(() => {});
|
||
}
|
||
};
|
||
}, [revealing]);
|
||
|
||
if (burned && !isSender) {
|
||
return (
|
||
<div className="flex h-32 w-48 items-center justify-center rounded-lg border border-dashed border-line bg-surface-3 text-xs text-fg-muted">
|
||
<EyeOffIcon className="mr-2 h-4 w-4" />
|
||
Angesehen am {new Date(revealedAt).toLocaleString()}
|
||
</div>
|
||
);
|
||
}
|
||
|
||
if (isSender) {
|
||
return (
|
||
<div className="relative">
|
||
<img src={src} alt="" className="max-h-72 rounded-lg" />
|
||
<span className="absolute left-2 top-2 inline-flex items-center gap-1 rounded-full bg-black/60 px-2 py-0.5 text-[10px] font-semibold text-white">
|
||
<EyeOffIcon className="h-3 w-3" /> Einmal ansehen
|
||
</span>
|
||
{burned && (
|
||
<span className="absolute right-2 bottom-2 rounded-full bg-emerald-500/80 px-2 py-0.5 text-[10px] font-semibold text-white">
|
||
Angesehen
|
||
</span>
|
||
)}
|
||
</div>
|
||
);
|
||
}
|
||
|
||
const startReveal = async (): Promise<void> => {
|
||
if (burnedRef.current) return;
|
||
burnedRef.current = true;
|
||
try {
|
||
await window.electronAPI?.setContentProtection?.(true);
|
||
} catch (err) {
|
||
console.warn('setContentProtection enable failed', err);
|
||
}
|
||
setRevealing(true);
|
||
};
|
||
|
||
const endReveal = async (): Promise<void> => {
|
||
if (!revealing) return;
|
||
setRevealing(false);
|
||
try {
|
||
await window.electronAPI?.setContentProtection?.(false);
|
||
} catch (err) {
|
||
console.warn('setContentProtection disable failed', err);
|
||
}
|
||
// Fire-and-forget mark-viewed. Updates the local tombstone on success;
|
||
// an error leaves the bubble in "lock card" state so the user can retry.
|
||
try {
|
||
const res = await markAttachmentViewed(supabase, attachmentId);
|
||
if (res.viewedAt) setRevealedAt(res.viewedAt);
|
||
} catch (err) {
|
||
console.warn('mark-viewed failed', err);
|
||
burnedRef.current = false; // allow retry on error
|
||
}
|
||
};
|
||
|
||
return (
|
||
<>
|
||
<button
|
||
type="button"
|
||
onPointerDown={() => void startReveal()}
|
||
onPointerUp={() => void endReveal()}
|
||
onPointerLeave={() => void endReveal()}
|
||
onPointerCancel={() => void endReveal()}
|
||
className="relative flex h-48 w-64 cursor-pointer flex-col items-center justify-center gap-2 rounded-lg border border-line bg-surface-3 text-fg-muted hover:border-accent/40 select-none"
|
||
>
|
||
<LockIcon className="h-6 w-6 text-accent" />
|
||
<span className="text-xs font-medium">Gedrückt halten zum Ansehen</span>
|
||
</button>
|
||
{revealing && (
|
||
<div
|
||
role="dialog"
|
||
aria-modal="true"
|
||
aria-label="Einmal-ansehen Bild"
|
||
className="fixed inset-0 z-50 flex items-center justify-center bg-black/95 p-6"
|
||
>
|
||
<img
|
||
src={src}
|
||
alt=""
|
||
className="max-h-full max-w-full select-none rounded-lg"
|
||
draggable={false}
|
||
/>
|
||
<span className="absolute bottom-6 left-1/2 -translate-x-1/2 rounded-full bg-white/10 px-3 py-1 text-xs font-semibold text-white">
|
||
Loslassen zum Schließen — Aufnahme blockiert
|
||
</span>
|
||
</div>
|
||
)}
|
||
</>
|
||
);
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 5: Build the Electron side + test reveal**
|
||
|
||
Rebuild the electron bundle:
|
||
|
||
```bash
|
||
pnpm --filter @chat-app/desktop run build:win
|
||
```
|
||
|
||
Then in dev mode:
|
||
|
||
```bash
|
||
pnpm desktop:dev
|
||
```
|
||
|
||
Test:
|
||
1. Receive a view-once image from another account
|
||
2. Press-and-hold the lock card → image appears fullscreen
|
||
3. While holding, try Win+PrtScr — the OS clipboard should NOT contain the image (instead a black rectangle where the window was)
|
||
4. Release → image closes, tombstone "Angesehen am …" appears in the next render after the mark-viewed RPC returns
|
||
|
||
- [ ] **Step 6: Typecheck**
|
||
|
||
```bash
|
||
pnpm --filter @chat-app/desktop typecheck
|
||
```
|
||
|
||
- [ ] **Step 7: Commit**
|
||
|
||
```bash
|
||
git add apps/desktop/electron/ipc-types.ts apps/desktop/electron/main.ts apps/desktop/electron/preload.ts apps/desktop/electron/preload-types.d.ts apps/desktop/src/components/ViewOnceImage.tsx
|
||
git commit -m "fix(view-once): hold-to-view pattern + content-protection during reveal"
|
||
```
|
||
|
||
---
|
||
|
||
# Group C — Multi-User Collab
|
||
|
||
## Task C1: Live cursors on the whiteboard
|
||
|
||
**Goal:** Every user editing the same whiteboard sees the others' pointer positions in real-time, as small coloured dots with their display name. Strokes themselves are unchanged (they already sync via `whiteboard_strokes` INSERT subscription).
|
||
|
||
**Files:**
|
||
- Create: `apps/desktop/src/lib/whiteboardCursors.ts` — Supabase broadcast wrapper for cursor presence
|
||
- Modify: `apps/desktop/src/components/WhiteboardCanvas.tsx` — emit cursor on pointerMove, render others' cursors as overlay
|
||
- Modify: `apps/desktop/src/components/WhiteboardModal.tsx` — thread `whiteboardId` through to enable cursor sync
|
||
|
||
**Out of scope:** in-progress stroke broadcast (deferred to a phase 9). Performance reason: 60 events/sec/user × N users with growing payloads is meaningfully heavier than presence-only cursors.
|
||
|
||
### Steps
|
||
|
||
- [ ] **Step 1: Create the cursor broadcast helper**
|
||
|
||
Create `apps/desktop/src/lib/whiteboardCursors.ts`:
|
||
|
||
```ts
|
||
// Live-cursor pubsub for the multi-user whiteboard. Uses Supabase's
|
||
// `broadcast` channel rather than `presence` because we want fire-and-forget
|
||
// position updates (no need to track join/leave) and presence has higher
|
||
// minimum latency due to its diff-and-merge semantics.
|
||
//
|
||
// Throttled to ~30 fps so a continuous drag doesn't flood the channel.
|
||
|
||
import type { RealtimeChannel } from '@supabase/supabase-js';
|
||
|
||
import { supabase } from './supabase';
|
||
|
||
const THROTTLE_MS = 33; // ~30 fps
|
||
|
||
export interface CursorEvent {
|
||
userId: string;
|
||
displayName: string;
|
||
// logical canvas coordinates (matches WhiteboardCanvas internal space)
|
||
x: number;
|
||
y: number;
|
||
}
|
||
|
||
export interface CursorSession {
|
||
send: (x: number, y: number) => void;
|
||
close: () => void;
|
||
}
|
||
|
||
export function openCursorSession(
|
||
whiteboardId: string,
|
||
self: { userId: string; displayName: string },
|
||
onCursor: (ev: CursorEvent) => void,
|
||
): CursorSession {
|
||
const channel: RealtimeChannel = supabase.channel('wb-cursor:' + whiteboardId, {
|
||
config: { broadcast: { self: false } },
|
||
});
|
||
channel.on('broadcast', { event: 'cursor' }, (payload) => {
|
||
const ev = payload.payload as CursorEvent | undefined;
|
||
if (!ev || ev.userId === self.userId) return;
|
||
onCursor(ev);
|
||
});
|
||
channel.subscribe();
|
||
|
||
let lastSentAt = 0;
|
||
let pending: { x: number; y: number } | null = null;
|
||
let flushTimer: ReturnType<typeof setTimeout> | null = null;
|
||
|
||
const flush = (): void => {
|
||
flushTimer = null;
|
||
if (!pending) return;
|
||
const { x, y } = pending;
|
||
pending = null;
|
||
lastSentAt = Date.now();
|
||
void channel.send({
|
||
type: 'broadcast',
|
||
event: 'cursor',
|
||
payload: { userId: self.userId, displayName: self.displayName, x, y } satisfies CursorEvent,
|
||
});
|
||
};
|
||
|
||
const send = (x: number, y: number): void => {
|
||
const now = Date.now();
|
||
const since = now - lastSentAt;
|
||
if (since >= THROTTLE_MS) {
|
||
pending = { x, y };
|
||
flush();
|
||
} else {
|
||
pending = { x, y };
|
||
if (flushTimer === null) {
|
||
flushTimer = setTimeout(flush, THROTTLE_MS - since);
|
||
}
|
||
}
|
||
};
|
||
|
||
const close = (): void => {
|
||
if (flushTimer !== null) clearTimeout(flushTimer);
|
||
flushTimer = null;
|
||
pending = null;
|
||
void supabase.removeChannel(channel);
|
||
};
|
||
|
||
return { send, close };
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 2: Render others' cursors in WhiteboardCanvas**
|
||
|
||
In `apps/desktop/src/components/WhiteboardCanvas.tsx`, add imports:
|
||
|
||
```ts
|
||
import { useAuth } from '../context/AuthContext';
|
||
import { openCursorSession, type CursorEvent, type CursorSession } from '../lib/whiteboardCursors';
|
||
```
|
||
|
||
Extend the `Props` interface to add an optional `whiteboardId`:
|
||
|
||
```ts
|
||
interface Props {
|
||
strokes: WhiteboardStroke[];
|
||
tool: WhiteboardTool;
|
||
color: WhiteboardColor;
|
||
width: WhiteboardWidth;
|
||
onStroke: (payload: WhiteboardStrokePayload) => void;
|
||
logicalWidth?: number;
|
||
logicalHeight?: number;
|
||
/** Enables live-cursor broadcast when set. */
|
||
whiteboardId?: string | null;
|
||
}
|
||
```
|
||
|
||
Update the function signature to accept the new prop. Inside the component, before the existing pointer handlers, add:
|
||
|
||
```ts
|
||
const { session } = useAuth();
|
||
const [remoteCursors, setRemoteCursors] = useState<Map<string, CursorEvent & { lastSeen: number }>>(
|
||
() => new Map(),
|
||
);
|
||
const cursorSessionRef = useRef<CursorSession | null>(null);
|
||
|
||
useEffect(() => {
|
||
if (!whiteboardId) return;
|
||
const me = session?.user;
|
||
if (!me) return;
|
||
const displayName =
|
||
(me.user_metadata as Record<string, unknown> | null)?.['display_name'] as string | undefined
|
||
?? me.email
|
||
?? me.id.slice(0, 8);
|
||
const s = openCursorSession(
|
||
whiteboardId,
|
||
{ userId: me.id, displayName },
|
||
(ev) => {
|
||
setRemoteCursors((prev) => {
|
||
const next = new Map(prev);
|
||
next.set(ev.userId, { ...ev, lastSeen: Date.now() });
|
||
return next;
|
||
});
|
||
},
|
||
);
|
||
cursorSessionRef.current = s;
|
||
return () => {
|
||
s.close();
|
||
cursorSessionRef.current = null;
|
||
};
|
||
}, [whiteboardId, session?.user]);
|
||
|
||
// Stale-cursor sweep: drop cursors that haven't been heard from in 2s. Cheap
|
||
// poll because the Map is tiny (at most one entry per active collaborator).
|
||
useEffect(() => {
|
||
if (remoteCursors.size === 0) return;
|
||
const id = setInterval(() => {
|
||
const now = Date.now();
|
||
setRemoteCursors((prev) => {
|
||
let changed = false;
|
||
const next = new Map(prev);
|
||
for (const [k, v] of next) {
|
||
if (now - v.lastSeen > 2000) {
|
||
next.delete(k);
|
||
changed = true;
|
||
}
|
||
}
|
||
return changed ? next : prev;
|
||
});
|
||
}, 1000);
|
||
return () => clearInterval(id);
|
||
}, [remoteCursors.size]);
|
||
```
|
||
|
||
Modify `handlePointerMove` to also broadcast the cursor (the function around line 83):
|
||
|
||
```ts
|
||
const handlePointerMove = (e: React.PointerEvent<HTMLCanvasElement>) => {
|
||
const [x, y] = canvasPoint(e);
|
||
cursorSessionRef.current?.send(x, y);
|
||
if (!draftRef.current) return;
|
||
draftRef.current.points.push([x, y, Date.now() - strokeStartRef.current]);
|
||
forceTick((n) => n + 1);
|
||
};
|
||
```
|
||
|
||
Wrap the existing `<canvas>` in a relatively-positioned container and render cursor markers on top. Replace the existing `return (...)` block with:
|
||
|
||
```tsx
|
||
return (
|
||
<div
|
||
className="relative"
|
||
style={{ aspectRatio: logicalWidth + ' / ' + logicalHeight, width: '100%' }}
|
||
>
|
||
<canvas
|
||
ref={canvasRef}
|
||
onPointerDown={handlePointerDown}
|
||
onPointerMove={handlePointerMove}
|
||
onPointerUp={handlePointerUp}
|
||
onPointerLeave={handlePointerUp}
|
||
className="block max-h-full max-w-full cursor-crosshair touch-none rounded-lg border border-line/40 bg-white shadow-2xl"
|
||
style={{ width: '100%', height: '100%' }}
|
||
/>
|
||
{Array.from(remoteCursors.values()).map((c) => {
|
||
const pctX = (c.x / logicalWidth) * 100;
|
||
const pctY = (c.y / logicalHeight) * 100;
|
||
return (
|
||
<div
|
||
key={c.userId}
|
||
aria-hidden="true"
|
||
className="pointer-events-none absolute"
|
||
style={{ left: pctX + '%', top: pctY + '%', transform: 'translate(-2px, -2px)' }}
|
||
>
|
||
<span
|
||
className="block h-2 w-2 rounded-full border-2 border-white shadow"
|
||
style={{ backgroundColor: colorForUserId(c.userId) }}
|
||
/>
|
||
<span className="ml-2 inline-block translate-y-[-2px] rounded-full bg-black/60 px-1.5 py-0.5 text-[9px] font-semibold text-white">
|
||
{c.displayName}
|
||
</span>
|
||
</div>
|
||
);
|
||
})}
|
||
</div>
|
||
);
|
||
```
|
||
|
||
Add a helper at the bottom of the file (after `renderStroke`):
|
||
|
||
```ts
|
||
function colorForUserId(userId: string): string {
|
||
// Deterministic hue from the user id so each collaborator gets a stable
|
||
// colour across sessions. Saturation/lightness fixed to keep the cursor
|
||
// legible against the white canvas.
|
||
let hash = 0;
|
||
for (let i = 0; i < userId.length; i++) hash = (hash * 31 + userId.charCodeAt(i)) | 0;
|
||
const hue = Math.abs(hash) % 360;
|
||
return 'hsl(' + hue + ', 70%, 50%)';
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 3: Pass whiteboardId through WhiteboardModal**
|
||
|
||
In `apps/desktop/src/components/WhiteboardModal.tsx`, pass the new prop to `<WhiteboardCanvas>` (around line 77):
|
||
|
||
```tsx
|
||
<WhiteboardCanvas
|
||
strokes={strokes}
|
||
tool={tool}
|
||
color={color}
|
||
width={width}
|
||
onStroke={(payload) => void insertStroke(payload)}
|
||
whiteboardId={whiteboardId}
|
||
/>
|
||
```
|
||
|
||
- [ ] **Step 4: Typecheck + manual smoke**
|
||
|
||
```bash
|
||
pnpm --filter @chat-app/desktop typecheck
|
||
pnpm desktop:dev
|
||
```
|
||
|
||
Open the same whiteboard in two windows (or two devices). Move the mouse in one — the other should show a coloured dot tracking your position, with the display name label. Stop moving for 2 seconds — the cursor fades.
|
||
|
||
- [ ] **Step 5: Commit**
|
||
|
||
```bash
|
||
git add apps/desktop/src/lib/whiteboardCursors.ts apps/desktop/src/components/WhiteboardCanvas.tsx apps/desktop/src/components/WhiteboardModal.tsx
|
||
git commit -m "feat(whiteboard): live cursors via broadcast channel"
|
||
```
|
||
|
||
---
|
||
|
||
## Task C2: Screen-share annotation overlay
|
||
|
||
**Goal:** While viewing someone's screen share inside a call, any participant can draw on top of the video stream. All viewers + the sharer see the strokes live, with auto-fade after 8 seconds.
|
||
|
||
**Files:**
|
||
- Create: `apps/desktop/src/components/ScreenShareAnnotations.tsx` — canvas overlay component
|
||
- Modify: `apps/desktop/src/components/ScreenShareViewer.tsx` — wrap the `<video>` and add a toggle button to show/hide the annotation layer
|
||
- Possibly add: `PencilIcon` to `apps/desktop/src/components/icons.tsx` if it doesn't exist yet (grep first to confirm)
|
||
|
||
**Approach:**
|
||
- Reuse the broadcast pattern from Task C1 — channel name `screen-annotation:<participantId>`.
|
||
- Stroke payload: `{userId, color, points: [[x, y], ...]}` in normalized 0..1 coordinates (different viewer canvases may have different pixel sizes).
|
||
- Each stroke fades over 8s via canvas-paint opacity calculation, then is GC'd from local state.
|
||
|
||
### Steps
|
||
|
||
- [ ] **Step 1: Confirm PencilIcon availability**
|
||
|
||
Run: `grep -n "PencilIcon" apps/desktop/src/components/icons.tsx`
|
||
|
||
If no match, append to `apps/desktop/src/components/icons.tsx`:
|
||
|
||
```tsx
|
||
export function PencilIcon(props: React.SVGProps<SVGSVGElement>) {
|
||
return (
|
||
<svg viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth={2}
|
||
strokeLinecap="round" strokeLinejoin="round" {...props}>
|
||
<path d="M12 20h9" />
|
||
<path d="M16.5 3.5a2.121 2.121 0 0 1 3 3L7 19l-4 1 1-4Z" />
|
||
</svg>
|
||
);
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 2: Create the overlay component**
|
||
|
||
Create `apps/desktop/src/components/ScreenShareAnnotations.tsx`:
|
||
|
||
```tsx
|
||
import { useCallback, useEffect, useRef, useState } from 'react';
|
||
|
||
import { useAuth } from '../context/AuthContext';
|
||
import { supabase } from '../lib/supabase';
|
||
import { PencilIcon, XIcon } from './icons';
|
||
|
||
interface Stroke {
|
||
id: string;
|
||
userId: string;
|
||
color: string;
|
||
// normalized 0..1 coordinates so any viewer's canvas size renders consistently
|
||
points: Array<[number, number]>;
|
||
bornAt: number;
|
||
}
|
||
|
||
interface Props {
|
||
/** Stable per-share key. Use the share's participantId. */
|
||
shareKey: string;
|
||
/** Render annotations transparently (off when the toolbar is closed). */
|
||
enabled: boolean;
|
||
onToggleEnabled: (next: boolean) => void;
|
||
}
|
||
|
||
const FADE_MS = 8000;
|
||
const COLORS = ['#ef4444', '#3b82f6', '#22c55e', '#facc15', '#a855f7', '#000000'] as const;
|
||
|
||
export function ScreenShareAnnotations({ shareKey, enabled, onToggleEnabled }: Props) {
|
||
const { session } = useAuth();
|
||
const userId = session?.user.id ?? 'anon';
|
||
const [color, setColor] = useState<string>(COLORS[0]);
|
||
const [strokes, setStrokes] = useState<Stroke[]>([]);
|
||
const draftRef = useRef<Stroke | null>(null);
|
||
const containerRef = useRef<HTMLDivElement | null>(null);
|
||
const canvasRef = useRef<HTMLCanvasElement | null>(null);
|
||
|
||
const broadcastRef = useRef<((s: Stroke) => void) | null>(null);
|
||
|
||
// Subscribe to remote strokes.
|
||
useEffect(() => {
|
||
const channel = supabase.channel('screen-annotation:' + shareKey, {
|
||
config: { broadcast: { self: false } },
|
||
});
|
||
channel.on('broadcast', { event: 'stroke' }, (payload) => {
|
||
const s = payload.payload as Stroke | undefined;
|
||
if (!s || s.userId === userId) return;
|
||
setStrokes((prev) => [...prev, { ...s, bornAt: Date.now() }]);
|
||
});
|
||
channel.subscribe();
|
||
broadcastRef.current = (s: Stroke) => {
|
||
void channel.send({
|
||
type: 'broadcast',
|
||
event: 'stroke',
|
||
payload: s,
|
||
});
|
||
};
|
||
return () => {
|
||
broadcastRef.current = null;
|
||
void supabase.removeChannel(channel);
|
||
};
|
||
}, [shareKey, userId]);
|
||
|
||
// Garbage-collect faded strokes after FADE_MS + a small grace window.
|
||
useEffect(() => {
|
||
if (strokes.length === 0) return;
|
||
const id = setInterval(() => {
|
||
const cutoff = Date.now() - FADE_MS - 500;
|
||
setStrokes((prev) => {
|
||
const next = prev.filter((s) => s.bornAt > cutoff);
|
||
return next.length === prev.length ? prev : next;
|
||
});
|
||
}, 1000);
|
||
return () => clearInterval(id);
|
||
}, [strokes.length]);
|
||
|
||
// Paint the canvas on every render tick.
|
||
useEffect(() => {
|
||
const cv = canvasRef.current;
|
||
const container = containerRef.current;
|
||
if (!cv || !container) return;
|
||
const rect = container.getBoundingClientRect();
|
||
if (cv.width !== rect.width || cv.height !== rect.height) {
|
||
cv.width = Math.max(1, Math.floor(rect.width));
|
||
cv.height = Math.max(1, Math.floor(rect.height));
|
||
}
|
||
const ctx = cv.getContext('2d');
|
||
if (!ctx) return;
|
||
ctx.clearRect(0, 0, cv.width, cv.height);
|
||
const now = Date.now();
|
||
const drawStroke = (s: Stroke) => {
|
||
const age = now - s.bornAt;
|
||
const alpha = Math.max(0, 1 - age / FADE_MS);
|
||
if (alpha <= 0) return;
|
||
ctx.save();
|
||
ctx.globalAlpha = alpha;
|
||
ctx.strokeStyle = s.color;
|
||
ctx.lineWidth = 3;
|
||
ctx.lineCap = 'round';
|
||
ctx.lineJoin = 'round';
|
||
ctx.beginPath();
|
||
for (let i = 0; i < s.points.length; i++) {
|
||
const [nx, ny] = s.points[i]!;
|
||
const x = nx * cv.width;
|
||
const y = ny * cv.height;
|
||
if (i === 0) ctx.moveTo(x, y);
|
||
else ctx.lineTo(x, y);
|
||
}
|
||
ctx.stroke();
|
||
ctx.restore();
|
||
};
|
||
for (const s of strokes) drawStroke(s);
|
||
if (draftRef.current) drawStroke(draftRef.current);
|
||
});
|
||
|
||
// Animation frame loop so faded strokes visually decay between paints.
|
||
useEffect(() => {
|
||
if (strokes.length === 0 && !draftRef.current) return;
|
||
let raf = 0;
|
||
const tick = () => {
|
||
// Nudge state to force a re-paint. Slightly hacky but cheaper than a
|
||
// dedicated refresh state.
|
||
setStrokes((prev) => prev.slice());
|
||
raf = window.requestAnimationFrame(tick);
|
||
};
|
||
raf = window.requestAnimationFrame(tick);
|
||
return () => window.cancelAnimationFrame(raf);
|
||
}, [strokes.length]);
|
||
|
||
const normalized = useCallback((e: React.PointerEvent<HTMLDivElement>) => {
|
||
const container = containerRef.current;
|
||
if (!container) return [0, 0] as [number, number];
|
||
const rect = container.getBoundingClientRect();
|
||
const nx = (e.clientX - rect.left) / rect.width;
|
||
const ny = (e.clientY - rect.top) / rect.height;
|
||
return [Math.min(1, Math.max(0, nx)), Math.min(1, Math.max(0, ny))] as [number, number];
|
||
}, []);
|
||
|
||
const onPointerDown = (e: React.PointerEvent<HTMLDivElement>) => {
|
||
if (!enabled) return;
|
||
(e.currentTarget as HTMLDivElement).setPointerCapture(e.pointerId);
|
||
draftRef.current = {
|
||
id: Math.random().toString(36).slice(2),
|
||
userId,
|
||
color,
|
||
points: [normalized(e)],
|
||
bornAt: Date.now(),
|
||
};
|
||
};
|
||
|
||
const onPointerMove = (e: React.PointerEvent<HTMLDivElement>) => {
|
||
if (!draftRef.current) return;
|
||
draftRef.current.points.push(normalized(e));
|
||
setStrokes((prev) => prev.slice()); // cheap re-render trigger
|
||
};
|
||
|
||
const onPointerUp = (e: React.PointerEvent<HTMLDivElement>) => {
|
||
const target = e.currentTarget as HTMLDivElement;
|
||
if (target.hasPointerCapture(e.pointerId)) target.releasePointerCapture(e.pointerId);
|
||
const draft = draftRef.current;
|
||
draftRef.current = null;
|
||
if (!draft || draft.points.length < 2) {
|
||
setStrokes((prev) => prev.slice());
|
||
return;
|
||
}
|
||
setStrokes((prev) => [...prev, draft]);
|
||
broadcastRef.current?.(draft);
|
||
};
|
||
|
||
return (
|
||
<>
|
||
<div
|
||
ref={containerRef}
|
||
onPointerDown={onPointerDown}
|
||
onPointerMove={onPointerMove}
|
||
onPointerUp={onPointerUp}
|
||
onPointerLeave={onPointerUp}
|
||
className={
|
||
'absolute inset-0 z-10 ' +
|
||
(enabled ? 'cursor-crosshair touch-none' : 'pointer-events-none')
|
||
}
|
||
>
|
||
<canvas
|
||
ref={canvasRef}
|
||
className="pointer-events-none absolute inset-0 h-full w-full"
|
||
/>
|
||
</div>
|
||
<div className="absolute right-3 top-12 z-20 flex flex-col items-end gap-1">
|
||
<button
|
||
type="button"
|
||
onClick={() => onToggleEnabled(!enabled)}
|
||
aria-pressed={enabled}
|
||
title={enabled ? 'Annotation aus' : 'Annotation an'}
|
||
className={
|
||
'flex h-8 w-8 cursor-pointer items-center justify-center rounded-full text-white shadow-lg transition ' +
|
||
(enabled ? 'bg-accent' : 'bg-black/60 hover:bg-black/80')
|
||
}
|
||
>
|
||
{enabled ? <XIcon className="h-3.5 w-3.5" /> : <PencilIcon className="h-3.5 w-3.5" />}
|
||
</button>
|
||
{enabled && (
|
||
<div className="flex items-center gap-1 rounded-full bg-black/60 p-1 shadow-lg">
|
||
{COLORS.map((c) => (
|
||
<button
|
||
key={c}
|
||
type="button"
|
||
onClick={() => setColor(c)}
|
||
aria-label={c}
|
||
aria-pressed={c === color}
|
||
className={
|
||
'h-5 w-5 cursor-pointer rounded-full border-2 transition ' +
|
||
(c === color ? 'border-white scale-110' : 'border-white/30 hover:scale-105')
|
||
}
|
||
style={{ backgroundColor: c }}
|
||
/>
|
||
))}
|
||
</div>
|
||
)}
|
||
</div>
|
||
</>
|
||
);
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 3: Integrate the overlay into ScreenShareViewer**
|
||
|
||
In `apps/desktop/src/components/ScreenShareViewer.tsx`, add the import:
|
||
|
||
```ts
|
||
import { ScreenShareAnnotations } from './ScreenShareAnnotations';
|
||
```
|
||
|
||
Add a local state above the existing `<video>` block:
|
||
|
||
```ts
|
||
const [annotateEnabled, setAnnotateEnabled] = useState(false);
|
||
```
|
||
|
||
Replace the `watching ? <video.../> : <button.../>` branch (around line 98-133). The `<video>` block at lines 99-113 needs to be wrapped:
|
||
|
||
```tsx
|
||
{watching ? (
|
||
<div className="relative h-full w-full flex-1">
|
||
<video
|
||
ref={videoRef}
|
||
autoPlay
|
||
playsInline
|
||
muted
|
||
onContextMenu={(e) => e.preventDefault()}
|
||
onDoubleClick={toggleFullscreen}
|
||
className="block h-full w-full cursor-zoom-in bg-black object-contain"
|
||
/>
|
||
<ScreenShareAnnotations
|
||
shareKey={share.participantId}
|
||
enabled={annotateEnabled}
|
||
onToggleEnabled={setAnnotateEnabled}
|
||
/>
|
||
</div>
|
||
) : (
|
||
// existing watch-button branch — unchanged
|
||
)}
|
||
```
|
||
|
||
(Preserve the existing `else`/non-watching branch verbatim.)
|
||
|
||
- [ ] **Step 4: Typecheck + manual smoke**
|
||
|
||
```bash
|
||
pnpm --filter @chat-app/desktop typecheck
|
||
pnpm desktop:dev
|
||
```
|
||
|
||
Test plan:
|
||
1. Start a call with two clients
|
||
2. One client shares screen
|
||
3. The other clicks the pencil icon top-right → draws a stroke on the share
|
||
4. Sharer + viewer both see the stroke, fading over 8 seconds
|
||
5. Toggle off the pencil — strokes still visible until they fade, but no new strokes captured
|
||
|
||
- [ ] **Step 5: Commit**
|
||
|
||
```bash
|
||
git add apps/desktop/src/components/ScreenShareAnnotations.tsx apps/desktop/src/components/ScreenShareViewer.tsx apps/desktop/src/components/icons.tsx
|
||
git commit -m "feat(call): shared annotation overlay on screen share"
|
||
```
|
||
|
||
---
|
||
|
||
# Group D — Soundboard & Avatars
|
||
|
||
## Task D1: Soundboard hotkeys always-on with local fallback
|
||
|
||
**Goal:** Soundboard hotkeys fire even when no call is connected. Outside a call, the sound plays through the local default audio output (not into a call pipeline that doesn't exist).
|
||
|
||
**Files:**
|
||
- Create: `apps/desktop/src/lib/soundboardLocalPlay.ts` — plays a SoundboardEntry to default output via a fresh `HTMLAudioElement`
|
||
- Modify: `apps/desktop/src/context/CallContext.tsx` (lift the `useEffect` guard at line 2262, add a fallback playback path)
|
||
|
||
### Steps
|
||
|
||
- [ ] **Step 1: Investigate the soundboard storage API**
|
||
|
||
Run: `grep -n "export" apps/desktop/src/lib/soundboardStorage.ts | head -30`
|
||
|
||
Identify how an entry's audio data is retrieved (e.g., `getSoundboardBlobUrl`, `loadSoundData`, a `dataUrl` field on the entry, etc.). Pick the simplest accessor that returns a playable URL or Blob. The snippet below assumes a function named `getSoundboardBlobUrl(id)` exists — **adapt the name to match what the codebase actually exports**.
|
||
|
||
- [ ] **Step 2: Create the local playback helper**
|
||
|
||
Create `apps/desktop/src/lib/soundboardLocalPlay.ts`:
|
||
|
||
```ts
|
||
// Plays a soundboard entry to the local default audio output. Used when no
|
||
// call pipeline is active (the in-call path routes via the LiveKit
|
||
// publishing pipeline so peers hear; this path is local-only). Reuses the
|
||
// blob URL caching in the existing soundboard storage so we don't redecode
|
||
// for every hotkey press.
|
||
|
||
import { getSoundboardBlobUrl, type SoundboardEntry } from './soundboardStorage';
|
||
|
||
const activeAudios = new Set<HTMLAudioElement>();
|
||
|
||
export async function playSoundboardLocal(entry: SoundboardEntry): Promise<void> {
|
||
const src = await getSoundboardBlobUrl(entry.id);
|
||
if (!src) return;
|
||
const el = new Audio(src);
|
||
// Mirror the per-entry local-volume preference if the entry shape exposes
|
||
// one. Adapt the field name to whatever soundboardStorage exports.
|
||
const vol = (entry as unknown as { localVolume?: number }).localVolume;
|
||
if (typeof vol === 'number') el.volume = Math.max(0, Math.min(1, vol));
|
||
activeAudios.add(el);
|
||
el.addEventListener('ended', () => {
|
||
activeAudios.delete(el);
|
||
});
|
||
el.addEventListener('error', () => {
|
||
activeAudios.delete(el);
|
||
});
|
||
try {
|
||
await el.play();
|
||
} catch (err) {
|
||
activeAudios.delete(el);
|
||
console.warn('soundboardLocalPlay failed', err);
|
||
}
|
||
}
|
||
|
||
export function stopSoundboardLocal(): void {
|
||
for (const el of activeAudios) {
|
||
try {
|
||
el.pause();
|
||
el.currentTime = 0;
|
||
} catch {
|
||
/* ignore */
|
||
}
|
||
}
|
||
activeAudios.clear();
|
||
}
|
||
```
|
||
|
||
If `getSoundboardBlobUrl` doesn't exist, write a small wrapper inside this file that loads via whichever accessor IS exported. Don't add new infrastructure — reuse what's there.
|
||
|
||
- [ ] **Step 3: Lift the hotkey-effect gate in CallContext**
|
||
|
||
In `apps/desktop/src/context/CallContext.tsx`, find the existing block (around line 2260-2268):
|
||
|
||
```ts
|
||
// Global soundboard hotkey registration — runs only while connected so the
|
||
// OS-level shortcuts don't fire when the user is outside of a call.
|
||
useEffect(() => {
|
||
if (state.kind !== 'connected') return;
|
||
const teardown = startSoundboardHotkeys((id) => {
|
||
void playSoundboard(id);
|
||
});
|
||
return teardown;
|
||
}, [state.kind, playSoundboard]);
|
||
```
|
||
|
||
Replace with:
|
||
|
||
```ts
|
||
// Global soundboard hotkey registration — always-on so the OS-level
|
||
// shortcuts fire even outside a call (Stream-Deck-style local SFX). Inside
|
||
// a call we route through `playSoundboard` so peers hear; outside a call
|
||
// we fall back to `playSoundboardLocal` which plays through the system
|
||
// default output only.
|
||
useEffect(() => {
|
||
const teardown = startSoundboardHotkeys((id) => {
|
||
if (state.kind === 'connected') {
|
||
void playSoundboard(id);
|
||
} else {
|
||
void (async () => {
|
||
const entries = await listSoundboard();
|
||
const entry = entries.find((e) => e.id === id);
|
||
if (entry) await playSoundboardLocal(entry);
|
||
})();
|
||
}
|
||
});
|
||
return teardown;
|
||
}, [state.kind, playSoundboard]);
|
||
```
|
||
|
||
Add to imports near the other soundboard imports:
|
||
|
||
```ts
|
||
import { playSoundboardLocal } from '../lib/soundboardLocalPlay';
|
||
```
|
||
|
||
Verify `listSoundboard` is already imported in this file. If it's not (it's imported in `soundboardHotkeys.ts` as `listSounds`), use the same name and import accordingly.
|
||
|
||
- [ ] **Step 4: Typecheck + manual smoke**
|
||
|
||
```bash
|
||
pnpm --filter @chat-app/desktop typecheck
|
||
pnpm desktop:dev
|
||
```
|
||
|
||
Without joining any call:
|
||
1. Open Settings → Soundboard → assign a hotkey to a sound (e.g. `F13` or `Ctrl+Shift+1`)
|
||
2. Close settings, do anything else — even minimize the window
|
||
3. Press the hotkey → the sound should play locally (you hear it through your speakers)
|
||
|
||
Inside a call: same hotkey → peers also hear it (existing behaviour, unchanged).
|
||
|
||
- [ ] **Step 5: Commit**
|
||
|
||
```bash
|
||
git add apps/desktop/src/lib/soundboardLocalPlay.ts apps/desktop/src/context/CallContext.tsx
|
||
git commit -m "feat(soundboard): hotkeys fire outside calls with local-only playback"
|
||
```
|
||
|
||
---
|
||
|
||
## Task D2: Animated avatars (GIF / APNG / animated WebP)
|
||
|
||
**Goal:** Allow uploading animated avatars. The renderer already plays animated `<img>` formats natively — the only blocker is `avatarUpload.ts` re-encoding via canvas to a static frame.
|
||
|
||
**Files:**
|
||
- Modify: `apps/desktop/src/lib/avatarUpload.ts` — branch on MIME, bypass canvas-resize for animated formats with dimension/size caps
|
||
|
||
### Steps
|
||
|
||
- [ ] **Step 1: Detect animated formats and skip the re-encode**
|
||
|
||
In `apps/desktop/src/lib/avatarUpload.ts`, replace `uploadAvatar` (around line 46-52) with the new branching version and add helpers. The full new contents of the file (after the existing `resizeToSquare` function ends, line 44):
|
||
|
||
```ts
|
||
const MAX_ANIMATED_BYTES = 2 * 1024 * 1024; // 2 MB hard cap on animated uploads
|
||
const ANIMATED_MIMES = new Set(['image/gif', 'image/apng', 'image/webp', 'image/png']);
|
||
|
||
export async function uploadAvatar(userId: string, file: File): Promise<string> {
|
||
if (!file.type.startsWith('image/')) {
|
||
throw new Error('only image files are accepted');
|
||
}
|
||
// Animated formats bypass the canvas re-encode (which would strip
|
||
// animation by sampling the first frame). We still validate dimensions
|
||
// and size so a 40-MB animated WebP can't slip through.
|
||
if (ANIMATED_MIMES.has(file.type) && (await isAnimated(file))) {
|
||
if (file.size > MAX_ANIMATED_BYTES) {
|
||
throw new Error('animated avatar too large (max 2 MB)');
|
||
}
|
||
const dims = await readDimensions(file);
|
||
if (dims.width > MAX_DIM || dims.height > MAX_DIM) {
|
||
throw new Error('animated avatar exceeds ' + MAX_DIM + 'px (got ' + dims.width + 'x' + dims.height + ')');
|
||
}
|
||
return uploadAvatarBlob(userId, file);
|
||
}
|
||
const blob = await resizeToSquare(file);
|
||
return uploadAvatarBlob(userId, blob);
|
||
}
|
||
|
||
async function readDimensions(file: File): Promise<{ width: number; height: number }> {
|
||
const url = URL.createObjectURL(file);
|
||
try {
|
||
const img = await new Promise<HTMLImageElement>((resolve, reject) => {
|
||
const i = new Image();
|
||
i.onload = () => resolve(i);
|
||
i.onerror = () => reject(new Error('image load failed'));
|
||
i.src = url;
|
||
});
|
||
return { width: img.naturalWidth, height: img.naturalHeight };
|
||
} finally {
|
||
URL.revokeObjectURL(url);
|
||
}
|
||
}
|
||
|
||
async function isAnimated(file: File): Promise<boolean> {
|
||
// GIF: any GIF89a/GIF87a header is treated as potentially animated. The
|
||
// static-GIF case (one image-descriptor block) is rare enough that
|
||
// re-encoding wouldn't save much, so we accept the false-positives.
|
||
if (file.type === 'image/gif') return true;
|
||
// APNG: presence of an 'acTL' chunk inside the PNG stream. Scan the
|
||
// first 64 KB — APNGs put acTL near the front, before IDAT.
|
||
if (file.type === 'image/apng' || file.type === 'image/png') {
|
||
const head = await file.slice(0, 65536).arrayBuffer();
|
||
return containsBytes(head, [0x61, 0x63, 0x54, 0x4c]); // 'acTL'
|
||
}
|
||
// Animated WebP: 'ANIM' chunk in the RIFF container.
|
||
if (file.type === 'image/webp') {
|
||
const head = await file.slice(0, 65536).arrayBuffer();
|
||
return containsBytes(head, [0x41, 0x4e, 0x49, 0x4d]); // 'ANIM'
|
||
}
|
||
return false;
|
||
}
|
||
|
||
function containsBytes(buf: ArrayBuffer, needle: number[]): boolean {
|
||
const view = new Uint8Array(buf);
|
||
const len = view.length;
|
||
const nlen = needle.length;
|
||
outer: for (let i = 0; i + nlen <= len; i++) {
|
||
for (let j = 0; j < nlen; j++) {
|
||
if (view[i + j] !== needle[j]) continue outer;
|
||
}
|
||
return true;
|
||
}
|
||
return false;
|
||
}
|
||
```
|
||
|
||
Then update `uploadAvatarBlob` so the file extension matches the actual MIME. Replace the line (around line 58):
|
||
|
||
```ts
|
||
const ext = blob.type === 'image/webp' ? 'webp' : 'jpg';
|
||
```
|
||
|
||
with:
|
||
|
||
```ts
|
||
const ext =
|
||
blob.type === 'image/webp' ? 'webp' :
|
||
blob.type === 'image/gif' ? 'gif' :
|
||
blob.type === 'image/apng' || blob.type === 'image/png' ? 'png' :
|
||
'jpg';
|
||
```
|
||
|
||
- [ ] **Step 2: Typecheck + manual smoke**
|
||
|
||
```bash
|
||
pnpm --filter @chat-app/desktop typecheck
|
||
pnpm desktop:dev
|
||
```
|
||
|
||
Test plan:
|
||
1. Settings → Profile → upload an animated GIF (<2 MB, <512×512). It should appear animated in the Avatar component instantly.
|
||
2. Upload a 5 MB GIF → error "animated avatar too large (max 2 MB)".
|
||
3. Upload a static PNG (no acTL chunk) → still goes through the canvas-resize path because `isAnimated` returns false.
|
||
4. Upload an animated WebP → preserved animated.
|
||
|
||
- [ ] **Step 3: Commit**
|
||
|
||
```bash
|
||
git add apps/desktop/src/lib/avatarUpload.ts
|
||
git commit -m "feat(profile): preserve animation on GIF/APNG/animated-WebP avatar uploads"
|
||
```
|
||
|
||
---
|
||
|
||
## Final Code Review (after all tasks land)
|
||
|
||
Dispatch a code-reviewer subagent over the full range:
|
||
|
||
```bash
|
||
git log --oneline c8f0e8e..HEAD
|
||
git diff --stat c8f0e8e..HEAD
|
||
```
|
||
|
||
Verify:
|
||
- All 7 task commits present (A1, A2, B1, C1, C2, D1, D2)
|
||
- Typecheck clean
|
||
- All vitest suites pass
|
||
- Manual smoke checklist below cleared
|
||
|
||
## Manual QA Checklist (post-merge)
|
||
|
||
| # | Behaviour | Pass criteria |
|
||
|---|-----------|---------------|
|
||
| 1 | Draft persistence | Type in chat A, switch to B, restart app, open A — text restored, replyTo restored if the quoted message is still loaded. |
|
||
| 2 | Draft clear-on-send | Send a message → composer empty AND draft gone (no flash of restored text on re-mount). |
|
||
| 3 | Voice speed | Speed chip persists across app restart. Each playback respects the chip. |
|
||
| 4 | View-once hold | Press-and-hold a view-once → image visible. Release → image closes, tombstone appears. |
|
||
| 5 | Screenshot block | While image is held, Win+PrtScr captures black where the window is. |
|
||
| 6 | Whiteboard cursors | Two clients on the same whiteboard see each other's cursors with display name labels. Cursor fades after 2s idle. |
|
||
| 7 | Screen-share annotation | Two clients in a call, one shares screen — both can draw, both see strokes, strokes fade after 8s. |
|
||
| 8 | Soundboard outside call | Hotkey plays the sound locally even when not in a call. |
|
||
| 9 | Soundboard inside call | Hotkey plays the sound AND peers hear it (existing behaviour intact). |
|
||
| 10 | Animated avatar | GIF avatar plays animated in the user-bar + every Avatar render. Static avatars still cropped to square via canvas. |
|
||
|
||
## Out of scope (phase 9 candidates)
|
||
|
||
- Whiteboard in-progress stroke broadcast (Excalidraw-style "you see my pen as I draw" — 60 events/sec/user is meaningfully heavier than presence cursors; revisit after the live-cursor work is in production)
|
||
- Screen-share annotation persistence (currently strokes fade; no save-as-image yet)
|
||
- Skribbl-style game mode on top of the multi-user whiteboard
|
||
- Avatar decorations (animated frames around the avatar)
|
||
- View-once for video attachments (today the path is image-only)
|