byGalax/ChatApp
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: file variety, user status, DND gate, drag-drop, mentions, link preview, emoji picker, soundboard, ringtone

Browse Source 
Attachments
- Video: native <video controls>, decrypted blob, metadata preload
- PDF: collapsible <object> viewer with download + preview toggle
- Generic: file card with lazy decrypt, mime-to-extension map
- MessageBubble dispatch: audio/image/video/pdf/generic by mime
- Composer accept widened from image/* to any; AttachmentPreview renders
  non-images as file cards

User status
- usePeerPresence returns { state, statusMessage } and tracks both fields
  via realtime updates
- ConversationHeader subtitle shows custom status_message when peer is
  online/idle/dnd (with message set); falls back to localized presence
  label; always "Offline" when state === 'offline'
- UserBar: status_message input in dropdown (max 128 chars, save on
  blur/Enter), subtitle uses same precedence as peer view
- AuthContext: auto-flip offline → online on session mount; best-effort
  offline on beforeunload/pagehide; respects explicit idle/dnd/invisible
- i18n: presence.status_placeholder (DE + EN)

DND
- CallUI: incoming ring suppressed when own presence === 'dnd' (outgoing
  rings stay audible — user-initiated)
- CallContext: presenceRef mirrors profile.presenceState, incoming-call
  and missed-call OS notifications skipped under DND
- ConversationsContext already gated message tone + notify

Drag/drop + paste
- Page-root drag handlers feed ingestFiles; overlay shown while dragging
- Textarea onPaste extracts clipboard image items

@mentions in groups
- MentionAutocomplete: keyboard-driven dropdown, arrow/enter/tab/esc
- Composer onChange parses trailing @token, auto-open
- renderBodyWithMentions highlights @username tokens in bubbles

Link previews
- Migration 20260421000003_link_previews.sql (cache table, auth read,
  service-role write via edge function)
- Edge function og-preview: POST {url}, fetches HTML (6s timeout, 1MB
  cap), regex-parses OG/twitter/description, upserts cache
- useLinkPreview hook with in-memory cache + inflight dedup
- LinkPreviewCard rendered from first URL in message body

Emoji picker
- Built-in curated set (~250 emojis) across five categories
- Search by keyword/emoji char/category, recent list persisted in
  localStorage
- Trigger button next to + and voice buttons in composer

Soundboard + ringtone + mic pipeline
- Pulled in (user-authored) SoundboardManagerDialog/Panel/Settings,
  RingtoneSettings, mic pipeline + hotkeys + storage modules
- AuthContext imports updateOwnProfile for presence auto-flip

Fixes
- AttachmentAudio min-width so bubbles don't collapse to 0px on peer
  side
- Focus flicker: visibility/online wake refresh throttled to 30s,
  focus listener dropped, loading flag only on first fetch
This commit is contained in:
Dennis Landmann
2026-04-21 09:13:30 +02:00
parent b89ec90813
commit 672c8738c7
34 changed files with 4394 additions and 100 deletions
Download Patch File Download Diff File Expand all files Collapse all files
apps/desktop/src/context/CallContext.tsx
+249 -45
View File
@@ -47,7 +47,15 @@ import {
getCallE2EESettings,
isE2EESupported,
} from '../lib/callE2EE';
import { createMicPipeline, type MicPipeline } from '../lib/micPipeline';
import { getParticipantVolume } from '../lib/participantVolumes';
import { startSoundboardHotkeys } from '../lib/soundboardHotkeys';
import { playEntry } from '../lib/soundboardPlayback';
import {
getPrefs as getSoundboardPrefs,
listSounds as listSoundboard,
updatePrefs as updateSoundboardPrefs,
} from '../lib/soundboardStorage';
import {
type DisplaySurfaceHint,
getPresetParams,
@@ -110,6 +118,11 @@ interface CallContextValue {
isDeafened: boolean;
/** identity -> their deafen state, received via data channel. */
remoteDeafen: Record<string, boolean>;
/** identity -> mute state. Broadcast from peer whenever mic-gain flips.
* Needed because we can't rely on LiveKit's native isMicrophoneEnabled —
* the mic pipeline keeps the track published with sound flowing even
* while the mic path is gain-silenced, so LK never sees "muted". */
remoteMute: Record<string, boolean>;
// Zero or more remote screen shares (LiveKit supports multiple simultaneous).
remoteScreenShares: RemoteScreenShare[];
// Remembers the conversation of the last call we left so a sidebar widget
@@ -139,6 +152,19 @@ interface CallContextValue {
dismissLastCall: () => void;
setCallMode: (mode: CallMode) => void;
setFocusedId: (id: string | null) => void;
/** Play a soundboard entry through the active call's mic pipeline.
* No-op when not connected. Default single-fire per id (spamming the
* hotkey cuts the previous instance); set overlap=true to layer. */
playSoundboard: (id: string, opts?: { overlap?: boolean }) => Promise<void>;
/** Stop every active sb source, or just the one matching `id` if given. */
stopSoundboard: (id?: string) => void;
/** Ids of soundboard entries currently emitting audio. Updated live so
* the in-call panel can show a stop icon on active pads. */
activeSoundboardIds: ReadonlySet<string>;
/** Apply new sb master/monitor gains to the live pipeline. Persists via
* updatePrefs in the storage module. */
setSoundboardMasterGain: (value: number) => Promise<void>;
setSoundboardMonitorGain: (value: number) => Promise<void>;
// Runtime mic switch. Persists choice so subsequent calls reuse it, and
// hot-swaps the input on an active call without a reconnect.
setAudioInputDevice: (deviceId: string | null) => Promise<void>;
@@ -157,7 +183,7 @@ function newCallId(): string {
}
export function CallProvider({ children }: { children: ReactNode }) {
const { session, device } = useAuth();
const { session, device, profile } = useAuth();
const { conversations } = useConversationsContext();
const myId = session?.user.id;
@@ -173,12 +199,18 @@ export function CallProvider({ children }: { children: ReactNode }) {
const [lastCallConversationId, setLastCallConversationId] = useState<string | null>(null);
const [callMode, setCallModeState] = useState<CallMode>('grid');
const [focusedId, setFocusedIdState] = useState<string | null>(null);
const [activeSoundboardIds, setActiveSoundboardIds] = useState<ReadonlySet<string>>(
() => new Set<string>(),
);
const signalChannelRef = useRef<RealtimeChannel | null>(null);
const presenceChannelRef = useRef<RealtimeChannel | null>(null);
const ringTimerRef = useRef<number | null>(null);
const soloTimerRef = useRef<number | null>(null);
const roomRef = useRef<Room | null>(null);
// Web Audio graph that mixes live mic + soundboard sources into a single
// published track. Created per call in joinRoom, destroyed in disconnectRoom.
const pipelineRef = useRef<MicPipeline | null>(null);
// Tracks whether the current call was ever in the connected state — needed
// so hangup/solo-timeout can emit a real duration message vs. "missed".
const everConnectedRef = useRef<boolean>(false);
@@ -191,12 +223,21 @@ export function CallProvider({ children }: { children: ReactNode }) {
// channel messages ({ type: 'presence', deafened: bool }). Exposed as
// state so consumer components re-render on change.
const [remoteDeafen, setRemoteDeafen] = useState<Record<string, boolean>>({});
const [remoteMute, setRemoteMute] = useState<Record<string, boolean>>({});
// Mirror of isMuted for use inside LiveKit-event callbacks that run outside
// the React component (ParticipantConnected rebroadcast etc).
const mutedRef = useRef<boolean>(false);
const stateRef = useRef<CallState>(state);
stateRef.current = state;
// Keep latest conversations accessible from signal-channel closures without
// re-subscribing the channel on every conversations update.
const conversationsRef = useRef(conversations);
conversationsRef.current = conversations;
// Mirror own presence state for use inside signal-channel callbacks. DND
// suppresses incoming-call OS notifications (ringtone is handled in CallUI
// which has direct access to the auth profile).
const presenceRef = useRef(profile?.presenceState ?? 'offline');
presenceRef.current = profile?.presenceState ?? 'offline';
// --- Helpers -----------------------------------------------------------
@@ -271,8 +312,25 @@ export function CallProvider({ children }: { children: ReactNode }) {
setIsDeafened(false);
deafenedActive = false;
setRemoteDeafen({});
setRemoteMute({});
setIsMuted(false);
mutedRef.current = false;
setIsE2EEActive(false);
// Tear down the mic pipeline AFTER LiveKit disconnects so the published
// track is unpublished cleanly first; then close AudioContext + stop
// raw mic + output tracks we own.
const pipeline = pipelineRef.current;
if (pipeline) {
try {
pipeline.destroy();
} catch {
/* ignore */
}
pipelineRef.current = null;
}
setActiveSoundboardIds(new Set<string>());
const pres = presenceChannelRef.current;
if (pres) {
try {
@@ -511,23 +569,36 @@ export function CallProvider({ children }: { children: ReactNode }) {
if (!participant?.identity) return;
try {
const text = new TextDecoder().decode(payload);
const msg = JSON.parse(text) as { type?: string; deafened?: boolean };
if (msg.type === 'presence' && typeof msg.deafened === 'boolean') {
const id: string = participant.identity;
const msg = JSON.parse(text) as {
type?: string;
deafened?: boolean;
muted?: boolean;
};
if (msg.type !== 'presence') return;
const id: string = participant.identity;
if (typeof msg.deafened === 'boolean') {
const deafened: boolean = msg.deafened;
setRemoteDeafen((prev) => {
if (prev[id] === deafened) return prev;
return { ...prev, [id]: deafened };
});
}
if (typeof msg.muted === 'boolean') {
const muted: boolean = msg.muted;
setRemoteMute((prev) => {
if (prev[id] === muted) return prev;
return { ...prev, [id]: muted };
});
}
} catch {
/* ignore malformed */
}
},
);
// When someone joins, re-send our current deafen state so they know.
// When someone joins, re-send our current presence (deafen + mute) so
// they know immediately instead of waiting for the next toggle.
r.on(RoomEvent.ParticipantConnected, () => {
void broadcastPresence(r, deafenedActive);
void broadcastPresence(r, deafenedActive, mutedRef.current);
});
// Track my own screen-share state via LocalTrack events so the toggle
@@ -557,18 +628,43 @@ export function CallProvider({ children }: { children: ReactNode }) {
}
try {
const inputId = getAudioSettings().inputDeviceId;
await r.localParticipant.setMicrophoneEnabled(true, {
echoCancellation: aParams.echoCancellation,
noiseSuppression: aParams.noiseSuppression,
autoGainControl: aParams.autoGainControl,
channelCount: aParams.stereo ? 2 : 1,
sampleRate: aParams.sampleRateHz,
// Plain string maps to `ideal` — if the device is gone we fall back
// to OS default instead of throwing NotFoundError.
...(inputId ? { deviceId: inputId } : {}),
// Grab the raw mic ourselves instead of going through LiveKit's
// setMicrophoneEnabled. The resulting MediaStreamTrack is routed
// through createMicPipeline, which mixes in soundboard buffers and
// exposes a single output track we hand to publishTrack. Mute / PTT
// are gain-based from here on, never track.enabled or device stop.
const rawStream = await navigator.mediaDevices.getUserMedia({
audio: {
echoCancellation: aParams.echoCancellation,
noiseSuppression: aParams.noiseSuppression,
autoGainControl: aParams.autoGainControl,
channelCount: aParams.stereo ? 2 : 1,
sampleRate: aParams.sampleRateHz,
...(inputId ? { deviceId: { ideal: inputId } } : {}),
},
video: false,
});
const rawTrack = rawStream.getAudioTracks()[0];
if (!rawTrack) throw new Error('no audio track from getUserMedia');
const pipeline = createMicPipeline(rawTrack);
pipelineRef.current = pipeline;
// Pull the user's last-saved soundboard gains onto the live pipeline
// before the first sound ever plays so nothing blasts at 100%.
try {
const prefs = await getSoundboardPrefs();
pipeline.setSoundboardGain(prefs.masterGain);
pipeline.setMonitorGain(prefs.monitorGain);
} catch (err: unknown) {
console.warn('getSoundboardPrefs failed', err);
}
await r.localParticipant.publishTrack(pipeline.outputTrack, {
source: Track.Source.Microphone,
red: true,
dtx: aParams.stereo ? false : true,
forceStereo: aParams.stereo,
});
} catch (micErr: unknown) {
console.error('setMicrophoneEnabled failed', micErr);
console.error('mic pipeline setup failed', micErr);
}
if (mediaKind === 'video') {
try {
@@ -807,12 +903,15 @@ export function CallProvider({ children }: { children: ReactNode }) {
}, []);
const toggleMute = useCallback(() => {
const r = roomRef.current;
if (!r) return;
const lp = r.localParticipant;
const shouldEnable = !lp.isMicrophoneEnabled;
void lp.setMicrophoneEnabled(shouldEnable).then(() => {
setIsMuted(!shouldEnable);
const pipeline = pipelineRef.current;
if (!pipeline) return;
setIsMuted((prev) => {
const nextMuted = !prev;
pipeline.setMicGain(nextMuted ? 0 : 1);
mutedRef.current = nextMuted;
const r = roomRef.current;
if (r) void broadcastPresence(r, deafenedActive, nextMuted);
return nextMuted;
});
}, []);
@@ -921,7 +1020,7 @@ export function CallProvider({ children }: { children: ReactNode }) {
// headphones-off badge. Data channel works on any LiveKit server
// version, unlike `setAttributes` which requires a newer server.
const r = roomRef.current;
if (r) void broadcastPresence(r, next);
if (r) void broadcastPresence(r, next, mutedRef.current);
return next;
});
}, []);
@@ -957,9 +1056,14 @@ export function CallProvider({ children }: { children: ReactNode }) {
let globalRegisteredFor: string | null = null;
const setMic = (on: boolean) => {
const pipeline = pipelineRef.current;
if (!pipeline) return;
pipeline.setMicGain(on ? 1 : 0);
const nextMuted = !on;
mutedRef.current = nextMuted;
setIsMuted(nextMuted);
const r = roomRef.current;
if (!r) return;
void r.localParticipant.setMicrophoneEnabled(on).then(() => setIsMuted(!on));
if (r) void broadcastPresence(r, deafenedActive, nextMuted);
};
const pressPtt = () => {
@@ -1088,14 +1192,16 @@ export function CallProvider({ children }: { children: ReactNode }) {
conv?.peer?.displayName ??
'…';
const isGroup = conv?.type === 'group';
void notify({
title: isGroup
? (conv?.name ?? 'Gruppenanruf')
: 'Eingehender Anruf',
body: isGroup
? callerName + ' ruft die Gruppe'
: callerName + ' ruft dich an',
});
if (presenceRef.current !== 'dnd') {
void notify({
title: isGroup
? (conv?.name ?? 'Gruppenanruf')
: 'Eingehender Anruf',
body: isGroup
? callerName + ' ruft die Gruppe'
: callerName + ' ruft dich an',
});
}
break;
}
case 'accept':
@@ -1136,10 +1242,12 @@ export function CallProvider({ children }: { children: ReactNode }) {
conv?.members.find((m) => m.userId === cur.fromUserId)?.profile?.displayName ??
conv?.peer?.displayName ??
'…';
void notify({
title: 'Verpasster Anruf',
body: callerName + ' hat aufgelegt',
});
if (presenceRef.current !== 'dnd') {
void notify({
title: 'Verpasster Anruf',
body: callerName + ' hat aufgelegt',
});
}
setState({ kind: 'idle' });
}
break;
@@ -1156,16 +1264,96 @@ export function CallProvider({ children }: { children: ReactNode }) {
setFocusedIdState(id);
}, []);
const markActive = useCallback((id: string, on: boolean) => {
setActiveSoundboardIds((prev) => {
const has = prev.has(id);
if (on && has) return prev;
if (!on && !has) return prev;
const next = new Set(prev);
if (on) next.add(id);
else next.delete(id);
return next;
});
}, []);
const playSoundboard = useCallback(
async (id: string, opts?: { overlap?: boolean }) => {
const pipeline = pipelineRef.current;
if (!pipeline) return;
const entries = await listSoundboard();
const entry = entries.find((e) => e.id === id);
if (!entry) return;
const handle = await playEntry(pipeline, entry, {
...(opts?.overlap !== undefined ? { overlap: opts.overlap } : {}),
onEnded: () => markActive(id, false),
});
if (handle) markActive(id, true);
},
[markActive],
);
const stopSoundboard = useCallback(
(id?: string) => {
const pipeline = pipelineRef.current;
if (!pipeline) return;
pipeline.stopAll(id);
if (id) {
markActive(id, false);
} else {
setActiveSoundboardIds(new Set<string>());
}
},
[markActive],
);
const setSoundboardMasterGain = useCallback(async (value: number) => {
const prefs = await updateSoundboardPrefs({ masterGain: value });
pipelineRef.current?.setSoundboardGain(prefs.masterGain);
}, []);
const setSoundboardMonitorGain = useCallback(async (value: number) => {
const prefs = await updateSoundboardPrefs({ monitorGain: value });
pipelineRef.current?.setMonitorGain(prefs.monitorGain);
}, []);
// Global soundboard hotkey registration — runs only while connected so the
// OS-level shortcuts don't fire when the user is outside of a call.
useEffect(() => {
if (state.kind !== 'connected') return;
const teardown = startSoundboardHotkeys((id) => {
void playSoundboard(id);
});
return teardown;
}, [state.kind, playSoundboard]);
const setAudioInputDevice = useCallback(async (deviceId: string | null) => {
updateAudioSettings({ inputDeviceId: deviceId });
const r = roomRef.current;
if (!r) return;
const pipeline = pipelineRef.current;
if (!pipeline) return;
try {
// LiveKit API: `switchActiveDevice(kind, deviceId)` hot-swaps without a
// reconnect. Pass empty string or `default` to revert to OS default.
await r.switchActiveDevice('audioinput', deviceId ?? 'default');
// We own the mic track (see joinRoom pipeline setup), so LiveKit's
// switchActiveDevice no longer applies. Fetch a new raw track with the
// updated deviceId + the same quality constraints, then hand ownership
// to the pipeline. It disconnects the old source, stops the old track,
// and rewires micGain onto the new source — the published track stays
// stable so peers don't see a republish.
const aParams = getAudioQualityParams(getAudioSettings().quality);
const newStream = await navigator.mediaDevices.getUserMedia({
audio: {
echoCancellation: aParams.echoCancellation,
noiseSuppression: aParams.noiseSuppression,
autoGainControl: aParams.autoGainControl,
channelCount: aParams.stereo ? 2 : 1,
sampleRate: aParams.sampleRateHz,
...(deviceId ? { deviceId: { ideal: deviceId } } : {}),
},
video: false,
});
const newTrack = newStream.getAudioTracks()[0];
if (!newTrack) throw new Error('no audio track for device');
pipeline.replaceMicTrack(newTrack);
} catch (err: unknown) {
console.error('switchActiveDevice(audioinput) failed', err);
console.error('setAudioInputDevice failed', err);
}
}, []);
@@ -1229,6 +1417,7 @@ export function CallProvider({ children }: { children: ReactNode }) {
isCameraEnabled,
isDeafened,
remoteDeafen,
remoteMute,
remoteScreenShares,
lastCallConversationId,
callMode,
@@ -1249,6 +1438,11 @@ export function CallProvider({ children }: { children: ReactNode }) {
setFocusedId,
setAudioInputDevice,
setAudioOutputDevice,
playSoundboard,
stopSoundboard,
activeSoundboardIds,
setSoundboardMasterGain,
setSoundboardMonitorGain,
}),
[
state,
@@ -1260,6 +1454,7 @@ export function CallProvider({ children }: { children: ReactNode }) {
isCameraEnabled,
isDeafened,
remoteDeafen,
remoteMute,
remoteScreenShares,
lastCallConversationId,
callMode,
@@ -1280,6 +1475,11 @@ export function CallProvider({ children }: { children: ReactNode }) {
setFocusedId,
setAudioInputDevice,
setAudioOutputDevice,
playSoundboard,
stopSoundboard,
activeSoundboardIds,
setSoundboardMasterGain,
setSoundboardMonitorGain,
],
);
@@ -1297,10 +1497,14 @@ export function useCall(): CallContextValue {
// audio elements. Toggled by toggleDeafen in sync with the React state.
let deafenedActive = false;
async function broadcastPresence(room: Room, deafened: boolean): Promise<void> {
async function broadcastPresence(
room: Room,
deafened: boolean,
muted: boolean,
): Promise<void> {
try {
const payload = new TextEncoder().encode(
JSON.stringify({ type: 'presence', deafened }),
JSON.stringify({ type: 'presence', deafened, muted }),
);
await room.localParticipant.publishData(payload, { reliable: true });
} catch (err: unknown) {