ShadowMe · preview
Listens to you,
not the room.
Turn detectors hear whoever is talking. When someone nearby keeps talking, the agent waits for them — or answers the moment they stop. ShadowMe learns your voice from the first seconds of the call and decides only on your turn.
of the caller finishing · v1: 35%
more than 1 s · v1: 30%
learned at the call's start
v1: 11.5%
Live replay
Same framework. Same VAD. Different turn model.
A caller talks to an agent while someone else talks nearby. The same call streams through Pipecat's Silero VAD twice — once with voxturn v1, once with ShadowMe. Press play and watch who waits for the caller.
How it works
It knows whose pause it is.
Your voice, from the call's start
The first 10 s of speech in the call become the enrollment — a ReDimNet speaker embedding, computed once in 19 ms. No sign-up recording needed; one can be given instead.
Four ways to listen for you
The embedding enters voxturn's speech LLM (Qwen3-ASR-1.7B) as prompt tokens, as per-token voice matching, as a target-speaker VAD and inside the audio encoder. Training on counterfactual pairs — same audio, other enrollment, other answer — makes it compare voices instead of guessing "the louder one".
VAD speech isn't always yours
Pipecat's VAD fires on anyone. While it does, the plugin asks the model every 200 ms whether it is the user speaking; if not, that is the user's silence — so the turn can end while the bystander talks on.
Results · someone talks nearby
More calls answered right on time.
80 phone calls with someone talking nearby, each replayed through Pipecat and Silero VAD with either turn model.
| What the agent did | voxturn v1 | ShadowMe |
|---|---|---|
| Answered within 1 s of the caller finishing | 35% | 54% |
| Jumped in before the caller was done | 35% | 33% |
| Kept the caller waiting more than 1 s | 30% | 14% |
False cut-offs at a 300 ms budget, with someone else in the audio · lower is better
Barge-in: should the agent stop talking? · while it speaks
Real two-party calls (TurnBench dev). The user's real interruptions and backchannels, and a bystander who starts talking, as loud as the user, close by or across the room. Each detector may stop for at most 10% of the user's backchannels. Hardest case: a bystander's real interruption close by (4.5%).
Cut-offs: the share of the user's pauses where the agent replies although the user goes on, at a 300 ms answer-latency budget (eot-bench protocol). The enrollment is ~10 s of the user's speech from elsewhere in the same call (phone) or from another meeting of theirs (DiPCo, AMI). Meeting numbers are high for every model: after a finished sentence, the other participants decide whether the floor passes.
Everything voxturn v1 is tested on
No bystander? Still ahead of v1.
Every test on the voxturn v1 page, rerun for both models with the same scripts, without an enrollment where the test has none. Hover the chart.
eot-bench English: false cut-offs vs latency budget
Every language, 300 ms budget (open models)
Pipecat + Silero VAD, 150 eot-bench English turns · turns cut off mid-turn
| Test | voxturn v1 | ShadowMe | |
|---|---|---|---|
| eot-bench English: cut-offs at 300 ms / latency at 5% cut-offs | 11.5% / 669 ms | 10.4% / 563 ms | better |
| eot-bench, 14-language mean cut-offs at 300 ms | 19.9% | 17.5% | 13 of 14 languages |
| TurnBench dev, end of turn (recall at FPR ≤ 0.1) | 0.930 | 0.930 | level |
| TurnBench dev, interruptions (recall at FPR ≤ 0.1, median latency) | 0.960 · 337 ms | 0.951 · 506 ms | level, answers later |
| Easy-Turn test (Chinese, dialog state) | 88.3% | 89.4% | better |
| Non-verbal sounds recognised as not speech (SLR99) | 99.6% | 99.6% | level |
| TEN, text only: English / Chinese | 80.5% / 94.4% | 94.9% / 93.4% | English better |
| SemanticVAD, text only: end of turn, en / zh | 100% / 99.0% | 99.5% / 98.5% | level |
| SemanticVAD, text only: interrupt vs backchannel, en / zh | 80.5% / 88.4% | 82.0% / 88.2% | level |
| One decision on an NVIDIA L40S | 8.5 ms | 22 ms · 15 ms without an enrollment | slower |
"Level": no significant difference (text tests paired per sample, McNemar p > 0.05; TurnBench within 3 of ~350 events). ShadowMe is a 2.1B model (Qwen3-ASR-1.7B + ReDimNet-B3) against v1's 0.8B; following an enrolled user is trained on English only. Full tables on the model card.
Use it
One analyzer per call. No sign-up recording.
from pipecat_voxturn_personal import VoxTurnPersonalAnalyzer
from pipecat.turns.user_stop import TurnAnalyzerUserTurnStopStrategy
from pipecat.turns.user_turn_strategies import UserTurnStrategies
analyzer = VoxTurnPersonalAnalyzer() # captures the user's voice from the call's start
strategies = UserTurnStrategies(stop=[TurnAnalyzerUserTurnStopStrategy(turn_analyzer=analyzer)])from huggingface_hub import snapshot_download
import sys; path = snapshot_download("AustinXiao/voxturn-personal-turn-detector"); sys.path.insert(0, path)
from voxturn_personal import VoxTurnPersonal
model = VoxTurnPersonal.from_pretrained(path)
enr = model.call_enrollment() # once per call
user = enr.add(chunk, is_speech) # every chunk, with your VAD's speech flag
model.predict(last_8s, user, questions=("eot", "target_active"))# no GPU: the hosted demo on Hugging Face Spaces (pip install gradio_client)
from gradio_client import Client, handle_file
client = Client("AustinXiao/voxturn-demo")
client.predict(handle_file("user_voice.wav"), # ~10 s of the user, or the call's start
handle_file("audio_so_far.wav"), # others may talk
False, # True while the agent speaks -> barge_in
api_name="/personal_decide")
# {'eot': {...}, 'target_active': {'user': 0.02, 'other': 0.98, 'none': 0.0}, ...}
plot, s = client.predict(handle_file("call_start.wav"), handle_file("turn.wav"), "", 0.85, 0.8,
api_name="/personal_timeline")
s["reply_personal"], s["reply_v1"] # when each model would replyThe hosted API takes several seconds per call (a GPU is allocated per call): use it to try ShadowMe; run the model yourself in a live agent. Arguments and outputs: model card.