AI 资讯 · 每日前沿动态

第二代骁龙音频平台至尊版发布：走出手机的AI正在接管你的耳机与PC

新浪科技

2026-09-24T09:11:21+08:00 · 语音音频,模型发布

The Curious Power of Punctuation

2026-09-21T10:00:00.000Z · 大模型,算力芯片,AI应用,具身智能,OpenAI,Google,Meta,Microsoft,语音音频,对话助手,搜索RAG,扩散模型,强化学习,招聘HR,网络安全,榜单评测,开发者生态

TiAb Review Plugin: A Browser-Based Tool for AI-Assisted Study Selection in Systematic Reviews

arXiv cs.LGarXiv:2604.08602v2 Announce Type: replace-cross Abstract: Server-based screening tools impose subscription costs, while open-source alternatives require coding skills, and full-text screening has remained outside the scope of no-code open-source tools. We developed TiAb Review Plugin, an open-source Chrome browser extension that provides no-code, serverless artificial intelligence (AI)-assisted study selection covering both title and abstract (T&A) screening and full-text screening. It uses Google Sheets as a shared database and Google Drive as a PDF store, and users supply their own large language model (LLM) API key. For T&A screening, it offers manual review, LLM batch screening, and machine learning (ML) active learning. For full-text screening, it retrieves open-access PDFs from PubMed Central, Europe PMC, Unpaywall, OpenAlex, and publisher pages, supports blinded dual review with structured exclusion reasons and adjudication, optionally obtains an LLM judgment with page-anchored evidence, and computes PRISMA 2020 flow counts. We re-implemented the default ASReview algorithm (TF-IDF with Naive Bayes) in TypeScript and compared it with the Python original using 10-fold cross-validation on six datasets. For LLM T&A screening, we compared 16 parameter configurations on a benchmark dataset, validated the best (Gemini 3.0 Flash, low thinking budget, TopP 0.95) on five public datasets (1,038 to 5,628 records; 0.5% to 2.0% prevalence), and benchmarked nine further models from four developers. The TypeScript classifier produced top-100 rankings identical to ASReview on all six datasets. LLM T&A screening achieved recall of 94% to 100% with precision of 2% to 15%, and work saved over sampling at 95% recall (WSS@95) of 46.3% to 89.3%. No additional model exceeded the 96.1% recall of the reference configuration; the most recent models traded recall for precision. The classification accuracy of the full-text stage has not yet been evaluated.

2026-09-24 04:00:00 · 大模型,Google,语音音频,扩散模型,强化学习,模型评测,招聘HR,榜单评测,论文,开发者生态

Backdoors Leave Structural Traces: FedMAST for Backdoor Detection and Containment in Federated Learning

arXiv cs.LGarXiv:2609.27760v1 Announce Type: new Abstract: Federated learning enables distributed training of a shared model without requiring clients to share their raw data. However, its reliance on the integrity of the client-submitted updates exposes the global model to stealthy backdoor poisoning. Although existing defenses often inspect isolated evidence sources, stealth-constrained attacks can adapt to these signals. In this paper, we show that such attacks can suppress isolated anomaly signals, but their poisoned updates still leave residual structural traces. We propose FedMAST, a Federated Multi-Axis Structural Tracing defense for backdoor detection in federated learning. FedMAST scores client updates using complementary structural, spectral, and historical evidence and then applies tiered filtering and round-level containment to limit adversarial influence. To capture traces that isolated signals may miss, FedMAST uses squeeze-pair coherence scoring to expose coupled feature distortions and signed spectral-drift tracking to reveal persistent directional changes over time. Across six federated backdoor attacks, namely Constrain-and-Scale, Neurotoxin, BC-Layers, LGA, DBA, and 3DFed, FedMAST achieves lower ASR than baseline defenses in all nine evaluated attack--defense comparisons. Across the complete 200-round runs, it attains an average ASR of 1.51% while maintaining 94.84% average main-task accuracy. Under the method-aware CovertLayers attack, FedAvg, MultiKrum, AlignIns, and FLAME yield full-run ASRs of 100.00%, 99.67%, 99.53%, and 32.84%, respectively. FedMAST achieves the lowest ASR among all evaluated methods, reducing it to 1.53% while maintaining 92.26% main-task accuracy.

2026-09-24 04:00:00 · AI应用,语音音频,搜索RAG,联邦学习,论文

A Systematic Benchmark of Explainable Methods for Temporal Attribution in Sequential Recommendation Systems

arXiv cs.LGarXiv:2609.27201v1 Announce Type: new Abstract: Sequential RecSys are central to modern personalization, exploiting user's historical interaction sequences to drive next-step decisions. Deep learning models, particularly CNN and Transformer-based architectures, have proven highly effective at capturing temporal dependencies in these histories. For transparency and trust, understanding which past interactions drive a given recommendation is increasingly important --- both for developers auditing model behavior and for users seeking a rationale. However, the non-linearities that give these models their predictive power also render them black boxes, making it difficult to attribute decisions to specific interactions. While gradient-based, perturbation-based, and attention-based explainability methods exist, a systematic benchmark of their faithfulness for sequential recommendation is missing. We address this gap by introducing a dual-model masking metric in which one model supplies per-timestep attribution scores and a separately trained, masking-robust probe measures the resulting change in predicted probability. Using this metric, we benchmark ten XAI methods across CNN, Transformer, SASRec, and BERT4Rec backbones on KuaiRand and MovieLens, complemented by analyses of temporal attribution patterns, item popularity confounding, and robustness to input corruption. Our key findings are: (1) gradient-based methods, particularly GradientSHAP and Integrated Gradients, yield the most faithful and robust attributions; (2) raw attention weights are unreliable, but gradient-weighted attention restores faithfulness on shorter sequences, with degradation on longer horizons as softmax attention probabilities converge toward uniform importance scores, diminishing the method's ability to identify informative interactions; and (3) temporal attribution patterns in faithful methods reflect genuine task structure rather than recency or popularity bias.

2026-09-24 04:00:00 · xAI,语音音频,Transformer,扩散模型,模型评测,端侧AI,论文

When Helpful Context Leaks: Privacy Risks in Domain-Adapted ASR

arXiv cs.CLarXiv:2605.28211v2 Announce Type: replace Abstract: SpeechLLMs are increasingly deployed in professional settings where domain customisation is standard practice: users supply context in prompts with sensitive information, fine-tune on proprietary recordings, or both. We identify and systematically investigate an overlooked privacy risk of such customisation: a model adapted to recognise domain-specific terminology can be nudged into transcribing a phonetically similar word from its context or training data, even when a different word is spoken, thereby leaking private information. To evaluate this risk, we propose a technique to automatically construct benchmarks of such attacks and apply it to measure leakage rates across two customisation mechanisms, prompting and fine-tuning. Both mechanisms cause measurable leakage, compounding when combined. We evaluate a prompt-level mitigation strategy and analyse the accuracy-leakage trade-off across customisation approaches, finding that fine-tuning without context prompts offers the best balance.

2026-09-24 04:00:00 · 大模型,语音音频,微调蒸馏,模型评测,提示工程,论文

RapidUn: Influence-Driven Parameter Reweighting for Efficient Large Language Model Unlearning

arXiv cs.CLarXiv:2512.04457v3 Announce Type: replace Abstract: Machine unlearning for large language models (LLMs) remains challenging because full retraining is costly, while approximate methods often struggle to remove targeted behaviors without degrading retained utility, especially under limited post-deployment supervision. We consider a practical PEFT setting for targeted behavioral contamination removal with a small forget set, a limited retain buffer, and LoRA-only updates, and propose RapidUn, an influence-guided framework that converts cross-sample influence estimates into fixed sample-specific weights for weighted LoRA unlearning. Across Llama-3-8B on Dolly-15k and Alpaca-57k, with cross-model validation on Mistral-7B + Dolly-15k, RapidUn achieves lower seen-trigger and OOD-trigger-family ASR than Fisher, GA, and LoReUn while maintaining competitive clean utility. On Llama-3-8B + Alpaca-57k, it achieves a 77x wall-clock speedup over the clean-corpus LoRA retraining reference. Complementary TOFU, semantic LLM-judge, and IFEval evaluations further support the effectiveness of influence-guided sample reweighting beyond the controlled trigger benchmark.

2026-09-24 04:00:00 · 大模型,Meta,Mistral,语音音频,强化学习,微调蒸馏,模型评测,论文,开发者生态

TEMPS: Temporal Sentence Embeddings for Temporal Information Retrieval

arXiv cs.CLarXiv:2609.28048v1 Announce Type: new Abstract: Modern information retrieval (IR) systems rarely represent time, yet many information needs depend on it: in clinical, journalistic, and legal search, when an event occurred can decide whether a document is relevant. Dense retrievers and Retrieval-Augmented Generation (RAG) pipelines match queries to documents well on topic but poorly on time, so they surface content that is on-topic yet temporally wrong. We introduce Temporal Textual Similarity (TTS), a task that measures how well two anchored texts align in time, independent of their topical similarity. We then present TEMPS (Temporal Embedding Model for Precise Search), a modular temporal branch that attaches to a frozen semantic retriever and trains on that signal. It resolves anchored temporal expressions to intervals and moment-matches each one to a Gaussian; the resulting ordering supervises an anchor-date-conditioned encoder, whose score we fuse with the semantic score at inference. Grounding supplies the supervision, so training uses no hand-labeled temporal data. The temporal score itself is the Gaussian-KL inclusion measure from distributional embeddings; what TEMPS adds is the grounding and the moment-matched supervision. On three temporal benchmarks, TEMPS improves MRR for every semantic backbone tested and, on TS- Retriever, lifts R@1 from 19.92 to 25.39 over the prior temporal state of the art.

2026-09-24 04:00:00 · AI应用,语音音频,搜索RAG,扩散模型,模型评测,向量数据库,招聘HR,榜单评测,论文

Six Layers Less: Encoder Pruning for Whisper with Label-Free Recovery

arXiv cs.CLarXiv:2609.27980v1 Announce Type: new Abstract: Pruning large pre-trained transformer-based ASR models such as OpenAI's Whisper has seen great adoption, as pruning the decoder led to significant end-to-end transcription speedups. For instance, the {\tt whisper-large-v3-turbo} variant reduced the decoder from 32 to 4 layers, while Distill-Whisper similarly reduced the decoder to only 2 layers. Although some attention has been put towards reducing the size of the encoder, no approach has seen wide adoption. This could be due to the need for custom inference implementations to take advantage of the compressed model. We present an approach that ranks encoder layers by the leave-one-layer-out change in Word Error Rate (WER). The six layers that cause the least change are removed, corresponding to $18.5\%$ of the encoder stack. The pruned model requires no custom inference code as it is simply a more shallow encoder with fewer layers. We further distill using unlabeled monolingual speech data to recover performance degradation caused by the zero-shot layer pruning. Mean WER across four languages increases to $20.1\%$ after distillation, compared to $21.9\%$ zero-shot, going from a baseline of $18.2\%$. We release all of our code (https://github.com/rasgaard/whisper-encoder-layer-prune) and the pruned model (https://huggingface.co/rasgaard/whisper-large-v3-turbo-encoder-pruned).

2026-09-24 04:00:00 · 开源,OpenAI,语音音频,Transformer,微调蒸馏,论文

Planned Test-Time Scaling with Coordinated Reasoning Paths

arXiv cs.CLarXiv:2609.27374v1 Announce Type: new Abstract: Test-time scaling with parallel branches is widely adopted to improve performance on challenging reasoning tasks. The predominant approach, repeated sampling, draws branches independently from a single policy, which can produce redundant attempts and thereby limit the gains from additional inference compute. To address this limitation, we propose Planned Test-Time Scaling (PTTS), which replaces independent sampling with a coordinated joint policy: a planner generates a solution outline for each branch, steering the branches toward distinct reasoning paths, and an executor produces a full solution conditioned on each outline. Formally, we show that PTTS strictly generalizes repeated sampling and, in a stylized setting, provably promotes coverage of complementary reasoning modes and yields better pass@k scaling. We instantiate PTTS on top of strong reasoning models, keeping them fixed as executors while replacing repeated sampling with PTTS inference to further enhance test-time scaling. Concretely, we develop two variants: PTTS-ZS prompts a model to jointly generate outlines for all branches in a single autoregressive pass, while PTTS-RL directly optimizes the planner against the pass@k reward using truncated execution rollouts for efficient training and a sharper reward signal. Across five mathematical reasoning benchmarks with Qwen3-1.7B and 4B, PTTS-ZS improves pass@64 over repeated sampling by up to 6.7 points, while PTTS-RL further increases the gain to up to 13.4 points. Further analysis indicates that broader coverage of distinct reasoning paths contributes to these gains. Overall, PTTS provides a general framework for improving test-time scaling by coordinating reasoning branches, with zero-shot and trainable instantiations that yield substantial performance gains.

2026-09-24 04:00:00 · 大模型,AI应用,阿里巴巴,语音音频,推理思考,搜索RAG,扩散模型,模型评测,提示工程,榜单评测,论文

Ruby-ASR: Evidence-Preserving Supervision for Joint Orthographic and Lexical-Reading Recognition

arXiv cs.CLarXiv:2609.27289v1 Announce Type: new Abstract: Conventional Japanese automatic speech recognition (ASR) is supervised by an orthographic transcript, although the same written form can correspond to different lexical readings realized in speech. Such utterances receive an identical target, so their reading distinction is absent from the supervision interface and cannot be recovered reliably by post-hoc text-only grapheme-to-phoneme conversion. We present Ruby-ASR, which refines the conventional target into a span-bound orthographic--lexical-reading sequence. Unlike separate full-sentence orthographic and phonological outputs, the ruby representation locally binds each written span to its realized reading and permits deterministic recovery of both views. We instantiate the target under subtitle-style and verbatim-style transcription conventions using a Qwen3-ASR backbone; a mora-level CTC objective provides auxiliary monotonic reading supervision. The experimental results across five Japanese benchmarks show that refining the recognition target can improve lexical-reading recovery without sacrificing readable orthographic transcription. We release the checkpoints and inference code.

2026-09-24 04:00:00 · 大模型,算力芯片,Google,阿里巴巴,语音音频,模型评测,论文

NADI 2026: The Second Multidialectal Arabic Speech Processing Shared Task

arXiv cs.CLarXiv:2609.27086v1 Announce Type: new Abstract: NADI 2026 is the seventh edition of the Nuanced Arabic Dialect Identification (NADI) shared task series and the second dedicated to multidialectal Arabic speech processing. This edition comprises five tasks and eight subtasks spanning Automatic Speech Recognition (ASR), Spoken Dialect Identification (SDID), Text-to-Speech (TTS), Spoken Language Translation (SLT), and Spoken Language Understanding (SLU). NADI 2026 emphasizes realistic evaluation through low-bandwidth, mixed-dialect, code-switched, out-of-domain, and zero-shot settings, while introducing TTS, SLT, and SLU to the series for the first time. The shared task attracted 21 participating teams from at least 13 countries, with 48 test-phase submissions and 14 submitted system-description papers. Results show that out-of-domain generalization remains a major bottleneck and highlight the effectiveness of recent Arabic-specialized speech models, multimodal dialect identification approaches, and ensemble methods. Overall, NADI 2026 provides a broader and more challenging benchmark for robust Arabic dialect speech processing.

2026-09-24 04:00:00 · 语音音频,多模态,扩散模型,模型评测,招聘HR,论文

Kyutai Releases Voice of Reason: A Speech-Native Model that Solves Spoken Math with Reinforcement Learning

MarkTechPost

Kyutai has released Voice of Reason, 2 open-weight speech-to-speech models that solve math problems out loud. Both start from GLM-4-Voice-9B and add supervised fine-tuning (SFT) and reinforcement learning (RL). There is no transcription step and no separate text LLM in the loop. On spoken GSM8K, accuracy climbs from 27.3% for the base model to 77.1%.

Is it deployable? Yes, for self-hosting. Kyutai ran both BF16 checkpoints on a single H100. You also need the GLM-4-Voice repo for its speech tokenizer and decoder. Weights inherit the GLM-4-Voice license, and no Hugging Face inference provider hosts them yet.

Why Speech Models Lag on Math

Cascaded pipelines (speech-to-text, text LLM, text-to-speech) still lead on reasoning. However, each stage adds latency, and the pipeline loses paralinguistic cues like tone. Speech-native models must emit audio at regular intervals to stay interactive. That limits how many hidden reasoning tokens they can afford.

Base GLM-4-Voice scores 27.3% on GSM8K. The earlier STITCH method raised that to 58.7% by adding reasoning chunks. The research team call their work the first application of RL to math reasoning in speech-native models.

How the Training Works

GLM-4-Voice interleaves its output: 13 text tokens, then 26 audio tokens, repeating.

Stage-1 SFT: Training uses 150,616 problems from Orca-Math. Qwen3-235B rewrote each problem for speech. Kyutai’s DSM TTS then voiced them in many voices. SFT alone lifts GLM-4-Voice from 27.3% to 61.7%.
Stage-2 RL: For each spoken question, the model samples 4 replies at temperature 0.9. A judge, Qwen3-235B-A22B-2507, scores the decoded text with a binary reward. The judge never sees the reference answer. On 100 hand-checked cases, it agreed with humans 88% of the time.

Rewards are centered within each group, forming a group-relative REINFORCE objective. It is related to GRPO but drops PPO clipping and KL regularization. Training ran on 16 H100 GPUs, with 1,500 RL updates.

2 design choices matter most:

Temperature correction: Logits are divided by the sampling temperature before the log-softmax in the loss. Without it, GSM8K collapsed from 65.5% to 12.3%.
Audio-token merging: At each audio position, all audio-vocabulary probabilities are summed into 1 abstract token. The loss asks only whether audio came next, not which audio token. The paper proves this estimator is unbiased and lower-variance under a value-invariance assumption.

Interactive Explainer

&&&

The 2 Released Checkpoints

glm-4-voice-of-reason-9b answers directly, with no extra reasoning tokens. Any step-by-step working is spoken aloud.
glm-4-voice-of-reason-stitch-9b writes silent 100-token reasoning chunks between spoken blocks. Kyutai says later chunks are generated while earlier speech plays, so thinking adds no extra latency. In the STITCH-R layout used here, the first reasoning chunk precedes the first spoken block.

Results

Model	Params	GSM8K (%)
PersonaPlex (full-duplex)	8B	3.2
GLM-4-Voice	9B	27.3
STITCH (Chiang et al.)	9B	58.7
Voice of Reason	9B	65.5 ± 1.1 (70.3 released)
Voice of Reason (Stitch)	9B	74.8 ± 1.1 (77.1 released)
Qwen2.5-Omni (text output)	7B	84.7
Qwen3-Omni (text output)	30B	94.6
Cascaded ASR-LLM-TTS-ASR	31B LLM	95.7

Paper scores use top-k 50 decoding, averaged over 3 seeds. Removing top-k gives 70.3% and 77.1%. The released checkpoints correspond to these runs. The omni and cascaded systems are larger top lines, not matched comparisons.

Other findings:

Gains survive in real speech: Transcribed with Qwen3-ASR-1.7B, the Stitch model scores 72.0 ± 1.9%.
Naturalness holds: UTMOSv2 moved from 4.067 to 4.069 (direct) and 4.174 to 4.164 (Stitch) after RL.
Gains do not come from longer answers: RL cut the direct model’s average reply from 41.9 to 36.4 seconds. Stitch reasoning tokens rose only from 167 to 176.
RL helps most with little data: With 10% of SFT data, a longer RL run reached 58.5%. SFT alone scored 43.9%.
General knowledge dips: Spoken TriviaQA fell from 40.6% to 34.0% for the direct model. The authors attribute this mainly to full-data SFT, not RL.
Contamination check: The team dropped AddSub, MultiArith, SingleEQ and SVAMP from evaluation. Of 678 checked questions, 54.0% overlapped with Orca-Math at the paraphrase level.

Evaluation audio came from GPT-4o-mini-TTS, a different TTS system than the training audio. GPT-4o served as the evaluation judge.

Key Takeaways

Kyutai’s RL recipe lifts GLM-4-Voice from 27.3% to 77.1% on spoken GSM8K.
The direct model reaches 70.3% without reasoning tokens, beating STITCH’s 58.7%.
Temperature correction is critical: removing it crashed accuracy to 12.3%.
Speech naturalness holds after RL; TriviaQA drops, mostly from SFT.
Both 9B checkpoints are open on Hugging Face and run on 1 H100.

Check out the Paper, the direct model and the Stitch model. All credit goes to the researcher of this project. Also, feel free to follow us on

2026-09-23 06:33:18 · 大模型,算力芯片,AI应用,具身智能,开源,OpenAI,Google,阿里巴巴,智谱,语音音频,推理思考,搜索RAG,扩散模型,强化学习,微调蒸馏,招聘HR,榜单评测,论文

NVIDIA Releases Nemotron 3 Diarization: A 100M-Parameter Open-Weight Model That Tracks 8 Speakers in Real Time

MarkTechPost

NVIDIA has released Nemotron 3 Diarization, an open-weight speaker diarization model on Hugging Face. It answers one question about any conversation: who spoke when. The 100M-parameter model tracks up to 8 speakers, including when voices overlap. One checkpoint handles both offline recordings and real-time streaming.

Is it deployable? Yes. The weights are released under the OpenMDW License 1.1, which permits commercial use. It runs on Linux through NVIDIA NeMo, using Ampere, Ada Lovelace, Hopper, or Blackwell GPUs.

Why Speaker Diarization?

Automatic speech recognition (ASR) gives you the words. It does not tell you who said them. Without attribution, a summarizer cannot tell who made a commitment or who raised an objection.

Diarization outputs the time intervals where each speaker is active. Those timestamps combine with ASR output to produce a speaker-attributed transcript. Meeting tools, call analytics, podcast pipelines, and voice-agent memory all depend on this step.

What Changed From Streaming Sortformer

NVIDIA’s earlier Streaming Sortformer checkpoint, diar_streaming_sortformer_4spk-v2.1, supported 4 speakers. Nemotron 3 Diarization doubles that limit to 8. According to NVIDIA’s announcement, the target is messy multi-party audio where people talk at once.

How the Architecture Works

The model accepts 16 kHz, single-channel audio in .wav, .flac, .opus, or .mp3 format. It converts the audio into Mel-spectrogram features with a 10 ms step. The features are stacked by a factor of 8, which produces 80 ms encoder frames.

A 31-layer Transformer encoder with rotary positional embeddings (RoPE) processes those frames. A Conv1D layer then upsamples the predictions back to 10 ms resolution. The output is a [T, 8] tensor of per-speaker activity probabilities.

This design handles overlap directly. If 2 people talk at the same time, 2 channels activate in the same frame.

The model follows the Sortformer approach of ordering speakers by arrival time. The first new voice takes channel 1, the next takes channel 2, and so on. This keeps labels stable across streaming chunks, so the model does not have to re-match speakers to channels for every chunk.

Streaming uses 2 memory mechanisms. The Arrival-Order Speaker Cache (AOSC) keeps speaker information from earlier chunks. A FIFO queue supplies recent frame context. The labels are anonymous, and mapping them to real identities is left to downstream applications.

4 Latency Operating Points

Input-buffer latency equals (chunk + right context) × 80 ms. The table uses DIHARD III full-set DER and batch-32 compiled throughput from the model card.

Configuration	Buffer latency	DIHARD III DER	RTFx (batch 32, compiled)
Offline style	30.4 s	12.73%	15,113×
Low latency	1.04 s	13.18%	865×
Very low latency	0.64 s	13.28%	579×
Ultra-low latency	0.32 s	13.55%	292×

This latency excludes compute, networking, and ASR time. The model can technically run with an 80 ms buffer, but 0.32 s is the lowest recommended setting.

Benchmark Results

In Voice Arena’s initial Diarization-Bench results, the model ranked first among 12 systems and 17 configurations. The test covered 139 English conversations totaling about 22 hours. It scored 14.72% DER against 19.3% for the next-ranked system, roughly a 24% relative reduction. NVIDIA notes these results may change once Voice Arena completes its Version 1 evaluation.

Against the 4-speaker baseline at 1.04 s latency, DER dropped on all 8 evaluation conditions. Relative reductions ranged from 9.0% on CALLHOME-Part2 to 65.2% on NOTSOFAR1 MHM. The unweighted mean across the 8 conditions was 41.0%.

There is one regression. On 2-speaker CALLHOME at 30.4 s, DER rose from 5.68% to 5.98%. Full-set CALLHOME-Part2 still improved from 10.32% to 9.10%.

Throughput also jumped. At 30.4 s, the model reached 15,113× RTFx versus 2,619× for the baseline. The tests used BF16 on an NVIDIA RTX PRO 5000 with torch.compile(). These are batched numbers, not single-stream application latency.

Training Data

Training combined about 10,000 hours of real conversations with 82,611 hours of simulated multi-talker mixtures. The mix included real-world multi-speaker audio licensed from David AI. Adding the David AI data cut compound DER from 11.19% to 10.42%. The licensed source audio for the simulated mixtures spans 21 languages.

Getting Started

Install NVIDIA NeMo Speech with Python 3.12 or later:

Copy CodeCopiedUse a different Browser

uv pip install 'nemo-toolkit[asr]'

Copy CodeCopiedUse a different Browser

from nemo.collections.asr.models impor

2026-09-23 18:17:14 · 算力芯片,AI应用,具身智能,开源,Google,NVIDIA,语音音频,Agent智能体,搜索RAG,Transformer,扩散模型,强化学习,模型评测,向量数据库,端侧AI,招聘HR,论文

Google Releases Gemini 3.8 Flash TTS and Flash-Lite TTS With Prompt-Based Voice Design

MarkTechPost

Google has released Gemini 3.8 Flash TTS and Gemini 3.8 Flash-Lite TTS, 2 new text-to-speech models in its Gemini Audio family. Google calls them its most expressive audio generation models yet. Flash TTS targets creative direction and character voices. Flash-Lite TTS targets high-volume, cost-efficient production. Both let developers direct delivery line by line using natural language.

Is it deployable? Yes, both models are rolling out now through the Gemini API and Google AI Studio. Access is API-only, with no open weights for self-hosting. Enterprise API access via Gemini Enterprise is listed as coming soon.

What Google Shipped

The release splits TTS into 2 tiers with shared direction controls:

Gemini 3.8 Flash TTS is built for deep creative direction and character design. Target uses include gaming, immersive audiobooks, podcasts and interactive media. It offers granular control over acting cues, pacing, dialect shifts and backchanneling.
Gemini 3.8 Flash-Lite TTS is built for high-volume, cost-efficient scale. Google positions it for dubbing, audio content creation and expressive voice agents. It offers fine-grained control over tone, pacing and expressive nuance.

In AI Studio, the playground links use the model identifiers gemini-3.8-flash-tts and gemini-3.8-flash-lite-tts.

Voice Design From a Text Prompt

Previous Gemini TTS offered 30 original voices. The 3.8 release moves to a much larger voice system.

Generative voice design: Flash TTS creates new voices from prompts describing role, accent and voice characteristics. This works across more than 100 languages and dialects. Google’s demos include a Melbourne DJ, a monotone robot and a Japanese dragon.
Voice library: Developers get 2,000+ production-ready voices. Coverage includes regional varieties like Mexican Spanish, Quebec French and Scots English.
Save and scale: Custom voices can be saved and reused, with minimal drift across projects.
Voice remixing (coming soon): Users will adjust a library voice’s timbre, pitch, pace and accent through prompts.

Directing the Performance

Both models accept stage directions written in the script. Gemini can also steer delivery from natural script cues.

Long-form generation: Voice quality, pacing and timbre hold across hours of continuous audio.
Native 2-speaker staging: A single script drives a multi-turn conversation with distinct, separated voices.
Vocal bursts: Non-verbal cues like <laughs>, <sigh> and <gasp> add conversational texture.
Backchanneling: Active-listening interjections like |mhm| and |yeah| control reaction beats and comedic timing.

Voice Replication and Safety Controls

Voice replication builds a consistent vocal profile from a 30-second audio sample. The sample must be your voice or one you have rights to use. Replication requires a verbal consent recording from the voice owner, matched against the reference speaker.

Every clip from Gemini Audio models carries a SynthID watermark. This imperceptible mark is embedded directly in the audio output. Replicated voices also carry C2PA content credentials. Google points to the Gemini 3.8 Audio model card for its broader safety approach.

Benchmark Results

Google reports these results for the new models:

Hume AI Voice Design Benchmark: Flash TTS ranks #1 overall with a score of 71.4, per Hume AI.
Accent modeling: Flash TTS leads with a score of 60.8.
Hume AI Overall Quality Index: Flash TTS ranks #1 and Flash-Lite TTS ranks #2.
Voice Arena blind preference: Both models take top positions in Japanese, Brazilian Portuguese, Vietnamese, Modern Standard Arabic, Mexican Spanish and Hindi.

Key Takeaways

Google launched Gemini 3.8 Flash TTS for creative work and Flash-Lite TTS for scale.
Flash TTS designs new voices from prompts across 100+ languages and dialects.
Developers get 2,000+ production voices, up from 30 originals.
Voice replication needs a 30-second sample plus a matching consent recording.
Flash TTS ranks #1 on Hume AI’s Voice Design Benchmark with 71.4.

FAQ

What is Gemini 3.8 Flash TTS? It is Google’s text-to-speech model for creative voice design and line-by-line performance direction. It is available through the Gemini API and Google AI Studio.
How is Flash-Lite TTS different? Flash-Lite TTS is optimized for high-volume, cost-efficient workloads like dubbing and voice agents. It ranks #2 on Hume AI’s Overall Quality Index.
Can I clone my own voice? Yes, with a 30-second sample and a verbal consent recording. It is unavailable in AI Studio in several regions, including the UK, EEA and India.

Check out the Technical Blog. All credit goes to the researcher of this project. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter<

2026-09-23 20:20:12 · 大模型,算力芯片,AI应用,具身智能,Google,语音音频,Agent智能体,搜索RAG,扩散模型,模型评测,提示工程,招聘HR,榜单评测,开发者生态

From Steel To Physical AI, Pittsburgh Is Building Again

Forbes InnovationSilicon Valley won the software race. Pittsburgh is betting the next one is physical AI, where the customer is a refinery, a railroad or a hospital.

· 语音音频

Alibaba launches Qwen Audio 3.1 with new models and slashes AI audio prices by up to 95 percent

The Decoder

Alibaba's AI team Qwen has released Qwen-Audio-3.1, a lineup of five models for speech recognition (ASR), text-to-speech (TTS), and real-time interaction. The ASR model improves multilingual and dialect recognition and automatically cleans up filler words and repetitions. ASR-Next adds multi-speaker identification with timestamps and detects emotions, ambient sounds, and machine noise. TTS handles multilingual synthesis […]

The article Alibaba launches Qwen Audio 3.1 with new models and slashes AI audio prices by up to 95 percent appeared first on The Decoder.

2026-09-23 12:31:20 · 大模型,阿里巴巴,语音音频,招聘HR

Google's new Flash TTS models let you design AI voices from scratch using text descriptions

The Decoder

Google is introducing two new text-to-speech models, Gemini 3.8 Flash TTS and Flash-Lite TTS, which support more than 100 languages. Flash TTS can create new voices from text descriptions, and both models let users add stage directions to individual lines and generate two-voice dialogue from a single script. A voice cloning feature can build a voice profile from a 30-second sample.

The article Google's new Flash TTS models let you design AI voices from scratch using text descriptions appeared first on The Decoder.

2026-09-23 17:39:10 · 大模型,Google,语音音频,强化学习,招聘HR

The Cost of Cleverness: A Backend Engineer’s Guide to Strategic Simplicity

DEV.to

There is a phase many backend engineers go through where making code shorter, smarter, and more abstract feels like progress.

You discover a new pattern, shave 200 lines down to 15, add a generic layer that can handle six future use cases nobody has asked for, and sit back like you have just advanced the field of computer science.

For a few minutes, it feels amazing.

Then one day production breaks, the logs are useless, the issue is hiding behind three layers of indirection, and suddenly your beautiful design starts to feel like an elaborate prank.

I’ve been on both sides of this. I’ve written code that made me feel very clever. I’ve also had to debug clever code while tired, annoyed, and increasingly convinced the original author had enemies.

That is usually when the romance ends.

Because in backend engineering, cleverness has a way of aging badly. It looks impressive when everything is calm. Under pressure, though, you start to care less about elegance and more about whether a normal human being can understand what is going on before the business catches fire.

And that, more than anything, has changed how I think about software.

Cleverness always sends an invoice

I do not think cleverness is inherently bad. Some of it is necessary. Some of it is brilliant. Some of it is the reason a system survives real scale.

But none of it is free.

Every clever decision comes with a bill attached. Sometimes you pay it immediately in added complexity. Sometimes you pay months later when a new engineer joins the team, opens the codebase, and looks like they have accidentally walked into the middle of season four of a show they have never watched.

That is the part people forget.

Clever code usually solves one problem by quietly introducing three more. You gain flexibility, speed, elegance, or abstraction. In return, you give up clarity, ease of debugging, and sometimes your weekend.

There are cases where that trade makes perfect sense.

If you are working on something close to the metal, dealing with huge throughput, brutal latency requirements, or systems where ordinary abstractions start sweating, then yes, you may need more advanced techniques. Custom memory handling, lock-free structures, tight optimizations, all of that can be justified when the problem is real and the constraints are merciless.

In those situations, cleverness is not decoration. It is survival.

But most teams are not building a database engine in a cave.

Most teams are building APIs, internal services, payment flows, dashboards, reporting jobs, queues, and all the other software that keeps companies moving. In those systems, the most valuable quality is often not brilliance. It is clarity.

You want code that can be read, changed, debugged, and trusted without requiring a spiritual guide.

How teams quietly drift into trouble

The trouble usually starts with good intentions.

Someone wants the system to be flexible. Someone else wants it to be reusable. Another person wants to make sure it can scale later. Before long, a straightforward piece of business logic has turned into a generic framework with adapters, handlers, strategies, events, and enough abstraction to power a small government.

Meanwhile the product just needs to create an order, charge a card, and send an email.

This is how you end up with architecture that sounds more ambitious than the business itself.

A company selling shoes somehow ends up with a platform designed to support every conceivable product category, every possible fulfillment flow, and a future multi-tenant expansion into industries that do not exist yet. You look at the code and think, this system is prepared for everything except the actual thing it does every day.

I have seen simple workflows turned into event chains so elaborate that tracing one request feels like following family drama across five seasons. Something as innocent as “user updates profile” becomes a journey through producers, consumers, retries, side effects, notifications, projections, and eventually a teammate whispering, “I think that service handles it, but I’m not completely sure.”

At that point, the code is no longer helping the team. The team is helping the code.

The hidden cost is almost always human

A lot of technical discussions focus on machine costs. CPU. Memory. Latency. Throughput.

Those matter, obviously.

But in everyday backend systems, the real pain usually shows up somewhere else. It shows up in the human cost.

How hard is this thing to debug?

How long does it take a new engineer to make a safe change?

Can someone follow the flow without opening twelve files and questioning their career choices?

Can the on-call engineer understand enough of the system at 3 AM to stop the bleeding?

Those questions do not sound glamorous, but they have a lot more to do with the long-term health of a backend system than whether the code looks clever in a review.

A machine will run unreadable code just fine. It has no pride. No confusion. No need for sleep. Humans are much less forgiving.

That is why I have become suspicious of code that feels too smart too early. Not because smart people wrote it, but because complex systems tend to demand complex maintenance.

And maintenance is where the real game is.

Debugging is where cleverness loses its shine

A calm code review is one thing. Production pressure is another.

There is a huge difference between reading a fancy abstraction over coffee and trying to understand it while dashboards are glowing red and someone is asking for updates every seven minutes.

That is when the gap appears.

The engineer who wrote the clever code understood every shortcut, hidden assumption, and elegant little trick at the time. Everyone else now gets the joy of reverse-engineering that thought process under stress.

This is why some systems quietly develop heroes.

There is one person who understands how the whole thing works. Everyone depends on them. People tag them in incident channels with increasing urgency. Their PTO becomes a source of organizational anxiety.

That is not a sign of architectural excellence. That is just a single point of failure wearing glasses.

When a system can only be safely changed by the person who invented its internal mythology, the problem is not staffing. The problem is design.

New engineers can feel the cleverness immediately

You do not need an audit to know whether a system carries too much cleverness. Just watch a new engineer try to work in it.

In a healthy codebase, they can follow the main paths without too much ceremony. They may not understand everything on day one, but they can answer basic questions fairly quickly.

Where does this request go?
Where is the database write?
What triggers this side effect?
What happens when this fails?

In a very clever system, those questions become archaeology.

You start at a controller, end up in a dispatcher, pass through a generic pipeline, land in a dynamically resolved strategy, then find out the actual work happens in a subscriber registered somewhere in a configuration format nobody has opened since the previous re-org.

By then, nobody is learning the business. They are just learning the maze.

And that maze has a cost.

A clear system lets a new hire contribute in weeks. A deeply clever one can make even strong engineers hesitant for months because every change feels like it might awaken something ancient.

Simplicity takes more discipline than people admit

2026-09-24 04:58:09 · 大模型,算力芯片,Meta,语音音频,办公效率,扩散模型,强化学习,招聘HR,榜单评测,开发者生态

Launch Discounts Arrive for New Mac Mini and Mac Studio, Plus Massive Deal on Samsung's Portable SSD

MacRumorsAmazon has the first post-launch discounts on Apple's brand new Mac mini and Mac Studio, with up to $49 off select models. One of the best deals is on the M5 Max Mac Studio with 36GB RAM and a 512GB SSD, on sale for $2,449.99, down from $2,499.00.

Note: MacRumors is an affiliate partner with Amazon. When you click a link and make a purchase, we may receive a small payment, which helps us keep the site running.

Regarding the Mac mini, you can get the 16GB RAM/256GB SSD Mac mini for $879.99, down from $899.00, and the 16GB RAM/512GB SSD model for $1,069.99, down from $1,099.00.

$49 OFF

M5 Max Mac Studio 36GB/512GB for $2,449.99

$19 OFF

M6 Mac mini 16GB/256GB for $879.99

$29 OFF

M6 Mac mini 16GB/512GB for $1,069.99

In terms of upgrades, Apple said the Mac mini with the M6 chip delivers up to 40% faster CPU performance, up to 4× faster performance for AI tasks in particular, up to 2× faster graphics performance, and up to 2× faster storage speeds compared to the previous-generation model with the 10-core M4 chip, 32GB of unified memory, and 2TB of storage.

You can match your new Mac mini or Mac Studio with Samsung's 4TB T7 Shield Portable SSD, which is available for $464.99 today only at B&H Photo, down from $1,144.99. This is a massive $680 discount on the SSD, a notable markdown amid the storage price hikes happening this year.

$680 OFF

Samsung 4TB T7 SSD for $464.99

If you're on the hunt for more discounts, be sure to visit our Apple Deals roundup where we recap the best Apple-related bargains of the past week.

Deals Newsletter

Interested in hearing more about the best deals you can find in 2026? Sign up for our Deals Newsletter and we'll keep you updated so you don't miss the biggest deals of the season!

Related Roundup: Apple Deals

This article, "Launch Discounts Arrive for New Mac Mini and Mac Studio, Plus Massive Deal on Samsung's Portable SSD" first appeared on MacRumors.com

Discuss this article in our forums

2026-09-23 14:02:46 · AI应用,语音音频,搜索RAG,端侧AI,招聘HR,榜单评测

继续滚动加载更多…