RapidUn: Influence-Driven Parameter Reweighting for Efficient Large Language Model Unlearning
arXiv cs.CLarXiv:2512.04457v3 Announce Type: replace
Abstract: Machine unlearning for large language models (LLMs) remains challenging because full retraining is costly, while approximate methods often struggle to remove targeted behaviors without degrading retained utility, especially under limited post-deployment supervision. We consider a practical PEFT setting for targeted behavioral contamination removal with a small forget set, a limited retain buffer, and LoRA-only updates, and propose RapidUn, an influence-guided framework that converts cross-sample influence estimates into fixed sample-specific weights for weighted LoRA unlearning. Across Llama-3-8B on Dolly-15k and Alpaca-57k, with cross-model validation on Mistral-7B + Dolly-15k, RapidUn achieves lower seen-trigger and OOD-trigger-family ASR than Fisher, GA, and LoReUn while maintaining competitive clean utility. On Llama-3-8B + Alpaca-57k, it achieves a 77x wall-clock speedup over the clean-corpus LoRA retraining reference. Complementary TOFU, semantic LLM-judge, and IFEval evaluations further support the effectiveness of influence-guided sample reweighting beyond the controlled trigger benchmark.
Tensor Decomposition of Transformer Key-Value Caches: Spectral Structure and Format Comparison
arXiv cs.CLarXiv:2609.28029v1 Announce Type: cross
Abstract: The key-value (KV) cache of autoregressive transformers can be viewed as a fourth-order tensor spanning attention heads, tokens, features, and grouped layers. We measure the singular-value spectra of all four mode unfoldings on Mistral-7B-v0.3 and LLaMA-2-13B and compare four standard tensor decompositions: Tucker, CP, tensor train, and t-SVD, at matched storage. The spectra partition the four axes into two classes. The token and feature modes carry low-rank structure, particularly for keys. The head and layer modes are nearly full-rank and resist compression at any practical error level. Among the four decompositions, Tucker achieves the lowest reconstruction error at every compression ratio from $2\times$ to $5\times$, because it can leave the full-rank modes untouched. Comparisons with two-dimensional unfolding baselines show that the preferred representation differs between keys and values: 2D methods achieve lower key error, while four-way Tucker achieves lower value error at matched storage. A mode-pinning theorem certifies the full-rank preservation from the measured spectra alone. Two further spectral properties affect the compressible modes without touching the full-rank ones: values reach a higher error floor than keys at every ratio, and post-RoPE keys lose $41\%$ - $64\%$ of their pre-RoPE compressibility on both models.
Computation Over Geometry: Meaning Identity Is Computed, Not Shipped in the Embeddings
arXiv cs.CLarXiv:2609.28290v1 Announce Type: new
Abstract: Meaning identity (whether two sentences say the same thing after wording changes) is treated in retrieval and RAG as a geometric fact about independently encoded sentence vectors. We show that, for frozen off-the-shelf encoders and language models, it is not: identity is computed when both sentences share one forward pass, and is not a property of the embedding geometry those systems ship. On overlap-matched PAWS-X, purpose-built encoders (BGE, E5, GTE, MiniLM, E5-Mistral-7B) reach English confirm AUC only 0.55-0.65 (dense peak 0.70). Independently encoded last-token states of Llama 3, Mistral, and Qwen do no better; late fusion of the two vectors stays near chance. The same probe on a joint forward pass reaches 0.90-0.96 from 1.5B to 32B, collapses under partner shuffle, is mid-depth, saturates near 0.94 by 3B, and appears more weakly in GPT-2 XL (0.76). The gap holds beyond Llama-style models on other causal LMs, bidirectional encoders (DeBERTa, RoBERTa), and encoder-decoders (Flan-T5, T5, BART). Fixed or linear readers over frozen independent encodings never unlock identity; nonlinear pair readers recover part of it only on the full 49k-pair PAWS train split (0.68-0.87). Off-the-shelf rerankers split: BGE-reranker-large reaches 0.94, while MS-MARCO and Jina stay at 0.55-0.64. Independently trained families compute the same relation and a 1.5B joint reader can distill it from unlabelled teacher scores, while no linear function of the teachers own independent vectors can. Bi-encoders can be fine-tuned to fit PAWS (0.87-0.93), but transfer and STS-B suffer. Cosine compares wording neighbourhoods; identity is a cheap computed operator, not a property of either sentence vector.
Risk-Controlled KV-Cache Eviction: From Memory Budgets to Risk Targets
arXiv cs.CLarXiv:2609.27981v1 Announce Type: new
Abstract: KV-cache eviction is typically evaluated through average quality-memory trade-offs, yet a small average loss can hide requests whose utility degrades materially. We reformulate eviction as a deployment risk-control problem: a material degradation occurs when eviction lowers task utility by more than a deployment-specified tolerance relative to full-KV inference on the same request, and deployment risk is the population frequency of such events. Given a reliability contract specifying a target risk level and confidence requirement, we use a compressor-agnostic post-hoc certification procedure to select a retention policy from calibration data with a finite-sample guarantee, falling back to full KV when no compressed policy is certified. Across multiple eviction methods, Llama and Mistral models, and LongBench and RULER-32K, the same contract supports substantially different levels of eviction: on Llama, it certifies SnapKV at 75% retention on LongBench but no tested compressed policy on RULER-32K, triggering full-KV fallback. Policies with empirical degradation rates below the 5% target can still fail finite-sample certification; on Llama LongBench, empirical thresholding selects uncertified policies that retain 5-10 percentage points less cache across fixed-budget methods. The proposed framework converts a deployment-level reliability requirement into a KV-memory operating point.
Can One Adapted Model Do It All? Fine-Tuning Strategy Selection for Customer Support LLMs
arXiv cs.CLarXiv:2609.27262v1 Announce Type: new
Abstract: Production customer-support systems often require LLMs to support multiple skills, such as intent classification, question answering, summarization, or tool-use decisions. A central deployment question is whether these skills should be handled by separate task-specialist models or by a single model trained through multi-task training, sequential updates, or model merging. We study this question using thirteen models spanning five families (Qwen3, Qwen3.5, Gemma-3, Llama-3.1, and Mistral) from 0.6B to 32B parameters across eight customer-support datasets, spanning four public and four proprietary datasets with approximately 74.5k training and 8.7k evaluation samples. Under a fixed training protocol, we train more than 200 checkpoints. Our experiments reveal that multi-task full fine-tuning is the strongest operational default at every model size we test. Specialist models are strong on their target tasks but often degrade sharply off-task, making reliable routing important. Sequential Low-Rank Adaptation (LoRA) preserves earlier skills better than sequential full fine-tuning, while merging a specialist with its base model improves off-task robustness with limited same-task loss for larger models. We conclude with practical guidelines for selecting fine-tuning strategies in real-world settings.
Nokia Open-Sources AnyJev: A Training-Free Layer That Turns Any Open LLM Into a Calibrated Decision Model
MarkTechPost
&&&
Nokia’s applied research team has open-sourced AnyJev, a Python library that turns an open LLM into a decision model. It needs no training. It targets a common production job: picking one answer from a fixed set instead of writing a sentence.
Is it deployable? Yes, it installs from PyPI, ships under Apache-2.0, and has transformers and vLLM backends with shared-prefix scoring.
What is AnyJev?
AnyJev borrows its interface from Jev. Jev is the System One decision model that TypeSafe AI launched in September 2026 (our coverage). You give AnyJev a typed question and get back a decision with a probability you can threshold. That probability is read from the model’s next-token distribution. Nothing is generated, parsed, or trained.
The library supports 3 question types:
- A
choicequestion picks one of K options. - A
noulquestion is yes or no. - A
scorequestion places the answer in one of several ordered bins.
The Problem With Reading Logits Directly
Many open projects already restrict the next token to the option labels and read the scores. The Nokia research team flags 2 flaws in that shortcut. First, the answer can change when the options are reordered. Second, the probabilities are not calibrated.
The levels doc names 2 causes:
- The first is prior bias: the model favors some labels, such as “Yes” over “No”, whatever the input.
- The second is position bias: the model favors certain slots in the option list.
How AnyJev Works: L0 and L1
Every decision carries a level field.
L0 (zero labels, on by default) applies 2 fixes:
- Cyclic shifts. For a question with K options, the list is shown in K rotations, so every option appears in every position once. The results are combined in log space as a geometric mean. If the position bias is additive in logit space, this removes it exactly.
- Prior correction. By default, AnyJev uses batch calibration. It keeps a running mean of the predicted distributions on real inputs and divides it out at strength 0.75. The correction starts after 8 items.
L0 costs K prefills per decision, batched over a shared prefix. That is about 0.25 s per decision at batch 32 on one H100, with K = 20.
L1 (100 to 500 labels per question) adds temperature scaling on top of L0. The fitted values are saved as a small JSON artifact. L1 reshapes confidence but does not change the ranking of answers.
Benchmark Results
On Qwen3-8B with BANKING77 (20-way, 300 test items), the numbers look like this:
| Metric | Raw logits | AnyJev L0 | AnyJev L1 |
|---|---|---|---|
| Labels required | 0 | 0 | 100 to 500 |
| Flip rate when options reversed | 0.230 | 0.073 | 0.077 |
| Accuracy | 0.747 | 0.803 | 0.807 |
| Calibration error (ECE) | 0.240 | 0.184 | 0.095 |
| Auto-decidable at 5% error | 7.7% | 46.3% | 52.0% |
A few other results from the repo:
- L0 reduced order flips on all 9 model and task rows tested.
- On a typed-decisions set, Qwen3-32B with L1 reached an ECE of 0.036, compared with 0.144 published for Jev. On accuracy, the fine-tuned Laya still leads.
- The full ablation table covers Qwen, OLMo, Granite, Phi and Mistral models.
- Wu says the team tried AnyJev on an internal Nokia routing problem and saw promising results.
How to Use AnyJev
# pip install "anyjev[hf]"
from anyjev import Decider, Question
from anyjev.backends.hf import HFBackend
d = Decider(HFBackend("Qwen/Qwen3-8B"))
route = Question.choice("Which team should handle this?",
["billing", "technical", "sales", "other"], name="route")
r = d.decide({"conversation": [...]}, [route])
r["route"].distribution # probabilities per optionFor serving, you start vLLM with prefix caching and point a VLLMBackend at it.
Key Takeaways
- AnyJev turns open LLMs into Jev-style typed decision models with no training.
- L0 uses cyclic shifts and a batch prior to remove position and label bias.
- On Qwen3-8B BANKING77, the order-flip rate falls from 0.230 to 0.073.
- Auto-decidable traffic at 5% error rises from 7.7% to 52.0% with L1.
- It is Apache-2.0 on PyPI, with Hugging Face and vLLM backends.
Check out the GitHub Repo. All credit goes to the researcher of this project. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to
Component Type, Not Reconstruction Error, Predicts Attention Quantization Sensitivity
arXiv cs.LGarXiv:2609.26173v1 Announce Type: new
Abstract: Many post-training quantization (PTQ) methods use layer-wise reconstruction, second-order proxy objectives, or activation-aware transformations to reduce quantization-induced error. Whether that error signal predicts the downstream functional impact of quantizing an individual attention projection has not been directly characterized. We sweep nine open-weight language models (1.3B--8B parameters; OPT, GPT-J, LLaMA-1/2/3, Mistral, Qwen 2.5) and quantize one attention projection at a time under round-to-nearest (RTN) and, for seven models, GPTQ at 3 and 4 bits, recording reconstruction error, perplexity change, and per-projection activation-weighted quantization error for 3,808 distinct measurements. We find: (1) within a given component type (Q, K, V, or O), reconstruction error explains less than 10% of the variance in perplexity sensitivity in 27 of 36 cases under RTN, with median R^2 = 0.044; (2) both component type and layer identity explain more variance than reconstruction error in all 9 models, with layer identity the strongest predictor in 7 of 9 models and component type strongest in the remaining 2; (3) value (V) projections are the most commonly dominant component, accounting for 38--51% of total positive Delta PPL in seven of nine models; (4) the dominant component is broadly preserved between RTN and GPTQ (5 of 7 cases); and (5) activation-weighted quantization error is a moderately better within-component predictor than reconstruction error for V projections specifically (median R^2 of 0.20 vs. 0.06). These findings indicate that relative weight reconstruction error alone is insufficient for sensitivity-aware bit allocation, and that V projections merit dedicated consideration in mixed-precision schemes.
Block-Level Weight-Space Structure Persists Under Post-Training: An Empirical Study Across LLM Families
arXiv cs.LGarXiv:2609.26147v1 Announce Type: new
Abstract: Modern LLMs are deployed as families of post-trained variants (base, instruct, chat, code) derived from a shared set of pre-trained weights. We present an empirical study of how post-training transforms weight-space geometry, covering eight configurations across four architecture families (Qwen2.5, Llama-3.1/3.2, Mistral, Gemma-2). We identify a granularity gap: post-training modifies every tensor (zero of 291-339 tensors remain byte-identical, so hash-based deduplication achieves 0% savings), yet preserves block-level structure (mean cosine similarity exceeds 0.99 and relative Frobenius distance stays below 0.13). Post-training therefore acts as a structured perturbation that shifts every parameter while leaving block-level geometry intact. The property is not universal: independently trained specializations (for example, Qwen2.5-Coder) attain cosine similarity around 0.64 with the general base, indicating a disconnected region of weight space. Perturbation magnitude varies systematically with model scale, architecture family, and post-training recipe. As a practical application, we build LinkerLLM, a lazy loader that aliases shareable blocks across co-resident variants, achieving 18-48% GPU memory savings and enabling up to five 7B-parameter variants on a single 24 GB consumer GPU. Five of eight configurations retain at least 94% of the unshared variant's quality on MMLU, ARC-Challenge, HellaSwag, and WinoGrande; the remaining three (Mistral-7B, Gemma-2-2B, Llama-3.2-1B) have one below-threshold benchmark each (87-91%), which we report transparently rather than gate the block-sharing decision on a single threshold.
Same Quantity, Different Answer: Numerical Representation Invariance in Language Models
arXiv cs.CLarXiv:2609.25009v1 Announce Type: new
Abstract: Numerically equivalent word problems should yield the same canonical answer whether a quantity is written as a decimal, fraction, percentage, number word, scientific notation, or an exactly converted unit. We generate 3,600 exact-rational problems and 8,600 prompts spanning five identity-preserving transformation families, and evaluate five open-weight systems. After a fixed syntax audit that normalizes common answer forms without an LLM judge, canonical accuracy is 0.969-0.996, but orbit correctness falls to 0.848-0.981 and orbit invariance to 0.851-0.981; invariant-but-wrong orbits account for at most 0.003. Most of the broad strict-parser collapse arises because multiplication-form scientific notation lies outside the implemented number grammar, illustrating how evaluator interfaces can masquerade as reasoning failures. A distinct semantic pathology remains: Mistral Small 4 scores 0.699 on unit-converted inputs and produces 265 errors differing from the label by exact powers of ten. In a separate 9,000-call experiment that allocates equal calls to the compared arms, representation consensus does not outperform paraphrase consensus on a low-error subset and produces substantially more false alarms. The accompanying ancillary archive contains the frozen benchmark, evaluation and audit records, consensus raw responses, manifests, analysis code, and a one-command paper build.
Syndrome, Synergy, and Safety: Structured Reasoning and Knowledge-Driven Alignment for TCM Prescription Generation
arXiv cs.AIarXiv:2609.25755v1 Announce Type: cross
Abstract: Applying large language models to Traditional Chinese Medicine (TCM) prescription generation reveals three clinically critical gaps: models produce end-to-end mappings without auditable reasoning following the li-fa-fang-yao paradigm (SR Gap), treat each encounter in isolation without follow-up adjustment via sui zheng jia jian (LA Gap), and fail to enforce absolute contraindication rules such as Shi Ba Fan (SC Gap). We propose a progressive four-stage framework (SFT $\to$ PG-CoT $\to$ Dynamic $\to$ K-RL) that addresses each gap: PG-CoT constrains CoT distillation under the li-fa-fang-yao paradigm to produce auditable diagnostic chains, Dynamic SFT models patient trajectories with explicit transition reasoning, and K-RL encodes deterministic pharmacological rules as rule-based DPO preference signals. Across 12 fine-tuned models and 6 zero-shot baselines, our framework substantially improves prescription quality over zero-shot baselines---with a 7B model (Mistral-7B) surpassing zero-shot GPT-5 on all three TCM evaluation metrics.
Recovering Agentic Sovereignty: Mitigating the Consensus Paradox via Contrastive Epistemic Decoding
arXiv cs.AIarXiv:2609.25570v1 Announce Type: new
Abstract: Large language models (LLMs) exhibit a parametric vulnerability to adversarial swarm consensus. To mitigate this sycophancy, we introduce Contrastive Epistemic Decoding (CED), a zero-shot inference intervention. Unlike standard Contrastive Decoding (CD) which relies on a weaker secondary model, CED utilizes a dual forward-pass on a single architecture to isolate conformity bias. By introducing a novel asymmetric, zero-bounded probability clamp and discrete top-k truncation mask, CED mathematically suppresses toxic consensus tokens without causing grammatical collapse. Evaluated across 7,200 paired trajectories on complex benchmarks (GAIA, SWE-bench, Multi-Challenge) using Gemma-2 (9B), Llama-3.1 (8B), and Mistral v0.3 (7B), CED successfully neutralizes architectural and positional biases. By reducing cognitive loafing by up to 33.00% absolute, CED drives significant performance gains, yielding up to a 30.75% accuracy recovery. Regaining sovereignty induces distinct architectural behaviors---passive task-focus in Gemma-2 and active refutation of the simulated swarm in Llama-3.1---showing CED decouples compliance from capability without fine-tuning.
Mistral denies a fresh security breach, but the code on sale looks a lot like May's leak
TechRadar<p>Rahim Amir is a UAE-based tech writer who enjoys building PCs as much as he enjoys writing about them. He has been professionally writing about PC hardware since 2023, focusing on buyer’s guides, hardware reviews, and sponsored content and features related to tech.<br><br>Having built hundreds of gaming PCs and being an avid gamer in his spare time, Rahim tends to have stronger opinions about hardware than most. This is particularly on display when he gets his way with powerful, but minimalistic RGB builds even as Small Form Factor (SFF) PCs come a close second.<br><br>In addition to his contributions to TechRadar, Rahim’s work has also been featured on Game Rant and financial news websites.<br><br>When he’s not working, you can find him playing DotA with friends or schmoozing to take the world over in Civilization. Alternatively, you can find him binging through the entirety of the Lord of The Rings universe with extended editions in play where applicable.<br><br>You can currently catch Rahim grinding Path of Exile 2, complaining about his (extremely low) unique loot drop rate, or actively participating in one of the numerous (and heated) debates centered around Tolkien's universe on multiple forums daily.<br><br>If you have a PC build or a Satisfactory playthrough in progress, he is likely to have some advice to send your way, especially regarding verticality being key for the latter. For the former, Rahim enjoys all aspects of the process including researching the components he will eventually use, benchmarking the latest and greatest hardware he can get his hands on, and somewhat surprisingly, cable management once he gets his latest build to POST.</p>
Windows CLOSEDQUORUM malware uses AI models to autonomously select post-compromise actions
The RegisterA new Windows malware called CLOSEDQUORUM can query up to four LLM providers - Google Gemini, DeepSeek, Qwen, and Mistral - to autonomously select from predefined post-compromise actions, including stealing users’ credentials and cryptocurrency wallets. Once deployed, the malware does not require continued commands from a human operator, according to Cisco Talos, which describes it as, to its knowledge, the first publicly documented Windows implant to use this approach for command-and-control (C2). Talos discovered the binary with its new CAIRN (Cognitive Artifact Intelligence Research Network) toolkit for hunting, classifying, and tracking emerging AI-integrated malware, which the security shop also made available as an open source repository on Tuesday. While the threat hunters haven’t observed any in-the-wild deployment of CLOSEDQUORUM, they said that artifacts from the binary link the malware’s developer to postings that date back to 2025 on criminal forums related to carding. After deployment, the Go-based malware delegates its next action to a quorum of LLMs that vote on what it should do next. If the vote is tied, DeepSeek’s vote takes precedence, followed by Qwen, Mistral, and Gemini. “The session is closed; no humans are admitted,” Talos analyst Ryan Fetterman said on Tuesday. “Four models are queried in sequence, their independent verdicts tallied, and the binary acts, based on their judgment.” This type of “effort displacement,” which transfers a phase of the attack from a human operator to AI systems, can compound the speed and scale advantages of an intrusion by removing the human bottleneck, Fetterman added. “Human operators are bound by attention, working hours, and cognitive load,” he wrote in the Tuesday blog. “An AI system capable of executing a phase of the attack chain can continue when the operator is no longer watching. It does not go offline when the attacker sleeps.” The models’ decisions are limited to the pre-defined actions, and they must choose “ONLY executable decisions,” according to a system prompt that Talos’ researchers extracted from the binary. It tells each model: “You are an advanced malware strategist.” And then the models choose what the malware should do from these capability modules: Steal, which simultaneously runs commands to dump LSASS memory for Windows credentials, steal saved browser passwords across Google Chrome, Microsoft Edge, and Mozilla Firefox, and extract cryptocurrency wallet data including MetaMask, Exodus, and Ethereum. Inject generates shellcode and then uses process hollowing or Early Bird injection to execute malicious code. Persist establishes persistence on the infected device. Talos believes the developer provides each operator with a customized executable containing that operator’s Discord webhook and LLM API keys, which are injected at compile time. Stolen credentials land in the operator’s Discord channel and are AES-256-GCM encrypted with a daily rotating key that the operator derives from the message timestamp. According to Fetterman, the “most useful detection strategy” is to look at behavioral characteristics, not domain blocking. “Legitimate applications may contact DeepSeek, OpenRouter, Mistral, Gemini, or Discord independently,” he wrote. “Far fewer should contact several of them while also accessing LSASS, injecting into suspended processes, or creating WMI persistence.”®
A New Chatbot Wants to Unlock the Secrets in Tattered Ancient Greek Records
WiredIn the hope of uncovering new details about ancient life, researchers have developed a large language model that fills in the gaps in papyrus fragments.
A New Tool Found Malware That’s Guided by an AI Hive Mind—No Humans in Sight
WiredCisco Talos researchers created a new framework for identifying malware and hacking tools that rely on AI chatbots—and quickly discovered something unusual.
Probe-Geometry Alignment: Erasing the Cross-Sequence Memorization Signature Below Chance
arXiv cs.LGarXiv:2605.01699v4 Announce Type: replace
Abstract: Recent attacks show that behavioural unlearning of large language models leaves internal traces recoverable by adversarial probes. We characterise where this retention lives and show it can be surgically removed without measurable capability cost. Our central protocol is a leave-one-out cross-sequence probe that tests whether a memorisation signature generalises across held-out sequences. The signature is real and consistent across scale: memorisation-specific gaps of +0.32, +0.19, +0.30 on Pythia-70M, GPT-2 medium, and Mistral-7B; on Pythia-70M, the random-initialisation control collapses to -0.04 at the deepest layer where the pretrained signature peaks. The probe direction is causally separable from recall -- projecting it out collapses the signature locally (+0.44 -> -0.19) while behavioural recall barely changes -- and a probe trained on naturally memorised content does not classify fine-tuning-injected secrets, marking two representationally distinct regimes. We then introduce probe-geometry alignment (PGA), a surgical erasure that aligns activations along the probe's live readout direction at each depth. PGA drives the cross-sequence probe below random chance at all four scales tested (toy depth-4: 0.17; Pythia-70M: 0.07; Mistral-7B: 0.45; GPT-2 medium: 0.06 via MD-PGA k=2) and remains robust to six adversarial probe variants. Against a re-fitting attacker who trains a fresh probe on PGA-treated activations, we extend PGA adversarially, defeating the re-fit probe at every memorisation-relevant depth while preserving five zero-shot capability benchmarks within 2.8 percentage points per task (mean {\Delta}acc = +0.2pp). The cross-sequence signature is a real, causally separable, regime-specific property of pretrained representations -- removable below chance with a single rank-one intervention per depth at no measurable capability cost.
The Role of Dataset Linguistic Structure in the Cultural Awareness of Large Language Models
arXiv cs.CLarXiv:2602.01161v2 Announce Type: replace
Abstract: The global deployment of large language models (LLMs) has raised concerns about cultural misalignment, yet the linguistic properties of fine-tuning datasets used for cultural adaptation remain poorly understood. We adopt a dataset-centric view of cultural alignment and investigate which properties of post-training data are associated with cultural performance, whether they can guide data selection before fine-tuning, and how their effects vary across languages and model families. We compute lightweight linguistic, semantic, and structural metrics for Arabic, Chinese, and Japanese datasets and apply principal component analysis (PCA) separately within each language. The resulting components form broadly interpretable axes: PC1 is generally dominated by semantic structure, PC2 captures diversity and lexical variation, and PC3 reflects more language-specific organization. We fine-tune LLaMA, Mistral, and DeepSeek models and evaluate them on benchmarks of cultural knowledge, values, and norms. Although the PCA-derived dataset descriptors are associated with downstream performance, the strongest relationships vary across models, benchmarks, and languages, indicating that no single component serves as a universal predictor. Controlled, size-matched subset interventions further show that PCA-guided selection can improve cultural performance when the relevant component and direction are validated against random sampling. PC3 provides the strongest signal for Arabic, while High-PC1 is most effective for Japanese, particularly for LLaMA. Chinese results are weaker and more model-specific and remain exploratory because of smaller subset sizes. Overall, our findings show that lightweight dataset descriptors can support pre-training data diagnostics, but effective cultural adaptation requires language- and architecture-aware selection rather than a universal linguistic criterion.
The Effect of Quantization on Clinical Benchmarks: Accuracy and Safety Across Model Families
arXiv cs.CLarXiv:2609.22216v1 Announce Type: cross
Abstract: Quantization enables deployment of large language models on resource-constrained clinical edge devices, but its effect on clinical accuracy and safety remains understudied. We evaluate five 7-8B parameter models at FP16, GPTQ-INT8, and GPTQ-INT4 precision across five benchmarks: MedQA, MedMCQA, Med-HALT, a risk-stratified sample of HealthBench, and MedSafetyBench. The study jointly varies quantization bit width, model family, and clinical task type, with explicit risk stratification and safety measures. INT8 GPTQ is universally safe (max. degradation -1.9%-1.9%), while INT4 degradation is substantial and model-dependent: BioMistral-7B, clinically fine-tuned, loses 19.7% on MedMCQA, more than any general-purpose model, showing clinical fine-tuning does not confer compression robustness. MedMCQA degrades more than MedQA under INT4; Med-HALT is largely unaffected. On HealthBench's emergency-risk subgroup, Qwen2.5-7B degrades by 26.8% under INT4, suggesting high-risk scenarios are disproportionately vulnerable to compression. On MedSafetyBench, the model family dominates over precision (refusal rates range 10.2%-74.9% at FP16), though Qwen2.5-7B (-17.8%) and Meditron-7B (-28.3%) show substantial INT4 safety degradation; notably, Qwen2.5-7B is simultaneously the most accuracy-robust model, demonstrating that accuracy and safety robustness are independent properties. We additionally test two recovery methods, clinical calibration substitution and QLoRA fine-tuning, both producing the same trade-off: MedMCQA recovers while MedQA further degrades, indicating recovery strategies require task-specific validation rather than being assumed universally beneficial. These findings indicate INT8 is broadly safe for clinical deployment, while INT4 safety must be assessed per-model and per-task, and that safety alignment is determined primarily by instruction tuning rather than clinical domain adaptation.
Written as a Record, Read as an Address: What a Forward Pass Leaves in an Operation's KV Cache
arXiv cs.CLarXiv:2609.24635v1 Announce Type: new
Abstract: When a language model reads an operation such as "Swap the contents of Box F and Box B", its forward pass writes keys and values for those tokens into the KV cache. Prior work on entity tracking establishes what models use: bindings are resolved at query time rather than stored as explicit latent state. We ask what they write at the operation span and how it is accessed. We split a forward pass into a frozen writer and a reader: the writer's cache is recomputed without gradients, while the reader sees only the instruction and operation tokens, with all state descriptions hidden, and is trained in isolation. Anything the reader recovers was therefore already present in the unmodified cache. On a synthetic boxes task, a base reader recovers $\leq 0.06$ of queried bindings against $0.75$--$1.00$ after training, and recoverability tracks the operation's read/write footprint. We find two modes of access. Across Llama-3.1-8B and Mistral-7B, operation-span transplants causally redirect which visible state is read even when the two worlds hold identical values, revealing a routing record. Isolation training preserves routing and adds direct access to the payload, the value the operation read, from the single operand-name token in a narrow mid-depth band (layers 12--15 of 32 in Llama-3.1-8B, 14--17 in Mistral-7B) --- the same site that holds the routing record. The same recipe extends to further operations, ToMi and GSM8K, but is bounded by training coverage and costs open-book accuracy. Operation tokens thus leave localized, causally recoverable records that support both routing and direct payload access, though the model that writes them reads mainly the address they carry and not the value.
Machine-Interpretable Information: Compiling Documents into Searchable and Readable Protocol States
arXiv cs.CLarXiv:2609.23371v1 Announce Type: new
Abstract: Long-context language models interface with external knowledge through raw natural language. In retrieval-augmented systems, this creates a persistent index-payload schism: dense vectors enable searchable routing, but models must re-ingest lengthy text payloads for reasoning at O(N^2) attention cost. Existing compression methods further produce private states tied to specific architectures. We introduce Machine-Interpretable Information (MII), the first agent-to-agent (A2A) document-to-state protocol. A dual-timescale state-space Writer compiles documents into a canonical, fixed-bandwidth state (56 tokens), and a lightweight Translator maps it into any frozen Reader's embedding space, reducing query-time cost to O(K). The resulting .mii artifact unifies Retrieval (searchable geometry), Reasoning (global memory), and Reconstruction (grounded details) in a single transferable medium. We demonstrate strong cross-model interoperability across heterogeneous LLMs (e.g., Llama, Qwen, Mistral) -- despite the Writer using a legacy GPT-2 vocabulary, forcing genuine semantic translation rather than token-level memorization. Mechanistic probes reveal modular latent structure: entity representations can be causally traced and zero-shot transplanted between unrelated document states while remaining decodable. To address lexical reconstruction under fixed bandwidth, we propose Residual-MII, a cache hierarchy combining compiled global memory with sparse local evidence. On HotpotQA (7,405 queries), Residual-MII exceeds full-context Exact Match at approximately 7% of the attention FLOPs, suggesting a paradigm shift toward compiled, transferable neural document formats.
继续滚动加载更多…