AI 资讯 · 每日前沿动态

Who wants to crowdsource LLM scans of articles submitted to Lobste.rs?

Lobsters

Like many of us who value the human side of the web, I am looking for ways to keep slop at bay. It seems like LLM detection technology works reasonably well nowadays, with Pangram as the main provider. It is expensive, though!

Assuming pangram scans are trustworthy, and it seems they are, we could create a centralized index of pangram scans of articles. I have already set up a prototype where I publish results of my own pangram scans. I could create more users, each tied to its own API key, so others can contribute with scans as well. This is way more resource efficient than everyone running their own scans.

What do you think?

Update: since this is receiving quite some attention, let me clarify one thing. I'm not proposing to change Lobste.rs itself. I'm wondering if there are enough people who'd like to join forces to maintain an independent index of pangram scans, for our own use, not intending to impose it on anyone (e.g., maybe we could have a browser extension that automatically shows the scan results for articles when browsing Lobste.rs or the orange site).

2026-11-08 16:36:17 · 大模型,Transformer,招聘HR,开发者生态

NetBSD 11 desktop on a Raspberry Pi 4

Lobsters

Comments

· 具身智能,开源,Transformer,扩散模型,招聘HR,榜单评测

Locally trained VLA model with minimal attention based vision encoder. [p]

Reddit r/MachineLearning

Hi y’all. Thought Id share the progress on my current local VLA agent project! using agent and language somewhat loosely here. Technically it is an agent, and technically it does adhere to text prompts. However very minimally.

Video Link: https://www.reddit.com/r/learnmachinelearning/comments/1xq23zi/local_rnn_style_vla_model_project/

Video Description:

The leftmost display shows what the models sees, The yellow triangle marker on screen shows which shape is NOT the target shape on that run (counter intuitive, I know). Importantly, the model does not see it, it is only to make it easier to see if the model is performing correctly. The upper right display shows the models visual attention, in which I discuss later. And the bottom right is the per step state matrix cell values. A small 128x128 view of the whole 768x768 matrix.

Preamble:

Recently, I have been extremely fascinated by machine learning and AI, for obvious reasons. Originally, I was intrigued by LLMs, that is, until I learnt how they work (RNN gang stand up), their dis-continuity to me isn’t as elegant as the carried state in RNNs (this is my opinion, in which I don’t really have any objective reasoning for). And what better way to lean than to do?

Architecture:

So I thought why not just make the recurrent state massive? What I have here is an attention based neural network that maintains a 768x768 matrix, with a per cell LSTM like gate. Similarish to a state space model if you will.
The total model parameter count is 25,561,354.

However, 12,976,128 of those parameters are dedicated to a legacy 49k vocabulary output decoder. Originally, this started in language modelling, but it didn’t work very well. It got to a CE loss of around 4.5 on a reasonably diverse data set across an 80 token rollout average, having seen 80 tokens of context first. This was trained for about ~150k steps with a batch count of 8 across 160 total tokens per episode. So about 200m total tokens seen. Perhaps not enough for how diverse the data set was (600m total tokens across wiki articles, science papers, online text forums, etc), but the loss seemed to indicate a long lasting plateau, decreasing only around 0.05 to 0.1 nats across the last 50k steps.

Side note: around 10% of the total training data was from 4chan (don’t hate), and must have all been collected from a very political board, specially around the 2016 us elections. The model was very much not PC lmao. But atleast now I know where NOT to collect “quality” data from…

The shown model is a continuation from the a legacy language checkpoint that was transitioned into VLA orientated tasks. My thinking was that having a basic understanding of language might help if the task involved text. However I did no comparison against a fresh checkpoint at the Vision/action switch over to compare bootstrapping speed, so I guess we’ll never know. And the model no longer retains any text output ability. When queried through the legacy text input, it defaults to repeating a single token or two indefinitely.

The model processes vision through learned, state-directed attention across raw pixel RGB values at a 128x128 resolution, without a separate pretrained vision encoder. Any visual recognition and spatial understanding is learned within the same recurrent architecture that produces the action outputs. I did try doing the pretrained CNN route, however because compute is heavily restricted in my case (I got that 12gb 3060 on me), and I don’t like spending money renting gpus, I count afford to locally train the model at any resonable speed with the added separate vision encoder. The attention based vision used here added very little extra compute, and seems to work well enough. An interesting side effect of this vision encoder is that the relative, per pixel attention values can be easily displayed (the top right display from the video), which proved useful for trouble shooting training. It can accept variable resolution sizes on inference without requiring pretraining. Although, it can usually only accept larger than the trained 128x128, as any smaller looses what little detail there already is.

Training:

The target task took inspiration from the Leaning to Reinforcement Learn (2016) paper. Rather than training the model to memorise fixed object identities, the objective is to develop a “general” ability to recognise and associate unfamiliar visual patterns with arbitrary names during inference, without updating its learned parameters. Importantly, the model received ZERO proprioception, only visual and text inputs.

The model is introduced to 2 randomly generated shapes ranging from a minimum of 3 cubes to a max of 10. There is, however, a minimum separation between the 2 shape cube counts on any 3 room task, set at around 5 The other 2 room and 1 room environments have no such minimum separation value. Each of the shapes are randomly assigned a name from a pool of 5 names (Dax, Mip, Kiv, Fep and Tav). The introduction lasts 3 steps per shape, as it is directly tied to the text introduction, which is always “this is [shape name]”. After introduction, the model then spawns into a simple 3d enviroment, and given the command “Go to [shape name]” with some random variance in how the command is said. Initially, the model needed a the chronological order of the shape introduced before it was able to make the name to shape distinction, for example; “go to the first seen shape, [shape name]”.

The 3D environment in the 2 and 3 room episodes are randomly generated with a doorway/s connecting the rooms at random point. Although the possable range in which they occupy is small, and in 3 room environments, the 2 attaching rooms are always on opposing sides of the spawn room. The wall colour and texture have remained fixed throughout all of VLA training, and the model performs noticeable worse on inference if the wall texture or colour is changes without pretraining.

Even though the training enviroment logic was inspired by the L2RL paper, kinda ironically, this model is not trained through RL. Instead I had to create increasingly convoluted trainers that match the complexity of the current curriculum. The training policy is technically imitation learning, however The trainer uses supervised action targets generated from the environment's observed state. For ambiguous decisions, such as choosing between doorways, some earlier losses are retrospectively adjusted according to the path the model actually took. Each episode, the models state is split in to two different trajectories just after introduction, with each one requesting a different shape, then combining the gradients from both.

I stuck with SL mostly because of compute limitations. Though RL, which I did try at the end, could only get at most one 4 branch run at a max rollout length of ~80 steps before I ran out of VRAM. Vs the batch 4 160 rollout length I was able to get through SL, which proved to be painfully slow, especially when i hadn’t optimised the reward signal/s yet meaning it would be like a 2 or 3 day wait just to find out that the step penalty was slightly t

2026-11-08 06:31:08 · 大模型,算力芯片,AI应用,Google,Agent智能体,推理思考,搜索RAG,Transformer,扩散模型,强化学习,预训练,提示工程,端侧AI,招聘HR,网络安全,榜单评测

I merged attention and MLP processing into a single gated Transformer block. GAP [R]

Reddit r/MachineLearning

Hi everyone!

I've been experimenting with an alternative Transformer architecture called GAP (Gated Attention Projection).

The main idea is to combine attention and MLP-style processing into a single gated block, instead of using the standard separate attention and feed-forward sublayers.

I tested it against a standard Transformer using 100 million scored training tokens per model from WikiText-103.

Some results:

GAP20 used 19.91% fewer parameters than the standard Transformer.

A smaller configuration can reduce parameters by 34.31%, although that version did not fair as well as the current model against the regular transformer, so i ended up giving 14% of my perameters savings to the GAP model for extra width, leaving the model at about 20% fewer perameters, which is why i called it in testing GAP20

The standard Transformer performed better on final language-model accuracy.

Compiling GAP20 with torch.compile improved training throughput by approximately 19.9% in one CPU benchmark compared with running GAP20 without compilation.

I've included the architecture code, configurable model sizes, speed benchmark and research paper.

This is an experimental architecture, not a claim that it outperforms Transformers overall. I'd be interested in feedback on the design, mathematics and possible improvements.

GitHub:

https://github.com/rickey1990/Gated-Attention-Projection

submitted by /u/WAMFT
[link] [comments]

2026-11-08 17:43:26 · 开源,Transformer,扩散模型,模型评测,招聘HR,论文

Unlocking any car from X brand in most of the world

Hacker NewsComments

2026-11-07T10:56:40.000Z · 大模型,算力芯片,AI应用,xAI,搜索RAG,Transformer,强化学习,招聘HR,榜单评测,开发者生态

Model-Agnostic FDR Control via Group Gaussian Mirror and Permutation SHAP

arXiv cs.LGarXiv:2608.00989v2 Announce Type: replace-cross Abstract: Most FDR-controlled feature selection methods are designed for coordinate-wise hypotheses, where each feature has a single weight or importance score. This abstraction fails in sequential and grouped models, where one original feature is represented by a block of sub-features, such as lags, recurrent states, or attention-based interactions. We propose a grouped-feature FDR control framework for such settings. For grouped linear models, we construct null-symmetric block-level mirror statistics with matrix-valued perturbations. For neural sequential models, we combine Permutation SHAP derivatives as model-agnostic block-level importance scores with kernel-based dependence measure. The framework is model-agnostic across network architectures, does not require specifying the covariate distribution, and reduces to Gaussian Mirror or Neural Gaussian Mirror when the block size is one. We prove FDR control for low- and high-dimensional grouped linear models and asymptotic symmetry of smoothed Permutation SHAP derivatives under fixed fitted nonlinear models. Experiments on simulated and real-world datasets show reliable FDR control and improved power under correlated grouped-feature signals.

2026-11-09 05:00:00 · Transformer,论文

Backdoor Channels Hidden in Latent Space: Extending Cryptographic Undetectability to Modern Neural Networks

arXiv cs.LGarXiv:2605.13214v4 Announce Type: replace-cross Abstract: Recent cryptographic results establish that neural networks can be backdoored such that no efficient algorithm can distinguish them from a clean model. These guarantees, however, have been confined to stylised architectures of limited practical relevance, leaving open whether comparable undetectability extends to modern, end-to-end trained networks. We construct such an attack mechanism for deep neural network architectures, closely aligned to the cryptographic notion of undetectability, by identifying backdoor channels as learned latent directions, and show that the question of undetectability reduces to a hypothesis test between two unknown distributions over model parameters, which we conjecture to be intractable in practice. The consequence of this reframing is significant: if exploitable channels within a network's latent space are statistically indistinguishable from naturally learned directions, an attacker need not introduce foreign structure but can instead exploit the geometry the network already possesses. Demonstrating the approach on ResNet and Vision Transformer architectures trained on standard image classification datasets, the attack achieves both consistently high success rates with negligible clean accuracy degradation, and resists a comprehensive suite of post-training defences, none of which neutralise the backdoor without rendering the model unusable. Our results establish that cryptographic backdoors need not be artefacts requiring exotic architectures or artificial constructions, but identifiable as latent properties inherent to the geometry of learned representations.

2026-11-09 05:00:00 · 办公效率,Transformer,论文

Spectrum-Adaptive Generalization Bounds for Trained Deep Transformers

arXiv cs.LGarXiv:2605.07297v2 Announce Type: replace-cross Abstract: Understanding why trained Transformers generalize well is a fundamental problem in modern machine learning theory, and complexity-based generalization bounds provide a principled way to study this question. While existing norm-based bounds for Transformers remove the explicit polynomial dependence on the hidden dimension, they typically impose fixed norm constraints specified a priori and can exhibit unfavorable exponential dependence on depth. In this paper, we derive spectrum-adaptive post hoc generalization bounds for multi-layer Transformers. Under layerwise spectral norm control, the bounds are expressed in terms of layerwise Schatten quantities of the query-key, value, and feedforward weight matrices. Since the Schatten indices need not be fixed a priori and can instead be selected after training, separately for each matrix type and layer, the bounds adaptively trade off spectral complexity against the dimension- and depth-dependent factors according to the learned singular-value profiles. Empirical comparisons of BERT-adapted proxies for the leading complexity factors suggest that the proxies induced by our bounds grow more slowly with depth and hidden dimension than the corresponding norm-based proxies. Overall, our results provide a complexity-based perspective on how the spectral structure of trained Transformers is reflected in generalization analyses.

2026-11-09 05:00:00 · Transformer,论文

Sparse Network Inference under Imperfect Detection and its Application to Ecological Networks

arXiv cs.LGarXiv:2604.18820v3 Announce Type: replace-cross Abstract: Recovering latent structure from count data has received considerable attention in network inference, particularly when one seeks both cross-group interactions and within-group similarity patterns in bipartite networks, which are widely used in ecology research. Such networks are often sparse and prone to imperfect detection. Existing models mainly focus on interaction recovery, while the induced similarity graphs are much less studied. Moreover, existing models for under-counted data place sparsity, if at all, on the detection model rather than on these graphs. To address these issues, we propose a nonnegative low-rank factorization framework that jointly estimates detection probabilities and promotes sparse within- and cross-group graphs through $\ell_{1/2}$ regularization. For the resulting nonconvex, nonsmooth problem, we develop an ADMM-based algorithm with adaptive penalization, and establish asymptotic feasibility and KKT stationarity of cluster points under stated assumptions. We also give sufficient conditions for exact recovery of within- and cross-group graphs. On synthetic data with up to 20% missing entries, the method recovers the graphs and detection probabilities with competitive accuracy. Across three ecological datasets, it matches the baseline held-out predictions while estimating detection probabilities and within- and cross-group graphs.

2026-11-09 05:00:00 · Transformer,扩散模型,招聘HR,论文

Optimal In-context Adaptivity and Distributional Robustness of Transformers

arXiv cs.LGarXiv:2510.23254v5 Announce Type: replace-cross Abstract: We study in-context learning problems where a Transformer is pretrained on tasks drawn from a mixture distribution $\pi=\sum_{\alpha\in\mathcal{A}} \lambda_{\alpha} \pi_{\alpha}$, called the pretraining prior, in which each mixture component $\pi_{\alpha}$ is a distribution on tasks of a specific difficulty level indexed by $\alpha$. Our goal is to understand the performance of the pretrained Transformer when evaluated on a different test distribution $\mu$, consisting of tasks of fixed difficulty $\beta\in\mathcal{A}$, and with potential distribution shift relative to $\pi_\beta$, subject to the chi-squared divergence $\chi^2(\mu,\pi_{\beta})$ being at most $\kappa$. In particular, we consider nonparametric regression problems with random smoothness, and multi-index models with both random smoothness and random effective dimension. We prove that a large Transformer pretrained on sufficient data achieves the optimal rate of convergence corresponding to the difficulty level $\beta$, uniformly over test distributions $\mu$ in the chi-squared divergence ball. Thus, the pretrained Transformer is able to achieve faster rates of convergence on easier tasks and is robust to distribution shift at test time. Finally, we prove that even if an estimator had access to the test distribution $\mu$, the convergence rate of its expected risk over $\mu$ could not be faster than that of our pretrained Transformers, thereby providing a more appropriate optimality guarantee than minimax lower bounds.

2026-11-09 05:00:00 · 大模型,Transformer,预训练,提示工程,论文

Intention Driven Identification of In-Possession Match Phases in Association Football through Temporal Graph Learning

arXiv cs.LGarXiv:2606.09289v4 Announce Type: replace Abstract: Recognising intentions from continuous movement trajectories is an important problem in spatiotemporal data mining. This becomes difficult when behaviours change over time and their boundaries cannot be directly observed. Football provides a useful setting for studying this problem because tactical intentions emerge from the coordinated movements of multiple players during possession. Existing methods for match phase recognition often classify individual frames, use predefined spatial rules, or identify only selected tactical patterns. They therefore cannot fully describe continuous possession sequences in which tactical intentions change over time. We address this gap by treating the identification of the in-possession match phase as a hierarchical sequence segmentation task. The framework includes three tactical intentions (Invade Opponent Space, Keep Possession, and Scoring) and six phases (Build Up, Progression, Counter Attack, Maintenance, Sustained Threat, and Finishing). We propose a Temporal Graph Attention Network (T-GAN) that combines player interaction graphs, contextual features, and Transformer based temporal modelling. The framework was evaluated on seven Bundesliga matches using leave-one-match-out validation and compared with rule-based, Random Forest, GNN, and Transformer baselines at both frame and sequence levels. The results show that temporal modelling is important for coherent phase segmentation, while graph modelling provides additional value for transition phases such as Counter Attack. The framework provides an interpretable method for discovering intention based behavioural segments from continuous player tracking data and supports automated tactical analysis in football.

2026-11-09 05:00:00 · Transformer,扩散模型,强化学习,图神经网络,招聘HR,网络安全,论文

Tyche: One Step Flow for Efficient Probabilistic Weather Forecasting

arXiv cs.LGarXiv:2605.06916v2 Announce Type: replace Abstract: Probabilistic weather forecasting requires not only accurate trajectories, but calibrated distributions over plausible atmospheric futures. Recent data-driven systems have achieved remarkable deterministic skill, and diffusion-based ensemble forecasters have substantially improved sample realism and uncertainty quantification. However, their inference cost scales with forecast horizon, ensemble size, and the number of denoising steps required for each transition, making large operational ensembles expensive. To address this, we present Tyche, a one-step conditional flow model for efficient probabilistic weather forecasting. Tyche models the conditional forecast distribution with a destination-aware average-velocity flow that maps Gaussian noise directly to future weather states in a single function evaluation (1-NFE). To make this one-step transport learnable in high-dimensional geophysical fields, we derive a JVP-regularized rectification objective that enforces temporal self-consistency across source and destination flow timesteps without explicitly forming Jacobians. The transport field is parameterized by an isotropic Swin-style transformer that preserves fine-scale spatial structure while remaining scalable on global grids. To improve ensemble reliability under autoregressive forecasting, we further introduce a rollout-based finetuning stage with curriculum CRPS calibration supervision. Experiments on ERA5 at 1.5$^\circ$ and 6-hour resolution show that our Tyche, using merely a single NFE, matches or exceeds the forecast skill and calibration of state-of-the-art multi-step generative baselines and the operational ECMWF IFS ensemble.

2026-11-09 05:00:00 · AI应用,搜索RAG,Transformer,扩散模型,论文

Distinct mechanisms underlying in-context learning in transformers

arXiv cs.LGarXiv:2604.12151v2 Announce Type: replace Abstract: Modern distributed networks, notably transformers, acquire a remarkable ability (termed `in-context learning') to adapt their computation to input statistics, such that a fixed network can be applied to data from a broad range of systems. Here, we provide a complete mechanistic characterization of this behavior in transformers trained on a finite set $S$ of discrete Markov chains. The transformer displays four algorithmic phases, characterized by whether the network memorizes and generalizes, and whether it uses 1-point or 2-point statistics. We show that the four phases are implemented by multi-layer subcircuits that exemplify two qualitatively distinct mechanisms for implementing context-adaptive computations. Minimal models isolate the key features of both motifs. Memorization and generalization phases are delineated by two boundaries that depend on data diversity, $K = |S|$. The first ($K_1^\ast$) is set by a kinetic competition between subcircuits and the second ($K_2^\ast$) is set by a representational bottleneck. A symmetry-constrained theory of a transformer's training dynamics explains the sharp transition from 1-point to 2-point generalization and identifies key features of the loss landscape that allow the network to generalize. Put together, we show that transformers develop distinct subcircuits to implement in-context learning and identify conditions that favor certain mechanisms over others.

2026-11-09 05:00:00 · Transformer,扩散模型,提示工程,端侧AI,收购并购,论文

Revisiting the Generic Transformer: Deconstructing a Strong Baseline for Time Series Foundation Models

arXiv cs.LGarXiv:2602.06909v2 Announce Type: replace Abstract: The recent surge of interest in Time Series Foundation Models has rapidly advanced the field. Yet, heterogeneous training setups across studies make it difficult to determine whether performance improvements came from architectural innovations or effective data engineering. In this work, we investigate whether a standard patch Transformer architecture, paired with a straightforward training protocol, can match the zero-shot forecasting performance of state-of-the-art TSFMs. We conduct a comprehensive ablation study covering model scaling, data composition, and training techniques to examine the ingredients behind this performance. Our findings show that pretraining data composition, contiguous patch masking, and context length have a substantial effect on forecasting accuracy, and that the generic architecture itself can deliver high performance with good scalability. By strictly controlling these experimental variables, we provide comprehensive empirical results on model scaling across multiple dimensions. Furthermore, we establish that the effectiveness of this training protocol generalizes robustly across benchmarks and alternative architectures. We release our pretrained models and document the detailed training protocol and findings to establish a transparent, reproducible baseline for future research.

2026-11-09 05:00:00 · Transformer,预训练,模型评测,论文,开发者生态

What Does a Sparse Autoencoder Feature Do? A Weight-Based Account

arXiv cs.LGarXiv:2601.22447v2 Announce Type: replace Abstract: Sparse autoencoders (SAEs) have emerged as a powerful technique for decomposing language model representations into interpretable features. Current interpretation pipelines infer feature semantics from activation patterns, implicitly assuming every feature has a semantic explanation while overlooking the computational roles features inherit from the SAE training objective. We introduce a weight-based interpretation framework that requires no activation data and grounds claims causally through targeted feature-ablation interventions. Across Gemma-2 and Llama-3.1, three depth-dependent signatures show that SAE features inherit the base model's geometry: semantic features form a U-shape under tied embeddings but bifurcate when untied, attention participation peaks mid-layer, and the two roles couple oppositely for input- vs. output-oriented features. This weight-based view supplies the causal half missing from activation-based interpretability.

2026-11-09 05:00:00 · 大模型,算力芯片,Google,Meta,Transformer,强化学习,向量数据库,端侧AI,招聘HR,论文

Concordance-Optimal Continuous Music Emotion Estimation: Variance Decomposition, Dual-Branch Temporal Fusion, and Conformal Prediction Intervals

arXiv cs.LGarXiv:2611.08966v1 Announce Type: cross Abstract: Continuous music emotion recognition (MER) estimates how Arousal and Valence evolve within a song, yet the field's standard metric, pooled frame-level correlation, is dominated by between-song differences and cannot distinguish a model that tracks emotional dynamics from one that only ranks songs by their average level. We decompose annotation variance into between- and within-song components and show that mean squared error (MSE) targets the former, while windowed Lin's concordance correlation coefficient (CCC = $r \cdot C_b$) targets the latter by penalising the variance compression MSE induces. A single weight $\lambda$ controls the trade-off: on 1,764 DEAM songs, raising $\lambda$ from 0 to 1 lowers pooled Arousal $r$ from 0.843 to 0.810 while raising mean within-song $r$ from 0.204 to 0.368. Building on this objective, we present MMAS (Multimodal Music Analysis System), an open-source model that predicts Arousal and Valence every 0.5 s from raw audio. It fuses frozen MERT-v1-95M embeddings with ten symbolic features from automatic transcription through a dual-branch temporal convolutional network with cross-modal attention. Because both branches are audio-derived, one model trains jointly on DEAM and PMEmo without domain conditioning. Under song-grouped 5-fold cross-validation (2,531 songs), pooled $r$ reaches 0.801/0.685 (Arousal/Valence) on DEAM and 0.742/0.746 on PMEmo. Against a ridge baseline on identical features, within-song Arousal $r$ roughly triples (0.344 vs. 0.125; $n = 353$ songs, Wilcoxon $p = 1.7 \times 10^{-21}$, $d = 0.56$). Complete transcription failure costs at most 0.04 pooled $r$. Split-conformal intervals attain 93.7% (Arousal) and 90.0% (Valence) marginal coverage at a 95% target, with the shortfall traced to within-song dependence. MMAS is released as a pip-installable package.

2026-11-09 05:00:00 · AI应用,多模态,搜索RAG,Transformer,扩散模型,向量数据库,招聘HR,论文

Enabling Retraining-Free Accurate Inference in Analog Compute in Memory Accelerators

arXiv cs.LGarXiv:2611.08693v1 Announce Type: cross Abstract: Analog compute-in-memory (CIM) accelerators compute matrix-vector products directly inside memory arrays, eliminating weight movement and promising order-of-magnitude gains in inference efficiency. However, converter and device errors make pretrained models unusable on these arrays, and hardware-aware retraining is costly. We enable retraining-free, accurate inference of pretrained models on simple analog arrays with low-cost, shared calibration. Our key observation is that analog error depends on how each layer is represented on the array, and that this representation can be changed offline without changing the computation. We center the inputs and apply Hadamard rotations designed for analog arrays, which keep shared reference errors spread and confine input mixing to each array. Across CNNs, vision transformers and LLMs up to 14B parameters on PCM and ReRAM, the method brings vision models and larger LLMs within a few points of floating-point accuracy and matches or approaches published hardware-aware training without training any weights. On ReRAM with 6-bit ADCs, it restores ViT-B/16 from 37.4% to 82.9% and Phi-4 from 27.0% to 72.2%, both within 2.2 points of floating-point accuracy.

2026-11-09 05:00:00 · 大模型,Transformer,预训练,端侧AI,论文

Just CaloCHILL: fast hadronic shower generation in a learned latent space

arXiv cs.LGarXiv:2611.08606v1 Announce Type: cross Abstract: Calorimeter showers are the most expensive part of detector simulation, and highly granular calorimeters, in which a single hadronic shower leaves thousands of energy deposits, make them more expensive still. We present CaloCHILL (Calorimeter Hits In Learned Latents), a generative model that does not produce these deposits one by one: instead it generates a compact summary of the shower in a learned latent space and expands it to full resolution only at the end. The complete shower, including the arrival time of every hit, is built in three stages: a flow-matching model first sets the number of hits and the earliest hit time in each calorimeter layer, a mean-flow model then generates the latent summary, and the decompressor of a variational autoencoder turns it into the full point cloud, giving every hit its position, energy and time. Sparse attention along the shower axis keeps this compression affordable for showers with thousands of hits. On positive-pion showers in the calorimeter of the International Large Detector, a proposed detector for the FCC-ee, CaloCHILL reproduces the main shower observables, including their time structure. This new model is 600 times faster than its predecessor, while improving on all accuracy metrics considered.

2026-11-09 05:00:00 · Transformer,招聘HR,论文

Fifty-Two Shades of Pig: What the UCR Archive Really Asks Classifiers to Do

arXiv cs.LGarXiv:2611.09843v1 Announce Type: new Abstract: The pigs were bleeding. That was the point of the experiment: could airway pressure reveal hemorrhage before it became critical? In the UCR Time Series Classification Archive, the de facto benchmark for Time Series Classification (TSC), the same recordings ask a different question: "which of 52 pigs produced this trace?". The dataset PigAirwayPressure is not an outlier. Although the archive is the standard benchmark, whether its tasks resemble the problems TSC is meant to solve has rarely been examined. We find that several datasets are repurposed from their original task, built from toy setups, or have flawed test splits. We further question how representative the archive's ordered non-temporal sequences (e.g. spectra, shape outlines) are of real TSC, and the potential downsides from the decomposition of multivariate problems. Our findings call for more careful interpretation of archive-wide scores and greater attention to deployment-relevant evaluation. The audit is at https://github.com/automl/ucr-provenance-audit.

2026-11-09 05:00:00 · 开源,Transformer,扩散模型,模型评测,论文

A Tight Hierarchy for Chain of Thought

arXiv cs.LGarXiv:2611.09759v1 Announce Type: new Abstract: Chain-of-thought (CoT) reasoning has emerged as a powerful paradigm for extending the computational capabilities of transformer-based models. A growing body of recent work has begun to uncover the fine-grained computational power of transformers augmented with CoT, as well as their connections to classical complexity theory. In this work, we advance this line of research by establishing the first hierarchy theorem for CoT-based computation. Our main result shows that for any reasonable resource bound $r(n) \geq n$ and any $\varepsilon > 0$, \[ \mathrm{CoT}(r(n)) \subsetneq \mathrm{CoT}(r(n)^{1+\varepsilon}), \] where $\mathrm{CoT}(r(n))$ denotes the class of decision problems solvable by transformers using $O(r(n))$ CoT reasoning steps on inputs of length $n$. Our result shows that increasing the number of CoT reasoning steps provably yields strictly greater computational power, establishing CoT steps as a fundamental resource in transformer-based computation. Unlike classical hierarchy theorems, which are typically proved via diagonalization, our approach leverages known coarse-grained simulations between CoT-augmented transformers and Turing machines in both directions. Building on these connections, we adapt the classical padding technique from complexity theory to this setting to obtain a tight hierarchy. A key technical contribution of our work is the implementation of padding within the transformer-based CoT model.

2026-11-09 05:00:00 · AI应用,推理思考,搜索RAG,Transformer,端侧AI,招聘HR,论文

继续滚动加载更多…