Topics / Machine Learning (Statistics) / Oct 5 - 11, 2026
Weekly issue
Top 20 Machine Learning (Statistics) Papers: Oct 5 - 11, 2026
- 1Benign Overfitting Under Heterogeneous Input Fusion
Benign overfitting is extensively studied when learning from a single high-dimensional input, but its behavior under heterogeneous input fusion remains largely unexplored. We study this question for minimum-norm linear interpolation under a heterogeneous Gaussian design, comparing two statistically dependent input blocks with their fusion while holding the underlying population task fixed.
- 2Cluster-Robust Prediction-Powered Inference
We introduce Cluster-Robust PPI++, which provides standard errors in closed form and asymptotically valid confidence intervals under arbitrary dependence within independent clusters, requiring no bootstrap or resampling.
- 3Explicit Asymptotic Bounds for Sequential Calibration Beyond $T^{2/3}$
We study the problem of sequential forecasting of binary outcomes.
- 4High-dimensional Online Calibration from Harmonic Weights
For forecasting $d$ binary outcomes simultaneously ($Y=[0,1]^d$), we give the first algorithm that achieves $\varepsilon$-calibration in a number of rounds that is polynomial in $d$ for every fixed accuracy.
- 5
Get papers like these the day they come out, not a week later.
Follow this category- 6Extreme Binary Classification: Extreme Value Theory for Extreme Constraint on False Negative
In this paper, we introduce the Extreme Binary Classification problem, where the objective is to learn a classifier whose false negative rate $\alpha$ is constrained by $\epsilon_{N_1}=o_{N_1\to\infty}(1/N_1)$, with $N_1$ denoting the number of positive examples in the training set. To address this problem, we propose a threshold adaptation method theoretically grounded in guarantees derived from Extreme Value Theory, together with a feature selection procedure based on a permutation test applied to sample maxima.
- 7Credal Machine Learning for Risk-Averse Decision Making
In practice, however, the effectiveness of this approach to risk aversion is undermined by the learner's uncertainty regarding the true loss distribution and, consequently, the true CVaR. To achieve reliable risk-aversion, we propose a method in which this (epistemic) uncertainty is represented in terms of credal sets, i.e., sets of probability distributions.
- 8Eigenvalues of The Hessian in Deep Learning: The Origin of Symmetry and Its Breaking
Hessian spectra at trained models in deep learning exhibit a persistent pattern: eigenvalues organize into distinct clusters, including a large bulk near zero and a few isolated outliers. This paper shows that a natural account of these spectral phenomena emerges when the original setting is understood as a departure from a nearby, otherwise hidden, highly symmetric reference.
- 9Transformer-Based Masked Diffusion Models Provably Learn Distributions with Latent Structure
In this work, we establish end-to-end statistical guarantees showing that transformer-based MDMs learn structured distributions with a sample complexity governed by their latent structure.
- 10Training Parallel Speculative Draft Models By Directly Minimizing Expected Decoding Rounds
Existing training objectives typically rely on block-local surrogates that ignore this cross-round coupling, and therefore do not directly optimize the global decoding efficiency. In this work, we develop a theoretical framework for training and evaluating these drafters by representing speculative decoding as a Markov reward process.
- 11Anchor Divergence for Semantic Geometry in Contrastive Learning
The key idea is to use an interplay between contrastive learning, exponential families, and information geometry to establish a correspondence between probability distributions over "anchors" and Bregman geometries on the representation space. We use this correspondence to define "Anchor Divergences", a method for specifying context-specific semantic geometries on fixed representations.
- 12FedRSPO+: A Heterogeneity-aware Algorithm for Decision-focused Federated Learning
We propose FedRSPO+, a heterogeneity-aware framework for decision-focused federated learning, built on RSPO+, a regularized predict-then-optimize surrogate that smooths the decision map through projection.
- 13Softmax Attention on Gaussian Mixtures: Linear When It Can, Selective When It Must
We show that softmax attention can represent and learn, via gradient-based methods, optimal solutions to a range of statistical tasks, including supervised classification and denoising.
- 14Bayesian Optimization on Function Spaces Via Sparse RKHS Manifolds
In this work, we propose $L^0$ Manifold Optimization (L0MO), a simple approach to FBO which searches the subset of a Reproducing Kernel Hilbert Space (RKHS) consisting of functions with a sparse representation in the kernel functions, optimizing both the kernel locations and their coefficients.
- 15Symmetry-Aware Feature Learning: A Polynomial Separation for Multi-Index Models
We study growing-rank multi-index models with high-dimensional Gaussian covariates in $\mathbb{R}^d$ and $r=\Theta(d^\delta)$ teacher directions forming a cyclic symmetry orbit, where $0<\delta<1/2$.
- 16Steering Diffusion Models to Rare Events with Sequential Monte Carlo
1/p_0[E]$ to compensate for an increasing rarity. In this paper, we present Diffusion Importance Sampling of Rare Events or DireSMC, a sequential Monte Carlo scheme that guides a population of weighted samples towards the rare event, giving access not only to samples but also to a calibrated estimate of its probability.
- 17LASER: Latent Space Adjoint Matching for Support-Constrained Entropy-Regularized Offline RL
However, naively optimizing the latent policy can easily cause the policy to collapse into a brittle mode or exploit sharp artifacts of the learned critic. In this work, we find that entropy regularization is essential in latent-space RL for addressing these challenges.
- 18Efficient Best-of-N Policy Evaluation for Inference-time Alignment
Evaluating BoN policies from logged data is challenging under sample-only access because standard off-policy estimators require density ratios that depend on unavailable response likelihoods. In this paper, we propose a sample-only framework for evaluating and selecting BoN policies without access to these likelihoods.
- 19ENCORE: Exact Non-equilibrium COntrol with Replica Exchange for Diffusion Generation
We propose Exact Non-equilibrium COntrol with Replica Exchange (ENCORE), the first exact parallel control method.
- 20DSReg: Provably Recovering Individual World Latents Without Reconstruction
Methods that recover individual latent variables of the world, from nonlinear ICA to dictionary learning and causal representation learning, anchor the latents to observations through reconstruction, auxiliary supervision, or distributional asymmetries such as non-Gaussianity. Methods without these anchors, including joint-embedding predictive architectures (JEPAs), identify the latent state only up to a linear transformation, so individual latents remain mixed.