Topics / Machine Learning (Statistics) / Oct 5 - 11, 2026

Weekly issue

Top 20 Machine Learning (Statistics) Papers: Oct 5 - 11, 2026

Updated · Category: Machine Learning (Statistics) (arXiv stat.ML) · Ranked by potential impact

Prefer new papers every day? It's free.Machine Learning Daily Papers (Statistics) →
  1. 1
    Benign Overfitting Under Heterogeneous Input Fusion

    Benign overfitting is extensively studied when learning from a single high-dimensional input, but its behavior under heterogeneous input fusion remains largely unexplored. We study this question for minimum-norm linear interpolation under a heterogeneous Gaussian design, comparing two statistically dependent input blocks with their fusion while holding the underlying population task fixed.

  2. 2
    Cluster-Robust Prediction-Powered Inference

    We introduce Cluster-Robust PPI++, which provides standard errors in closed form and asymptotically valid confidence intervals under arbitrary dependence within independent clusters, requiring no bootstrap or resampling.

  3. 3
  4. 4
    High-dimensional Online Calibration from Harmonic Weights

    For forecasting $d$ binary outcomes simultaneously ($Y=[0,1]^d$), we give the first algorithm that achieves $\varepsilon$-calibration in a number of rounds that is polynomial in $d$ for every fixed accuracy.

  5. 5

Get papers like these the day they come out, not a week later.

Follow this category
  1. 6
    Extreme Binary Classification: Extreme Value Theory for Extreme Constraint on False Negative

    In this paper, we introduce the Extreme Binary Classification problem, where the objective is to learn a classifier whose false negative rate $\alpha$ is constrained by $\epsilon_{N_1}=o_{N_1\to\infty}(1/N_1)$, with $N_1$ denoting the number of positive examples in the training set. To address this problem, we propose a threshold adaptation method theoretically grounded in guarantees derived from Extreme Value Theory, together with a feature selection procedure based on a permutation test applied to sample maxima.

  2. 7
    Credal Machine Learning for Risk-Averse Decision Making

    In practice, however, the effectiveness of this approach to risk aversion is undermined by the learner's uncertainty regarding the true loss distribution and, consequently, the true CVaR. To achieve reliable risk-aversion, we propose a method in which this (epistemic) uncertainty is represented in terms of credal sets, i.e., sets of probability distributions.

  3. 8
    Eigenvalues of The Hessian in Deep Learning: The Origin of Symmetry and Its Breaking

    Hessian spectra at trained models in deep learning exhibit a persistent pattern: eigenvalues organize into distinct clusters, including a large bulk near zero and a few isolated outliers. This paper shows that a natural account of these spectral phenomena emerges when the original setting is understood as a departure from a nearby, otherwise hidden, highly symmetric reference.

  4. 9
    Transformer-Based Masked Diffusion Models Provably Learn Distributions with Latent Structure

    In this work, we establish end-to-end statistical guarantees showing that transformer-based MDMs learn structured distributions with a sample complexity governed by their latent structure.

  5. 10
    Training Parallel Speculative Draft Models By Directly Minimizing Expected Decoding Rounds

    Existing training objectives typically rely on block-local surrogates that ignore this cross-round coupling, and therefore do not directly optimize the global decoding efficiency. In this work, we develop a theoretical framework for training and evaluating these drafters by representing speculative decoding as a Markov reward process.

  6. 11
    Anchor Divergence for Semantic Geometry in Contrastive Learning

    The key idea is to use an interplay between contrastive learning, exponential families, and information geometry to establish a correspondence between probability distributions over "anchors" and Bregman geometries on the representation space. We use this correspondence to define "Anchor Divergences", a method for specifying context-specific semantic geometries on fixed representations.

  7. 12
    FedRSPO+: A Heterogeneity-aware Algorithm for Decision-focused Federated Learning

    We propose FedRSPO+, a heterogeneity-aware framework for decision-focused federated learning, built on RSPO+, a regularized predict-then-optimize surrogate that smooths the decision map through projection.

  8. 13
    Softmax Attention on Gaussian Mixtures: Linear When It Can, Selective When It Must

    We show that softmax attention can represent and learn, via gradient-based methods, optimal solutions to a range of statistical tasks, including supervised classification and denoising.

  9. 14
    Bayesian Optimization on Function Spaces Via Sparse RKHS Manifolds

    In this work, we propose $L^0$ Manifold Optimization (L0MO), a simple approach to FBO which searches the subset of a Reproducing Kernel Hilbert Space (RKHS) consisting of functions with a sparse representation in the kernel functions, optimizing both the kernel locations and their coefficients.

  10. 15
    Symmetry-Aware Feature Learning: A Polynomial Separation for Multi-Index Models

    We study growing-rank multi-index models with high-dimensional Gaussian covariates in $\mathbb{R}^d$ and $r=\Theta(d^\delta)$ teacher directions forming a cyclic symmetry orbit, where $0<\delta<1/2$.

  11. 16
    Steering Diffusion Models to Rare Events with Sequential Monte Carlo

    1/p_0[E]$ to compensate for an increasing rarity. In this paper, we present Diffusion Importance Sampling of Rare Events or DireSMC, a sequential Monte Carlo scheme that guides a population of weighted samples towards the rare event, giving access not only to samples but also to a calibrated estimate of its probability.

  12. 17
    LASER: Latent Space Adjoint Matching for Support-Constrained Entropy-Regularized Offline RL

    However, naively optimizing the latent policy can easily cause the policy to collapse into a brittle mode or exploit sharp artifacts of the learned critic. In this work, we find that entropy regularization is essential in latent-space RL for addressing these challenges.

  13. 18
    Efficient Best-of-N Policy Evaluation for Inference-time Alignment

    Evaluating BoN policies from logged data is challenging under sample-only access because standard off-policy estimators require density ratios that depend on unavailable response likelihoods. In this paper, we propose a sample-only framework for evaluating and selecting BoN policies without access to these likelihoods.

  14. 19
    ENCORE: Exact Non-equilibrium COntrol with Replica Exchange for Diffusion Generation

    We propose Exact Non-equilibrium COntrol with Replica Exchange (ENCORE), the first exact parallel control method.

  15. 20
    DSReg: Provably Recovering Individual World Latents Without Reconstruction

    Methods that recover individual latent variables of the world, from nonlinear ICA to dictionary learning and causal representation learning, anchor the latents to observations through reconstruction, auxiliary supervision, or distributional asymmetries such as non-Gaussianity. Methods without these anchors, including joint-embedding predictive architectures (JEPAs), identify the latent state only up to a linear transformation, so individual latents remain mixed.