Statistical inference · Statistical learning theory / Open question
Logistic Regression at the Separability Threshold
Near separability, tiny changes in regularization or sample size can have large effects on coefficients and confidence even in this simple convex model. Efficiency payoff: Guide finite-data regularization and stopping choices near a sharp statistical transition instead of relying on unstable asymptotic extrapolations.
Seed catalog · Showing the bundled record.
Problem statement
Let and with fixed signal norm and no intercept. Define . Determine the finite-size crossover law for its norm, direction and test classification error when approaches the known separability threshold simultaneously with . Identify the scaling windows and limiting laws rather than only iterated fixed-parameter limits.
Assumptions and scope
- Labels are in {-1,1}, examples are independent and signal strength lies in a fixed compact subset of (0,infinity). Kappa_c(s) is the Gaussian-design threshold for existence of the unregularized MLE, not a fitted empirical constant.
- For lambda>0 the minimizer is unique. At lambda=0 restrict finite-parameter MLE statements to nonseparable samples; on separable samples use the normalized hard-margin limit, not a nonexistent finite minimizer.
- State how lambda_n and d/n-kappa_c(s) scale with n, how any diverging norm is normalized, and whether the conclusion concerns expectation, concentration or a distributional limit. Test classification error and logistic probability risk are different quantities and must not be interchanged.
What would settle it?
A coupled finite-size/vanishing-regularization theorem with explicit parameter ranges, error terms and matching limiting regimes, or a counterexample to a proposed universal crossover scaling.
Selected for a theoretical or computational target. A fitted scaling curve or improved benchmark score alone does not establish the stated learning-theoretic guarantee.
Status and evidence
Gaussian MLE existence thresholds, high-dimensional regularized classifier limits, finite-sample logistic rates in restricted regimes and max-margin implicit bias are established. This coupled critical-window program is an editorial target requiring a dedicated follow-up audit, not an assertion that the basic phase transition, convex optimization or implicit bias is open. This is an editorial, source-motivated research target. The preliminary review through 2026-11-20 does not certify novelty or exclude an unlocated resolution.
Preliminary source check; not an expert-verified open status or an independent audit of proposed proofs. The compilation date is not a literature-review cutoff.
Sources and further reading
- Candès and Sur — The Phase Transition for the Existence of the Maximum Likelihood Estimate in High-Dimensional Logistic Regression ↗partial theory · source checked 2026-11-20
Primary abstract checked: explicit sharp Gaussian-design MLE existence threshold at fixed aspect ratio. This is a baseline, not the proposed coupled finite-size/vanishing-ridge crossover law.
- Mignacco et al. — The Role of Regularization in Classification of High-Dimensional Noisy Gaussian Mixture ↗partial theory · source checked 2026-11-20
Scholarly abstract checked: rigorous high-dimensional regularized convex-classifier risk in a Gaussian-mixture design. Fixed-ratio/fixed-regularization limits already exist; the marginal design differs from Gaussian-covariate logistic labels.
- Kuchelmeister and van de Geer — Finite Sample Rates for Logistic Regression with Small Noise or Few Samples ↗partial theory · source checked 2026-11-20
Scholarly abstract checked: finite-sample directional and norm guarantees for Gaussian covariates with Gaussian-link labels and a norm constraint. This is a different label/model regime from the logistic critical-window benchmark.
- Soudry et al. — The Implicit Bias of Gradient Descent on Separable Data ↗partial theory · source checked 2026-11-20
Primary abstract and revision notice checked: unregularized separable linear logistic GD converges in direction to hard-margin SVM. Version seven adds a missing multiclass assumption. Basic linear max-margin implicit bias is not an open target.