Corruption does not always make a constrained machine-learning problem harder. In a mathematical analysis, a deliberately constructed example cuts the constrained Bayes risk, the best expected loss available within a restricted model class, to half its original value after corruption. That makes the usual data-processing inequality fail in the constrained setting. The result is theoretical: the work uses probability distributions and constructed models, not an empirical dataset or a participant sample.
The paper asks whether a data-processing inequality can be generalized to constrained learning and what determines whether corruption cannot lower constrained risk. Here, a loss function scores predictions, a model class limits the available prediction rules, and a joint distribution describes attributes and labels. The central comparison is whether one fixed loss and model class give no greater risk on the clean distribution than on the corrupted one.
To make that comparison, the paper represents experiments and corruptions with Markov kernels and defines constrained Bayes risk as the lowest expected loss available over the chosen model class. Risk is evaluated under a joint distribution of attributes and labels. This is a formal setup, not an analysis of observed cases.
The usual rule can break
The first counterexample uses a singleton model class, meaning the class contains only one permitted model. Under the constructed corruption, the corrupted-to-original constrained-risk ratio is 0.5. In plain terms, the restricted problem has half the constrained Bayes risk after corruption. That is enough to show that the classical inequality does not hold for every constrained model class.
A second construction uses logistic regression with log loss. The corrupted constrained risk is expressed as the binary entropy associated with a noise parameter and is below the clean reference value. The finding is specific to the constructed distribution and model class, not an empirical estimate.
With the unrestricted model class, the same construction moves in the opposite direction. The corrupted risk remains the binary entropy of the noise, but it is above the clean unrestricted risk of zero, consistent with the classical data-processing direction. The paired calculations tie the reversal in this example to the restriction on the model class.
A geometric test for the rule
To explain when the usual ordering holds, the paper turns to geometry. It studies superprediction sets, sets built from the losses a function class can attain, and compares their convexified versions. A theorem says that containment of one such set in another is equivalent to ordering the corresponding support functions for every probability measure. The result turns a question about risks into a question about the containment of loss sets.
For a fixed loss, model class and joint corruption kernel, the main characterization is sharper: generalized data processing holds for every joint distribution exactly when the corrupted convexified superprediction set is contained in the clean one. When that containment is present, clean constrained Bayes risk is no greater than corrupted risk across all distributions.
The same framework compares different loss-model combinations. One combination has no greater Bayes risk on every distribution exactly when its convexified superprediction set contains the other combination's set. That gives the paper a general way to order constrained learning problems rather than relying on a single example.
When corruption cannot improve risk
For label corruption, the paper gives a sufficient condition rather than a complete characterization. The loss must have the relevant symmetry under the label transformations, and the model class must be invariant to those transformations. When those conditions hold, clean constrained risk is no greater than risk after label corruption for every distribution.
For attribute corruption, model invariance alone is identified as sufficient in the stated framework. When the model class is invariant to the relevant attribute transformations, clean constrained risk is no greater than the corrupted risk for every distribution. The result does not say that invariance is necessary in every possible setting.
With the unrestricted model class, the framework recovers the classical Bayes-risk data-processing inequality for all losses and label priors in the stated attribute-corruption setting. The paper is therefore not claiming that the classical rule fails for unrestricted models. Its distinction is between the full model class and constrained classes, where geometric containment, symmetry or invariance can determine the risk ordering.
An appendix states a stronger version of the generalized inequality. It introduces an inflation coefficient that is at least one under the stated sufficient condition and uses it to lower-bound corrupted Bayes risk by that coefficient times clean risk for every distribution. The result remains a theoretical strengthening of the comparison.
A theoretical result, not a field test
Nothing here is an empirical performance test. No human, animal, participant or observational sample is used, and the counterexamples are mathematical constructions. The conclusions concern formal distributions and model classes.
The authors also describe the label- and attribute-corruption results as sufficient rather than necessary. Satisfying the listed conditions guarantees the stated ordering under the framework, but failing to satisfy them does not by itself show that corruption will lower risk.
The supplied manuscript is a preprint, identified as arXiv version 2 and dated 29 September 2026. Its mathematical message is clear: before assuming corruption must raise constrained risk, one needs to examine the model class and the geometry of its attainable losses. A general guarantee is available when the corresponding superprediction-set containment, or one of the stated sufficient conditions, is present.
Paper data and sources
Original title: Comparing Corrupted Constrained Learning Problems
Authors: Laura Iacovissi, Rabanus Derr, Robert C. Williamson
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-26
DOI: Not available
Original paper · Full text