AI text detectors may need different signals for different kinds of writing. A new preprint reports that measures of probability fluctuation become more useful as a passage gets longer, while confidence measures perform better on short sentences. In the tested settings, spectral indicators were most useful for long, continuous generation, whereas confidence indicators were more useful when text had been locally edited.
Researchers studied training-free detection by converting each passage into a sequence of token probabilities from a proxy language model. They compared confidence measures, fluctuation or spectral measures, and fusion scores combining both. The main setup used Llama2-13B to generate text and GPT-J-6B to score it; samples were ranked by detector score and evaluated with AUC for human-versus-LLM discrimination. The default settings were temperature 1.0, Top-p 1.0, Top-k 50 and 150-word continuations.
The standard English tests used XSum, WritingPrompts and Reddit ELI5. Each contributed 150 human-written examples. The first 30 tokens of each text served as the prompt for paired sets containing 150 human and 150 machine examples with matched lengths. The broader evaluation also included SemEval sentence comparisons, interleaved CoAuthor writing and MixText tasks involving AI polishing and humanizing edits.
The signal is in the fluctuation
The proposed explanation starts with the low-probability tail of the proxy model's distribution. Human text entered that tail more often than AI text in the analysis. Under the study's theoretical assumptions, those tail events were associated with greater variance in the probability sequence and more spectral energy. SpecDetect uses negative spectral energy as its machine-likeness score.
The spectral and confidence families also behaved as though they were tracking different dimensions. On the Writing dataset, principal component analysis assigned 84.2% of the variation to its first component and 8.8% to its second. The second component separated SpecDetect from mean-probability measures, supporting a distinct fluctuation dimension. The result is descriptive and does not establish causal independence.
Length was a major dividing line. SpecDetect was weak at 30 words, but fluctuation-based indicators gained more than confidence metrics as sequences lengthened, making tail events easier to observe. Generation choices mattered as well: broader sampling reduced AUC across methods, although fluctuation measures degraded more slowly. At temperatures above 1.2, the performance pattern could invert, and the analysis describes that result as conditional on the source model.
Short and edited text changes the picture
On short, pure-generation sentences, confidence metrics led. SemEval's LogLikelihood had an AUC of 0.9085, compared with 0.8411 for SpecDetect. In CoAuthor's human-versus-LLM comparison, LogRank reached 0.8138. But collaborative mixing weakened the separation: in the human-versus-collaborative comparison, LogLikelihood fell from 0.8044 to 0.6459, and LogRank from 0.8138 to 0.6514.
Editing exposed another weakness. For GPT-4 token-level polishing, Entropy reached 78.67% pairwise accuracy, LogLikelihood 65.00% and SpecDetect 52.67%, near chance. Continuous completion looked different: SpecDetect reached 81.00% for GPT-4 and 97.00% for Llama-2. Rewrites depended on the generator: LogRank reached 96.00% on Llama-2 Rewrite, while GPT-4 Rewrite was close to random for most metrics.
Humanizing operations were uneven too. SpecDetect's pairwise accuracy rose from 67.67% on token-level adaptation to 77.00% on sentence-level adaptation. More destructive humanization could disrupt detector signatures even when it reduced text naturalness. Across 12 MixText tasks, edit density alone offered little guidance: the largest metric-level mean correlation between edit density and signed score movement was 0.127. That weak relationship suggests that the kind of edit may matter more than simply how much text changed, but it does not show that a humanized passage was written by a person.
A detector should match the text
Taken together, the results point toward matching detector signals to the text regime. Spectral evidence is most useful in long, continuous generation, while confidence measures have an advantage on short passages; collaborative mixing and rewriting can change the balance. The two signal families therefore appear complementary, but exact performance depends on the benchmark and proxy model, and the editing findings are operation-specific.
The theoretical derivation assumes shared head-and-tail conditional statistics. The report gives no inferential uncertainty estimates, and the sentence-level comparisons give no confidence intervals or significance estimates. The authors say the complete codebase and configurations are available in listed repositories. The document is an arXiv preprint, version 2, dated 27 Aug 2026; its implementation section reports a single 80GB NVIDIA H800 GPU but no funding source.
Paper data and sources
Original title: Unveiling Spectral Mechanisms in Training-Free LLM Text Detection
Authors: Haitong Luo, Xuying Meng, Weiyao Zhang et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-26
DOI: Not available
Original paper · Full text