A preprint study of graph-based question answering found that carefully rewritten questions could derail a system even when the correct answer remained in the information it had retrieved. In one main test, the gold answer was present in 74.0% of the retrieved subgraphs, but exact-match accuracy was only 0.68%. The contrast separates two tasks that can look like one: having the answer somewhere in the working graph, and being able to reach it through that graph.
The study asked which stage of a GNN-based knowledge-graph question-answering pipeline fails when an input question is adversarially rewritten. Its main tests covered 3,531 CWQ test questions and 1,639 WebQSP test questions, using Freebase for denotation checks.
A test designed to preserve the answer
The researchers used two kinds of answer-preserving perturbation. Compositional Restructuring, or CR, was treated as a test of subgraph retrieval, the step that builds the smaller working graph around a question. Relation Synonym Swap, or RS, was treated as a test of graph reasoning. To isolate the stages, the researchers compared the full perturbed pipeline with a run in which the clean subgraph was held fixed, separating a subgraph gap from a decoder gap.
The attack model assumed only that an adversary could submit a rewritten question. Candidate rewrites had to keep perplexity below 50, BERTScore above 0.85, and the same Freebase denotation as the original. In ordinary terms, the rewritten question had to remain similar in meaning and preserve the same answer in the knowledge graph.
Entity linking, the step that maps words in a question to knowledge-graph entities, was not the reported bottleneck. Under both CR and RS, ELQ SeedHit changed by no more than 0.3 percentage points from the clean question. The sharpest degradation appeared later, when the system assembled and used its working subgraph.
The subgraph, not the final decoder
Under CR, ELQ exact-match accuracy fell to 0.68% on CWQ and 0.49% on WebQSP. For CWQ, the isolation analysis assigned 52.08 percentage points of the measured loss to the subgraph gap, while the decoder gap accounted for 0.14 percentage points. The result places most of the observed failure in the system’s retrieved working graph rather than in the final answer-generation stage.
The RS results supported the same stage-level distinction. When the clean subgraph was held fixed, the system reached 51.69% accuracy on CWQ and 70.59% on WebQSP. Those results indicate that the decoder could still perform substantially better when it received the original working graph, even with the rewritten question.
Other comparisons pointed to retrieval design as an important factor in the tested systems. GraftNet with oracle seeds reached 29.82% CWQ exact match under CR, compared with 14.98% for ELQ combined with cosine retrieval. In a separate reasoning setup, injecting a predicted relational path at inference time reached 51.43% CWQ Hit@1 under CR without task-specific fine-tuning, within one percentage point of the reported RoG+RA result.
The result was not universal across the controls
An architectural control produced a markedly different result. EPR-KGQA, a PPR-free single-shot system, retained 59.22% CWQ and 64.25% WebQSP Hit@1 under CR. Under RS, it retained 59.76% on CWQ and 63.09% on WebQSP. That contrast shows that the sharp collapse reported for the main pipeline was not shared by every architecture tested.
A separate MetaQA control also showed a different pattern. It covered 14,872 two-hop questions and 14,274 three-hop questions. CR cost 0.11 percentage points on the two-hop set and 6.91 percentage points on the three-hop set, rather than reproducing the CWQ result in which ELQ exact match fell to 0.68%.
The paper reports that all five primary conclusions survived Bonferroni adjustment, a correction for the risk of false positives across multiple statistical comparisons. The document is identified as an arXiv preprint, so the findings remain a report of the evaluated experiments rather than a settled verdict on graph-based question answering as a whole.
The central lesson is narrower and more concrete: finding the answer somewhere in a retrieved subgraph is not the same as making it reachable through the graph. In the CWQ CR isolation test, the measured subgraph gap was 52.08 percentage points, compared with 0.14 points for the decoder gap. For the pipeline studied here, the structure of what was retrieved mattered far more than the final step that generated the answer.
Paper data and sources
Original title: Query-Side Attacks on GNN-Based KGQA: Tracing Failures from Entity Linking to Answer Generation
Authors: Pankaj Kumar, Subhankar Mishra
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-26
DOI: Not available
Original paper · Full text