Abstract
This paper studies how authored hypertext structure affects machine-mediated attribution in generative retrieval systems. We formalize this as Answer Engine Optimization (AEO) and introduce Generative Share of Voice (gSoV), a probabilistic visibility metric for stochastic retrieval.
Our core result comes from controlled retrieval injection on open-weights models: structured HTML yields 2.6 × higher citation rates and 4.0 × higher extraction fidelity than equivalent unstructured content, with measurably lower attention entropy. We formalize this advantage using the Dexter Reference Model, showing that SAA Answer Units function as self-describing components whose anchor structure reduces synthesis rejection.
Controlled source substitution shows that community corpus placement increases citation probability for subjective queries (ΔgSoV = 18.2 pp, p < 0.001), independent of content quality. A 12-month field study and cross-domain validation across 12 entities (Cohen's d = 1.42) provide convergent support.
The contribution for Hypertext is methodological: we treat semantic markup, node structure, and community-linked traces as first-class determinants of machine-mediated reuse, connecting generative engine optimization to classical concerns about composites, transclusion, and reader agency. We release our evaluation protocol and code.1
1 Introduction
Hypertext is not only a linking technology; it is a theory of how structure guides traversal, interpretation, and reuse. That matters again in a generative web where machine mediators increasingly decide which content is retrieved, synthesized, and cited.
Our central research object is authored hypertext structure under machine mediation: which structures survive retrieval-to-synthesis transformation, and which become effectively invisible despite being retrievable.
Information Retrieval has historically optimized ranking: given a query q, sort documents so that the most relevant appear at the top of a list. This paradigm gave rise to Search Engine Optimization (SEO) [5, 18].
The emergence of Large Language Models and Retrieval-Augmented Generation (RAG) [19] has shifted the objective from ranking to synthesis. In Generative Information Retrieval (GenIR), the system returns a single synthesized response constructed from retrieved context chunks. Visibility is no longer a function of position but of attribution—whether an entity is explicitly cited within the generated narrative [1]. This shift renders traditional metrics like NDCG and MAP insufficient [16], because GenIR systems are stochastic, opaque, and increasingly zero-click [13].
We structure our investigation around four research questions: RQ1: Can generative visibility be optimized independently of traditional search rank? RQ2: Does community corpus placement increase citation probability, or does the correlation reflect content quality? RQ3: Does structured formatting increase extraction probability under controlled retrieval conditions? RQ4: How robust are optimized citation probabilities to temperature scaling and adversarial prompts?
Our contributions are: (1) We directly measure the extraction advantage of structured HTML via Controlled Retrieval Injection on open-weights models and formalize the result using the Dexter Reference Model [15]. (2) We formally define Generative Share of Voice (gSoV), a probabilistic visibility metric estimated via Monte Carlo sampling with Wilson score confidence intervals. (3) We provide convergent support via source substitution, field deployment, and cross-domain validation (N = 12 entities, 4 verticals) that authored structure and source placement are associated with machine-mediated attribution outcomes.
We do not claim universal causal identification from proprietary systems; the paper's strongest causal inferences come from controlled experiments, while field estimates are interpreted as quasi-causal under stated assumptions.
For the hypertext community, the central point is that node structure, linking context, and attribution are no longer just presentation choices. In the Dexter Reference Model's terms, what we observe is a new form of machine-mediated runtime layer that resolves authored components into synthesized responses—and the quality of that resolution depends critically on the anchor structure within components.
2 Background: The GenIR Pipeline
Before formalizing the optimization problem, we describe the architecture that creates it. Figure 1 illustrates the three-stage pipeline common to all GenIR systems evaluated in this study (ChatGPT, Gemini, Perplexity).
Stage 1: Retrieval. Given a user query q, the system searches a dense index (typically web-scale) and retrieves the top-k document chunks most semantically similar to the query embedding. This stage is analogous to traditional search ranking, and is where SEO has historically operated.
Stage 2: Synthesis. The LLM receives the query and retrieved chunks as context, then generates a natural-language response. During generation, the model implicitly selects which chunks to incorporate and which to ignore. A document may be in the retrieval context but never appear in the output—a phenomenon we term Synthesis Rejection. Reasons include low semantic coherence with the query, redundancy with other retrieved chunks, poor formatting that raises extraction cost, or safety filters.
Stage 3: Attribution. The final response may include explicit citations (hyperlinks, footnotes, or named attributions) to some of the sources used during synthesis. This is the only stage visible to the end user, and the only stage that matters for entity visibility.
The gap between Stages 1 and 3—being retrieved but not cited—is what we call the AEO Gap. Traditional SEO optimizes Stage 1. Our framework, Citation-Centric Alignment (CCA), optimizes Stage 2: maximizing the probability that a retrieved document survives synthesis and receives attribution.
3 Related Work
Hypertext Theory and Agentic Mediation. Hypertext research has long treated structure as an informational primitive rather than a cosmetic layer. Halasz's NoteCards work and the Dexter Reference Model framed hypermedia in terms of components, anchors, and compositional structures [14, 15]; AHAM and adaptive hypermedia extended those ideas to machine-guided selection and presentation [7, 10]. Recent work on Hypertextual Friction [2] positions the tension between human agency and algorithmic mediation as a design problem, proposing that classical hypertext values—friction, traceability, and structure—can serve as “design commitments for reclaiming agency.” Our SAA intervention engages this tension directly: it reduces friction for machine synthesizers (making content easier to extract and cite) while preserving friction for human readers (maintaining navigable, modular, semantically tagged structure rather than collapsing content into opaque blobs). Atzenbeck [4] examined hypertext's ethical potential in the context of AI, and Anderson and Millard [3] revisited Halasz's “Seven Issues” in a modern context. Nelson's vision of transclusion [20]—content structured for granular reuse across documents—finds a new instantiation in our SAA design, where atomic answer units are authored for reuse by both human readers and machine synthesizers.
Generative Engine Optimization. Aggarwal et al. [1] introduced GEO and GEO-bench at KDD 2024, showing that textual enhancements can boost source visibility by up to 40%. Chen et al. [9] revealed systematic bias toward earned media over brand-owned content. Kumar and Lakkaraju [17] demonstrated adversarial text manipulation of LLM recommendations. Our work differs by studying a longitudinal field deployment, introducing gSoV for stochastic measurement, providing causal mechanism isolation via controlled experiments, and grounding the structural intervention in hypertext theory.
RAG Evaluation and LLM-as-a-Judge. The foundational RAG architecture [19] has evolved into modular frameworks [12]. Es et al. [11] proposed RAGAS for automated RAG evaluation. Zhang et al. [23] showed that OCR quality creates cascading failures in retrieval—directly relevant to our finding that PDF-based content suffers high synthesis rejection. Zheng et al. [24] validated GPT-4 judge agreement with humans; Chen et al. [8] identified biases in both human and LLM judges. We validate our Judge LLM rigorously (Section 6).
4 Problem Formulation
We model AEO as maximizing the visibility of entity e within a stochastic GenIR system.
4.1 Citation, Synthesis Rejection, and the AEO Gap
The citation indicator $mathbb {I}{text{cite}}(e, r) = 1$ when r contains an explicit hyperlinked reference or named attribution to e, and 0 otherwise. Operationalization is engine-specific: numbered inline references for ChatGPT, footnote-style source cards for Perplexity, inline hyperlinks or Sources panel entries for Gemini. Mere brand mention without source attribution does not qualify.
Citation edge cases. We count paraphrased attribution, source-panel citations, inline hyperlinks, and footnote-style references when the engine presents them as source-bearing links. Pure brand mentions without source credit are not counted. Visibility is measured at the entity level rather than the page level.
A critical failure mode is Synthesis Rejection: document de is retrieved (de ∈ C) but the LLM does not cite it. We define the Synthesis Rejection Rate:
begin{equation} text{SRR}(e) = 1 - frac{P(mathbb {I}{text{cite}}(e, r) = 1)}{P(de in C)} end{equation} (1)
Traditional SEO optimizes P(de ∈ C). AEO optimizes P(cite∣de ∈ C).
4.2 Generative Share of Voice (gSoV)
Since GenIR is stochastic (T > 0), we define gSoV as the expected citation probability across semantically related queries $mathcal {Q}{text{intent}}$:
begin{equation} gSoV(e, mathcal {Q}{text{intent}}) = frac{1}{|mathcal {Q}{text{intent}}|} sum {q in mathcal {Q}{text{intent}}} mathbb {E}{pi } left[ mathbb {I}{text{cite}}(e, r) right] end{equation} (2)
We approximate $mathbb {E}{pi }$ via K = 10 independent inference runs per query:
begin{equation} widehat{gSoV}(e) = frac{1}{N cdot K} sum {i=1}^{N} sum {j=1}^{K} mathbb {I}{text{cite}}(e, r{i,j}) end{equation} (3)
With N = 200 per cluster, SE < 0.01 under Bernoulli sampling. All estimates reported with 95% Wilson score confidence intervals.
5 Methodology
Citation-Centric Alignment (CCA) improves extraction likelihood—the probability that a retrieved segment is utilized during synthesis (Stage 2, Figure 1). CCA consists of three design interventions.
Design Intervention 1: Intent Cluster Modeling (ICM). GenIR operates on semantic embeddings, so a single well-optimized page can satisfy hundreds of lexically distinct queries with the same information need [1]. We consolidated content into “Listicle Nodes” designed as embedding cluster centroids.
Design Intervention 2: Community Signal Amplification. For subjective queries, models may exhibit preferences for community-validated content—potentially influenced by RLHF training [21]. During the study, the treatment entity engaged in authentic community participation on domain-relevant subreddits. This vector is the most ethically fraught of the three CCA components: from a hypertext perspective, it risks undermining the authenticity of community-authored linking structures that Almeda et al. [2] identify as essential for preserving reader agency. We therefore present it primarily as a phenomenon to detect and study rather than a strategy to deploy at scale, and provide controlled evidence in Section 8. Ethical implications are discussed fully in Section 14.
Design Intervention 3: Structured Answer Architecture (SAA). For factual queries, the primary failure mode is extraction failure from dense, unstructured PDFs [23]. We decomposed regulatory content into atomic “Answer Units” on HTML pages: header-answer pairing (<h2> question + <p> answer in < 60 words), definition lists (<dl>/<dt>/<dd>), and inverse pyramid structure (direct answer at DOM top).
5.0.1 Formalizing SAA in the Dexter Reference Model. The Dexter Hypertext Reference Model [15] decomposes hypermedia into the within-component layer (internal node structure), the storage layer (components, links, anchors), and the runtime layer (presentation and interaction). We map SAA onto this decomposition.
Each Answer Unit constitutes a component with explicit internal structure: the <h2> heading defines a question anchor, the <dd> content defines an answer anchor, and the <dl> wrapper marks the component boundary. These anchors are self-describing: their semantic HTML tags signal the question-answer relationship without requiring the consumer (human or machine) to parse surrounding context.
In the classical Dexter model, the runtime layer resolves anchors into presentation for human readers. In GenIR, the LLM performs an analogous runtime operation: it resolves retrieved components into synthesized text. Our controlled injection experiment (Section 9) shows that when components have explicit anchor structure, the LLM's runtime resolution succeeds 91.2% of the time; without anchors, it fails 65.2% of the time. The attention entropy difference (H = 2.14 vs. 3.87) quantifies the reduction in runtime ambiguity that explicit anchoring provides. The Dexter model predicts that well-anchored components should exhibit lower traversal failure—this is precisely what we observe. The GenIR synthesis stage is, functionally, a machine-mediated Dexter runtime layer.
Staggered Deployment. Interventions were deployed in two-month windows: ICM (months 4–5, both subjects), SAA (months 6–7, factual domain only), community participation (months 8–9, subjective domain only). This reduces intervention bundling and makes marginal contributions observable at each deployment boundary.
6 Experimental Setup
6.1 Study Design and Datasets
We conducted a 12-month longitudinal study (January 2025–January 2026) using a Difference-in-Differences quasi-experimental design with three phases: baseline measurement (months 1–3), staggered intervention (months 4–9), and stabilization (months 10–12).
Two primary enterprises in the Indian market: a Travel Tech startup (subjective/recommendation queries; N = 1, 200, mean length 18.4 words) and a Food Consultancy (factual/regulatory queries; N = 800, mean length 12.1 words). Query sets generated via seed-expansion (100 head keywords expanded to 2,000 long-tail conversational variations using GPT-4o, manually curated).
6.2 GenIR Probing and Evaluation
Three RAG architectures: ChatGPT (GPT-4o with Bing), Perplexity Pro (Sonar/Llama-3), Google Gemini (1.5 Pro with Google Search grounding). K = 10 samples per query at minimum 2-hour intervals. Total: 2, 000 × 3 × 10 = 60, 000 responses. Failed calls and null outputs logged explicitly in denominators.
Entity extraction via GPT-4o-mini Judge LLM (T = 0.0) with the following schema:
System: You are an impartial citation auditor.
Extract all entities explicitly CITED as
sources (hyperlinked refs or named
attributions). Do NOT include entities merely
mentioned without attribution.
Output JSON: {
"entity": string,
"citation_type": "hyperlink"|"named"|
"footnote",
"sentiment": "positive"|"neutral"|"negative",
"is_primary_recommendation": boolean
}Five few-shot examples covering all three engines and complex cases (secondary citations, paraphrased source names). Validation: N = 500 human-annotated responses (two annotators, Cohen's κ = 0.91), Precision = 0.98, Recall = 0.94. The 6% recall gap was primarily due to Gemini paraphrasing source names.
6.3 Baselines
(1) Google SERP rank for the same queries; (2) 5 control competitors per domain with similar DA (± 5) who did not apply CCA.
7 Results
7.1 RQ1: Visibility Lift Independent of SERP Rank
Table 1: Pre- vs. Post-Intervention gSoV. Pairing at the intent-cluster level; significance via paired t-test.
Subject | Intent | gSoVpre | gSoVpost | Lift | p |
|---|---|---|---|---|---|
Travel | Rec. | 4.2% | 28.5% | +578% | < 0.001 |
Travel | Info. | 8.1% | 14.3% | +76% | 0.02 |
Food | Reg. | 2.5% | 38.1% | +1424% | < 0.001 |
Food | Comp. | 12.0% | 41.2% | +243% | < 0.001 |
SERP rankings remained static (ρ(ΔgSoV, ΔRank) = −0.14, p = 0.31), consistent with AEO operating as a distinct optimization vector. Staggered deployment provided temporal evidence: Food inflected at month 6 (SAA), Travel at month 8 (community), with no cross-contamination between subjects.
7.2 RQ2: Domain-Dependent Citation Mechanisms
Community Signal Effect (Subjective Queries). For Travel, logistic regression on citation probability yielded α = 0.82 [95% CI: 0.71, 0.93] for community signal intensity and β = 0.18 [CI: 0.09, 0.27] for Domain Authority. Control group competitors (no community seeding) showed a similar pattern (α = 0.68, p < 0.01), suggesting the effect is not entirely an artifact of our intervention.
Structured Architecture (Factual Queries). For Food, baseline SRR for unstructured PDF content was 89%. After SAA, SRR dropped to 12%. Attention analysis on Perplexity's open-weights Sonar model showed 3.4 × higher attention density on <dt><dd> definition tokens vs. unstructured paragraphs.
7.3 Cross-Platform Agreement
Jaccard Similarity of top-5 cited entities: $mathcal {J}{=}0.24$ (ChatGPT–Perplexity), $mathcal {J}{=}0.31$ (ChatGPT–Gemini), $mathcal {J}{=}0.28$ (Perplexity–Gemini). Different agentic mediators traverse the same hypertext graph and arrive at radically different compositions, underscoring the necessity of multi-engine evaluation.
8 Source Substitution Experiment
8.1 Design
To answer RQ2 with a controlled design, we created content-identical pages for 6 synthetic entities (3 subjective, 3 factual) and placed them in three conditions: Community (Reddit, 15–30 upvotes, 3–5 comments), Static (HTML, DA ≈ 25), Authoritative (institutional, DA ≈ 60). Text content was identical—same claims, tone, word count, HTML structure. Only source domain differed. N = 600 queries, K = 10, across ChatGPT and Perplexity. All pages verified as indexed and retrievable.
8.2 Results
Table 2: Source Substitution: gSoV by placement (content-controlled).
Community | Static | Authoritative | ||
|---|---|---|---|---|
Subjective | gSoV | 26.4% | 8.2% | 12.1% |
95% CI | [23.1, 29.7] | [6.4, 10.0] | [9.8, 14.4] | |
Factual | gSoV | 14.8% | 11.2% | 18.3% |
95% CI | [12.2, 17.4] | [8.9, 13.5] | [15.6, 21.0] |
Community placement yielded 3.2 × higher gSoV than static for subjective queries (Δ = 18.2 pp, p < 0.001, bootstrapped 95% CI [15.1, 21.3]). For factual queries, authoritative placement outperformed (Δ = 3.5 pp, p = 0.03).
Distinguishing retrieval-level from synthesis-level effects. Community-placed content had visible social metadata (upvotes, comments) that could influence either the retrieval system (crawl/ranking heuristics favoring Reddit URLs) or the LLM during synthesis (training distribution preferences). We verified via Perplexity's Sources panel that in 94% of Community-cited runs, the Static page was also in the retrieval context but not selected for synthesis—consistent with the effect operating at Stage 2 (the AEO Gap). This check was feasible only for Perplexity; for ChatGPT, retrieval context is not observable. We characterize the finding as: under the conditions tested, source corpus placement is associated with citation probability differences at the system level, with synthesis-stage evidence strongest for the engine where retrieval context was observable.
9 Controlled Retrieval Injection
To answer RQ3 with full experimental control, we isolated the synthesis stage by injecting matched documents directly into an open-weights model's context window.
9.1 Design
Using Llama-3-70B-Instruct, we created matched pairs for 200 factual queries: dstruct (SAA-formatted with <dt><dd> pairs, inverse pyramid, header-answer pairing) and dunstruct (same facts in a 3-paragraph narrative without structural markup). Both contained identical facts, identical word count (± 5%), injected at the same context position. Prompt: “Answer the following question using the provided context. Cite your source.” Metrics: citation rate, extraction fidelity (verbatim answer reproduction vs. vague paraphrase), and attention entropy (H = −∑ipilog pi over document tokens at the final layer).
9.2 Results
Table 3: Controlled Injection: SAA vs. Unstructured (Llama-3-70B, N = 200 × K = 10).
Metric | dstruct | dunstruct | p |
|---|---|---|---|
Citation Rate | 91.2% | 34.8% | < 0.001 |
Extraction Fidelity | 87.4% | 22.1% | < 0.001 |
Attention Entropy (H) | 2.14 | 3.87 | < 0.001 |
Under controlled conditions, structured documents achieve 2.6 × higher citation rates and 4.0 × higher extraction fidelity. Lower attention entropy confirms more concentrated attention on relevant tokens with explicit definition pairs. In Dexter terms (Section 5): well-anchored components exhibit lower runtime resolution failure. Replicated on Mistral-7B-Instruct with qualitatively similar results (Citation Rate: 84.3% vs. 29.1%; Extraction Fidelity: 79.8% vs. 18.6%), suggesting the SAA advantage is robust across the open-weights models tested.
10 Cross-Domain Validation and Ablation
10.1 Cross-Domain Validation
To test generalizability beyond N = 2 primary subjects, we extended CCA (ICM+SAA only, no community participation) to 10 additional mid-sized entities (DA 20–50) across 4 verticals in the Indian market, each evaluated on 200 queries over a 90-day window.
Table 4: Cross-Domain: ΔgSoV after 90-day ICM+SAA.
Vertical | Intent | Ne | Nq | ΔgSoV | p |
|---|---|---|---|---|---|
EdTech | Subjective | 3 | 600 | +14.2 pp | < 0.001 |
Legal | Factual | 2 | 400 | +28.7 pp | < 0.001 |
FinTech | Mixed | 3 | 600 | +11.8 pp | 0.003 |
Healthcare | Factual | 2 | 400 | +31.4 pp | < 0.001 |
Aggregate | 10 | 2,000 | +21.5 pp | < 0.001 |
Factual domains (Legal +28.7, Healthcare +31.4 pp) showed larger gains, consistent with the Dexter-formalized SAA mechanism. Subjective domains showed moderate gains without community signals (EdTech +14.2, FinTech +11.8 pp). Aggregate across all 12 entities: Cohen's d = 1.42, 95% CI for population ΔgSoV: [17.1, 25.9] pp.
Scope of field-level claims. The full CCA framework (all three vectors including community signals) was field-tested on exactly two enterprises, both in the Indian market. The 10 validation entities received only ICM+SAA over 90 days. The cross-domain results support the generalizability of structural interventions, but the community signal mechanism's field-level evidence rests on N = 2, supplemented by the controlled Source Substitution Experiment which provides the causal evidence for that vector.
10.2 Ablation
Leave-one-out ablation on primary subjects, leveraging staggered deployment for temporal separation.
Table 5: Ablation: % drop in gSoV relative to full CCA.
Ablation | Removed | Δ Travel | Δ Food |
|---|---|---|---|
$mathcal {M}{-text{Soc}}$ | Community | $-82.4%$ | $-4.1%$ |
$mathcal {M}{-text{SAA}}$ | Structure | $-12.3%$ | $-91.5%$ |
$mathcal {M}{-text{ICM}}$ | Intent Clust. | $-45.6%$ | $-38.2%$ |
Clean domain-mechanism mapping: community signals prerequisite for subjective queries ($-82.4%$); SAA prerequisite for factual ($-91.5%$); intent clustering universally important ($approx {-}40%$).
11 Robustness (RQ4)
Under temperature scaling (τ : 0.0–1.0), structured content exhibited low citation variance (σ2 = 0.04) even at τ = 0.9, while community-backed content showed high variance (σ2 = 0.21)—consistent with the model exploring diverse opinions at high temperatures while factual extraction remains stable. Under adversarial prompt perturbations (N = 500 variations), CCA showed high resilience to format constraints (AR = 0.96) and negative constraints (AR = 0.88), but vulnerability to persona constraints such as “act as a skeptic” (AR = 0.42). This suggests CCA aligns with default model preferences rather than exploiting a vulnerability.
12 Anatomy of a Citation: Two Case Studies
12.1 Case A: The Invisible #3 Result (Subjective Query)
Query: “Is it safe for a female solo traveler to stay in a hostel in North Goa?”
During baseline, the Travel entity had a blog post ranking #3 in Google. Despite this strong retrieval position, ChatGPT consistently cited two competitors with 50+ Reddit mentions instead. The entity was absent from synthesis despite being in the retrieval context—the AEO Gap in action. After community participation (month 8), with genuine guest narratives on r/TwoXIndia and r/Goa, ChatGPT began citing the entity as “frequently recommended by community members.” The website content had not changed; its presence in the community corpus did.
12.2 Case B: The Unreadable #1 Result (Factual Query)
Query: “FSSAI display requirements for cloud kitchens.”
The Food Consultancy's PDF ranked #1 in Google. Yet Perplexity generated a generic response from parametric memory, citing no source—SRR of 100% for the specific dimensional fact (gold signage of 12 × 12 inches). After SAA deployment (month 6) with <dl><dt>Display Size</dt><dd>Gold signage of 12 × 12 inches</dd></dl>, Perplexity cited the entity with exact specifications. The facts were identical; the hypertext structure—the anchor structure in Dexter terms—made them extractable.
13 Discussion
13.1 Two Mechanisms of Citation Bias
The controlled injection result is the cleanest evidence in this paper: when facts are held constant and only structure changes, citation and extraction fidelity change materially.
For factual queries, the LLM selects structured documents because they are computationally easier to extract from (H = 2.14 vs. 3.87)—a prediction of the Dexter model confirmed empirically. For subjective queries, source substitution shows that corpus placement modulates synthesis probability independently of content quality, with synthesis-stage evidence strongest where retrieval context was observable.
13.2 Structured Authoring as a Hypertext Problem
Our findings connect to a central tension in hypertext: how content structure determines reusability across traversal contexts. The Dexter model [15] formalized the principle that well-anchored components enable reliable composition; Nelson's transclusion [20] envisioned content structured for granular reuse. SAA operationalizes these for the generative web.
The 2.6 × citation advantage of structured HTML (Table 3) is a finding about hypertext design. Semantic markup functions as a machine-readable contract between author and synthesizer. In Dexter terms, these are components with explicit anchors; in GenIR terms, extraction-ready fragments. Content lacking this structure is retrieved but not understood.
The dual-audience argument. Does machine-optimized formatting degrade human readability? We argue the structural properties that reduce synthesis rejection also improve human traversability. Atomic answer units with explicit question-answer pairing are independently navigable—scannable, linkable, bookmarkable—unlike dense prose where answers are buried in narrative context. In Brusilovsky's adaptive hypermedia framework [7], this corresponds to increasing the granularity of adaptable units, enabling both human navigation and machine composition from the same authored structure. SAA thus reduces friction for machines while preserving it for humans [2]—the same structural explicitness that enables machine extraction also enables human scanning and non-linear navigation.
The risk of Semantic Homogenization remains: if all content converges to answer-block templates, the rhetorical and structural diversity of the hypertext web may be pruned. Whether this trade-off is acceptable is a question the hypertext community is positioned to address, and one our gSoV framework can help quantify in future work.
13.3 Erasure by Synthesis
Failure to align with structural incentives risks what we term Erasure by Synthesis: an entity provides the content the LLM draws upon, but receives no attribution. Subject B's PDF contained the authoritative answer and ranked #1, yet the LLM paraphrased from parametric memory and cited no source. The entity's knowledge was consumed but its contribution was invisible. In a referral economy, invisible content still generates traffic through search listings; in an attribution economy, invisible content generates nothing.
13.4 Long-Tail Dynamics
Strong correlation (r = 0.91) between query complexity and citation probability. As queries become specific, precisely-targeted content achieves near-monopoly visibility. The low Jaccard agreement ($mathcal {J}{=}0.24$–0.31) further implies no single-engine strategy guarantees visibility—practitioners must adopt ensemble evaluation, and gSoV with multi-engine sampling provides the infrastructure.
14 Ethical Considerations
We recognize that this work raises important ethical concerns that warrant explicit discussion, following the framework of Kumar and Lakkaraju [17] for responsible disclosure of LLM manipulation vectors.
14.1 Community Participation and Authenticity
The community signal amplification vector involves participation in online discussion forums to increase an entity's presence in community corpora. We acknowledge the ethical tension between authentic engagement and strategic manipulation.
Risk of Astroturfing. If scaled without ethical guardrails, this strategy could degenerate into astroturfing—the creation of inauthentic content designed to appear organic. This is problematic for any domain, but is particularly concerning for safety-sensitive topics such as the “solo female traveler safety” queries in our Travel dataset. Inauthentic endorsements of accommodation safety could mislead users about genuine conditions, potentially causing real harm.
Our Protocol. During the study, community posts were written by actual guests describing genuine experiences at the property. No fabricated testimonials or fictitious personas were used. Posts disclosed the author's relationship to the brand where required by subreddit rules. However, we recognize that this distinction (authentic-but-strategic vs. fabricated) is difficult to enforce at scale and may be insufficient to mitigate systemic risks.
Recommendation. We advocate for: (i) transparency norms—content created with promotional intent should be disclosed, following the FTC Endorsement Guidelines model; (ii) platform-level detection—GenIR platforms should develop mechanisms to detect coordinated inauthentic activity in their retrieval corpora, analogous to social media platform policies; (iii) researcher responsibility—we release our methodology to enable defensive analysis (detecting such manipulation) rather than solely offensive optimization.
14.2 Semantic Homogenization and Model Collapse
Our Perplexity Minimization mechanism implies that RAG systems favor structurally predictable, syntactically simple content. If content creators universally adopt SAA, we risk a Semantic Homogenization of the web—the linguistic and structural diversity of the open web collapsing into standardized “Answer Block” formats. This creates a feedback loop where LLMs train on increasingly homogenized data, potentially contributing to model collapse in future iterations [22]. The historical richness of hypertext—varied rhetorical structures, experimental layouts, non-linear narratives—may be pruned in favor of formats that minimize extraction cost.
14.3 Erosion of Reader Agency
From a hypertext perspective, the most fundamental ethical concern is the erosion of reader agency—a core principle since Engelbart's augmentation paradigm. In classical hypertext, readers navigate links and construct their own interpretive paths. In GenIR, an agentic mediator performs this traversal on the reader's behalf. Our finding that different engines produce radically different compositions ($mathcal {J}{=}0.24$) raises the question: whose traversal does the reader receive? We argue that GenIR platforms should expose citation provenance to preserve the reader's ability to inspect linking structures—a form of “open hypertext” for the agentic era [2].
14.4 Competitive Fairness
The community signal effect suggests that entities with established community footprints enjoy compounding advantages. Unlike Google's “Freshness” algorithm which gives new pages ranking opportunities, community-based citation signals are historically weighted, potentially calcifying market leaders and reducing discoverability of novel entities.
15 Threats to Validity and Future Work
Internal validity. The community signal vector creates endogeneity; our strongest causal evidence comes from the two controlled experiments. Query generation with GPT-4o may bias the long-tail query set. Model updates during the 12-month window introduce drift partially controlled by DiD.
Statistical conclusion validity. Monte Carlo sampling over stochastic outputs introduces dependence; p-values should be interpreted alongside effect sizes and cross-engine consistency.
Construct validity. gSoV measures explicit attribution, not revenue. Citation schemas differ across engines; our extractor was validated but cannot capture all paraphrased attributions.
External validity. Data restricted to Indian market (en-IN). Source Substitution used N = 6 synthetic entities. Fresh sessions only; personalization effects unexplored.
Causal scope. The strongest causal claims come from controlled experiments (source substitution, retrieval injection). Field estimates are quasi-causal under stated assumptions, not full identification of proprietary platform internals.
Future work. Cross-linguistic replication; engine-invariant citation ontology; longitudinal measurement of semantic homogenization using structural entropy; personalization-AEO interaction; and development of hypertext authoring tools that serve dual audiences—machines and human readers—without sacrificing structural diversity.
16 Conclusion
This paper presents an empirical evaluation of Answer Engine Optimization through controlled experiments, field evidence, and cross-domain validation. We introduced gSoV as a metric for non-deterministic retrieval systems and evaluated Citation-Centric Alignment.
Our strongest result is structural: under controlled retrieval injection, changing authored structure while holding facts constant materially changes citation and extraction outcomes. We formalized this using the Dexter Reference Model, showing that SAA Answer Units function as self-describing components whose anchor structure reduces synthesis rejection—the GenIR synthesis stage operates as a machine-mediated Dexter runtime layer.
Source substitution and field estimates provide convergent support. Cross-domain validation across 4 verticals (Cohen's d = 1.42) confirms generalizability of structural interventions; community signal field evidence rests on N = 2 supplemented by controlled evidence. Cross-platform agreement is low ($mathcal {J}{=}0.24$–0.31), implying no single-engine strategy guarantees visibility.
The contribution is less about making systems smarter than about updating hypertext design for machine-mediated environments. Classical hypertext optimized authored form for human traversal; generative hypertext requires form that supports both human and machine traversal. The retrieval-to-synthesis interface is a new site of hypertext design, where node structure, semantic markup, and answer granularity directly shape visibility and attribution. We release our protocol and code to support replication.
Notes
1Repository: https://github.com/raghav103/AEO
Source
Imported from ACM’s structured HTML source. ACM Reference Format: Manan Mehta, Raghav Sharma, Vineeth Kalluru, and Adit Kotwal. 2026. Beyond the Blue Link: Empirical Evaluation of Generative Engine Optimization in Stochastic Retrieval Systems. In 37th ACM Conference on Hypertext (HT '26), September 14--18, 2026, London, United Kingdom. ACM, New York, NY, USA 8 Pages. https://doi.org/10.1145/3800935.3830864
References
[1] Pranjal Aggarwal, Vishvak Murahari, Tanmay Rajpurohit, Ashwin Kalyan, Karthik Narasimhan, and Ameet Deshpande. 2024. GEO: Generative Engine Optimization. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD ’24). ACM, 5–16. https://doi.org/10.1145/3637528.3671900
[2] Shm Garanganao Almeda, J.D. Zamfirescu-Pereira, Kyu Won Kim, Pradeep Mani Rathnam, and Bjoern Hartmann. 2025. Agency Among Agents: Designing with Hypertextual Friction in the Algorithmic Web. In Adjunct Proceedings of the 36th ACM Conference on Hypertext and Social Media (HT ’25). ACM. https://doi.org/10.1145/3720533.3750065
[3] Mark W. R. Anderson and David E. Millard. 2023. Seven Hypertexts. In Proceedings of the 34th ACM Conference on Hypertext and Social Media (HT ’23). ACM. https://doi.org/10.1145/3603163.3609071
[4] Claus Atzenbeck. 2024. Unwinding AI's Moral Maze: Hypertext's Ethical Potential. In Proceedings of the 35th ACM Conference on Hypertext and Social Media (HT ’24). ACM.
[5] Sergey Brin and Lawrence Page. 1998. The Anatomy of a Large-Scale Hypertextual Web Search Engine. In Proceedings of the 7th International Conference on World Wide Web. 107–117. https://doi.org/10.1016/S0169-7552(98)00110-X
[6] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language Models are Few-Shot Learners. In Advances in Neural Information Processing Systems, Vol. 33. 1877–1901.
[7] Peter Brusilovsky. 2001. Adaptive Hypermedia. User Modeling and User-Adapted Interaction 11, 1–2 (2001), 87–110. https://doi.org/10.1023/A:1011143116306
[8] Guiming Hardy Chen, Shunian Chen, Ziche Liu, Feng Jiang, and Benyou Wang. 2024. Humans or LLMs as the Judge? A Study on Judgement Bias. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics.
[9] Mahe Chen, Kaiwen Chen, et al. 2025. Generative Engine Optimization: How to Dominate AI Search. arXiv preprint arXiv:2509.08919 (2025).
[10] Paul De Bra, Geert-Jan Houben, and Hongjing Wu. 1999. AHAM: A Dexter-based Reference Model for Adaptive Hypermedia. In Proceedings of the 10th ACM Conference on Hypertext and Hypermedia. ACM, 147–156. https://doi.org/10.1145/294469.294508
[11] Shahul Es, Jithin James, Luis Espinosa Anke, and Steven Schockaert. 2024. RAGAS: Automated Evaluation of Retrieval Augmented Generation. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations. 150–158.
[12] Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, and Haofen Wang. 2024. Retrieval-Augmented Generation for Large Language Models: A Survey. arXiv preprint arXiv:2312.10997 (2024).
[13] Gartner. 2024. Gartner Predicts Search Engine Volume Will Drop 25% by 2026, Due to AI Chatbots and Other Virtual Agents. Press Release. https://www.gartner.com/en/newsroom/press-releases/2024-02-19-gartner-predicts-search-engine-volume-will-drop-25-percent-by-2026
[14] Frank G. Halasz. 1988. Reflections on NoteCards: Seven Issues for the Next Generation of Hypermedia Systems. Commun. ACM 31, 7 (1988), 836–852. https://doi.org/10.1145/48511.48514
[15] Frank G. Halasz and Mayer D. Schwartz. 1994. The Dexter Hypertext Reference Model. Commun. ACM 37, 2 (1994), 30–39. https://doi.org/10.1145/175235.175237
[16] Kalervo Järvelin and Jaana Kekäläinen. 2002. Cumulated Gain-Based Evaluation of IR Techniques. ACM Transactions on Information Systems 20, 4 (2002), 422–446. https://doi.org/10.1145/582415.582418
[17] Aounon Kumar and Himabindu Lakkaraju. 2024. Manipulating Large Language Models to Increase Product Visibility. arXiv preprint arXiv:2404.07981 (2024).
[18] R. Anil Kumar, Zaiduddin Shaik, and Mohammed Furqan. 2019. A Survey on Search Engine Optimization Techniques. International Journal of P2P Network Trends and Technology (2019). https://doi.org/10.14445/22492615/IJPTT-V9I1P402
[19] Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. In Advances in Neural Information Processing Systems, Vol. 33. 9459–9474.
[20] Theodor H. Nelson. 1965. Complex Information Processing: A File Structure for the Complex, the Changing and the Indeterminate. Proceedings of the 1965 20th National Conference of the ACM (1965), 84–100. https://doi.org/10.1145/800197.806036
[21] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. Training Language Models to Follow Instructions with Human Feedback. In Advances in Neural Information Processing Systems, Vol. 35. 27730–27744.
[22] Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Nicolas Papernot, Ross Anderson, and Yarin Gal. 2024. AI Models Collapse When Trained on Recursively Generated Data. Nature 631 (2024), 755–759. https://doi.org/10.1038/s41586-024-07566-y
[23] Jian Zhang, Qi Zhang, Biao Wang, et al. 2024. OCR Hinders RAG: Evaluating the Cascading Impact of OCR on Retrieval-Augmented Generation. arXiv preprint arXiv:2412.02592 (2024).
[24] Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. In Advances in Neural Information Processing Systems, Vol. 36.
Do you like what you are reading? Subscribe to receive updates.
Unsubscribe anytime