Semantic Monitoring Under AI Component Evolution

Executive Summary

I studied a narrow failure mode in an AI-enabled runtime monitor. A learned extractor reads a natural-language workflow message and converts it into a structured fact. A deterministic checker then uses that fact to decide whether an execution is safe. I asked whether changing only the extractor could produce a schema-valid fact with the wrong meaning and cause the unchanged checker to approve an unsafe execution.

The rule in the pilot was simple: an artefact must pass validation before it is consumed. Each message could mention two artefacts, A and B. In the harmful case, B passed validation, A did not, and A was consumed. The correct verdict was therefore unsafe. A semantic binding error occurred when the extractor kept the status passed but attached it to A rather than B. The JSON still matched its schema, so ordinary structural validation accepted it.

I constructed a corpus of 240 workflow traces and evaluated 168 of them. Of these, 140 were unsafe and 112 were primary at-risk cases where the binding error could flip an unsafe verdict to safe. The experiment compared a Qwen2.5 7B baseline with a Qwen2.5 3B downsize, Llama 3.1 8B, prompt simplification, prompt synonym changes, and upstream paraphrasing. The ground truth came mechanically from the constructed execution rather than from another language model.

The main harmful result came from downsizing Qwen2.5 7B to Qwen2.5 3B. The smaller model produced 5 false-safe verdicts among 112 at-risk executions, or 4.46%, with an exact 95% interval from 1.47% to 10.11%. The baseline produced 0 of 112. In the highlighted 3B arm, all 336 extractor outputs remained schema-valid, but 12 of 336 attached the correct validation status to the wrong artefact. Five of those errors propagated into a false-safe decision.

This established a controlled failure, but not a general production problem or a need for a new runtime mechanism. The dataset was synthetic, the effect involved one rule and one small model, and entity order or naming patterns may have contributed. A generic disagreement gate could also reject harmless changes, suggesting that careful pre-deployment validation might be sufficient.

Main lesson. Structural validity is not semantic validity. A deterministic checker can be logically correct and still return the wrong verdict when a learned upstream component changes which real-world entity a valid field refers to.

Research Arc and Verification

Ideation and exploration

I started with a broad concern: a deterministic monitor can depend on facts supplied by a learned component. The useful narrower question was whether a component change could preserve JSON validity while changing which entity a status described. I constructed a two-artefact workflow because it makes the semantic binding and the downstream safety rule mechanically checkable.

Understanding

The decisive experiment compared the Qwen2.5 7B extractor with the smaller Qwen2.5 3B replacement while holding the rule, checker, and constructed execution fixed. The claim would have failed if structural errors were caught before the checker, if wrong bindings did not change any safety verdict, or if the baseline produced the same false-safe pattern.

Verification and red team

• The harness generated the true execution, consumed artefact, and validation state; an LLM judge did not provide ground truth.
• The baseline used the larger model under the same task and rule.
• The record distinguishes 12 wrong bindings from the 5 that actually propagated into false-safe verdicts.
• Schema validation was tested directly and accepted all highlighted outputs; this rules out a formatting-only explanation.
• The main result was treated as an existence demonstration, not a stable rate: five events produce a wide exact confidence interval.
• The red team remains incomplete. Entity order, naming, prompt wording, quantisation, and the synthetic two-artefact setting can still explain the observed effect.

The record therefore supports a specific, mechanically verified failure mode. It does not support a general claim about production prevalence, all model replacements, or the need for a new runtime mechanism.

Motivation & Context

Production AI systems often combine learned and deterministic components. A model may extract facts, classify an event, or label a trace, while ordinary code applies policies to that output. Teams may replace the model to reduce cost or latency, change the prompt, or move to a different model family.

Most validation focuses on output format, aggregate accuracy, or disagreement between old and new models. I wanted to examine a more specific dependency: a checker may rely not only on the value passed, but also on which artefact that value describes.

The tested pipeline was:

True execution
Language message
Learned extractor
Structured fact
Fixed deterministic checker
Safe or unsafe verdict

Only the learned extractor or its immediate input wording changed. The rule and checker remained fixed.

This pilot connects to my broader interest in reliability under component evolution. It tests whether an old guarantee remains trustworthy when the learned observation supplying its facts changes. Agentic Data Environments similarly treats the stateful environment and its enforcement points as part of the agent system rather than passive context 1.

Hypotheses

H1: Schema-valid semantic regressions can change a safety verdict

An extractor can return valid JSON with the correct status but bind it to the wrong artefact, causing the checker to return safe for an unsafe execution.

Falsification condition: all schema-valid extractor errors are harmless or caught before the checker makes its decision.

H2: Model downsizing increases harmful binding errors

Replacing Qwen2.5 7B with Qwen2.5 3B will produce more wrong-artefact assignments and false-safe verdicts.

H3: Simple validation signals are insufficient

Schema validation and the marginal frequency of passed will not detect an error that preserves both the structure and status value.

H4: The effect generalizes beyond one constructed setup

The failure should survive new messages, randomized entity order, randomized prefixes, additional extractors, and other temporal rules.

This hypothesis was not tested adequately in the pilot.

Methods & Experiments

System and rule

The deterministic monitor enforced:

validate(A) before consume(A)

An unsafe execution could contain:

B219 passed validation.
A219 did not pass validation.
A219 was consumed.

The extractor received a natural-language message and returned a record containing an artefact identifier and validation status. The monitor combined extracted validation facts with the known consumption event.

Experimental data

The 112 primary cases were those in which a wrong entity binding could turn the monitor's verdict from unsafe to safe.

Experimental arms

Ground truth and metrics

The harness constructed the execution and therefore knew which artefact passed, which artefact was consumed, and the correct safe or unsafe verdict. No LLM judge determined the ground truth.

The main metrics were:

• schema-valid output rate
• wrong-artefact assignment rate
• monitor disagreement from baseline
• false-safe rate among the 112 at-risk unsafe cases

A false-safe verdict was the primary harmful outcome because it allowed an execution that violated the rule.

Success and failure conditions

The pilot would support H1 if a component change produced at least one reproducible, schema-valid semantic error that propagated into a false-safe verdict while the checker and rule remained fixed.

It would support a broader research claim only if the effect survived stronger controls, appeared across multiple models or prompts, and was not adequately handled by ordinary shadow evaluation or regression testing.

Results

The downsize produced false-safe verdicts

The exact 95% interval for the 3B false-safe rate was 1.47% to 10.11%. With only five events, the interval is wide and the estimate should not be treated as a stable production rate.

Figure 1. False-safe verdicts among 112 at-risk unsafe executions for Qwen2.5 7B and Qwen2.5 3B, with exact confidence intervals. The controlled failure exists, but its frequency is estimated imprecisely.

The failure remained structurally valid

For the highlighted 3B arm:

The extractor often preserved the correct status while changing its referent. For example:

// Correct binding
{"artifact_id":"B219", "status":"passed"}

// Harmful but schema-valid binding
{"artifact_id":"A219", "status":"passed"}

In the second output, the monitor sees evidence that A passed and approves A's consumption. The deterministic logic behaves exactly as written, but the fact no longer means what the checker assumes.

Simple checks did not resolve the problem

• Schema validation accepted the harmful output.
• Counting how often passed appeared could not detect the change because the status value was preserved.
• Generic old-versus-new disagreement could detect changes, but it also reacted to harmless arms that changed extracted details without flipping the final verdict.

The stored results did not contain a complete arm-by-arm numerical table for every model and prompt condition. I therefore do not infer missing rates.

Qualitative example

The A219/B219 example is the preserved mechanism example from the experiment. It is illustrative, not a randomly sampled case. The original record does not preserve a labeled random sample of all messages, so a random qualitative panel remains an artifact to reconstruct from the raw data.

Analysis: Why It Failed (or was inconclusive)

The pilot successfully demonstrated one controlled failure, but the broader research framing did not survive.

1. The dataset was constructed. The messages were designed to create two-entity ambiguity. This isolates the mechanism but does not establish prevalence in production traces.
2. Only one rule was tested. The precedence rule validate(A) before consume(A) does not represent absence, response, authorization, counting, or other runtime properties.
3. The harmful result was small. Five false-safe cases show existence, not a stable effect size.
4. One smaller model drove the main effect. The result may reflect Qwen2.5 3B specifically rather than model downsizing in general.
5. Position and naming were possible confounds. If A usually appeared before B, or particular prefixes were associated with the target artefact, the model may have exploited superficial patterns.
6. A new runtime mechanism was not shown to be necessary. Careful pre-deployment testing, shadow comparison, abstention, or blocking on disagreement may be sufficient for this setting.
7. The baseline question needed more precision. Any component swap can sometimes change behavior. The interesting question is not whether a swap can fail, but whether a specific validation procedure can bound the downstream false-safe risk at an acceptable cost.

Alternative explanations include prompt sensitivity, quantization or runtime differences, and errors in semantic-equivalence pairs. A second-person review and a new unseen test set were needed before treating the effect as a model capability difference.

The negative result teaches that showing a downstream failure is not enough to establish a research problem. The work also needs a real setting, a defined change class, a strongest operational baseline, and evidence that existing deployment testing does not already handle the failure.

Key Insight

A valid value can refer to the wrong thing

The monitor did not fail because passed changed to failed. It failed because passed moved from B to A.

This distinction matters in composed AI systems:

Syntactic contract: artifact_id is a string and status is an allowed value.
Semantic contract: status describes the real artifact named by artifact_id.
Behavioral dependency: the checker applies that fact to the artifact being consumed.

Schema tools verify the first line. The pilot failure occurred in the second line and propagated through the third.

The experiment also separates deterministic and stochastic behavior. The checker was deterministic: the same structured facts produced the same verdict. The learned extractor was not reliably stable under component change. The system-level outcome depended on both. Calling the whole pipeline deterministic because its final rule was code would therefore be misleading.

What I Learned & Research Skill Demonstrated

• I reduced a broad question about evolving AI systems to one explicit dependency boundary.
• I constructed a mechanical oracle instead of relying on an LLM judge.
• I distinguished schema validity, semantic correctness, and downstream decision correctness.
• I measured propagation: 12 wrong bindings did not automatically mean 12 false-safe decisions.
• I included harmless changes and learned that generic disagreement can overreact.
• I reported the exact confidence interval and did not treat five events as a production prevalence estimate.
• I accepted my professor's criticism that the system, input, output, allowed change, and success criterion must be defined before claiming a research problem.
• I learned that an existence demonstration and a publishable gap are different achievements.

Open Questions & Future Directions

1. Replication with stronger controls: Create a new unseen set, have a second person review semantic-equivalence pairs, randomize whether A or B appears first, and randomize entity prefixes.
2. Model breadth: Test additional small extractors to determine whether the failure is model-specific.
3. Rule breadth: Compare precedence, absence, response, authorization, and counting properties.
4. Operational baseline: Evaluate whether a tuned shadow-comparison gate catches harmful changes without rejecting harmless ones.
5. Real data: Search for a production-like benchmark where a learned extractor supplies facts to a mechanical downstream consumer.
6. Decision procedure: Define how many executions are required to accept a replacement at a stated false-safe risk, rather than asking whether the replacement behaves identically.

The planned follow-up proposed about 200 at-risk cases. That number was a practical expansion target, not a fully justified power calculation. A new study should derive sample size from the smallest false-safe increase that would change an actual deployment decision.

Appendix

Supported and unsupported conclusions

Figure reconstruction checklist

• Recover per-arm output and verdict tables from the raw pilot artifact.
• Sample qualitative cases using a fixed random seed.
• Plot extractor error type separately from downstream verdict impact.
• Show exact confidence intervals for all harmful arms.
• Label constructed examples separately from sampled examples.

References

1. E. Ang, C. Huang, G. Liargkovas, J. Liu, J. Liu, N. Pagonas, C. Summers, H. Wang, J. Xu, T. Zhou, Y. Zhang, Z. Yu, Z. Zhang, T. Peng, K. Kaffes, and E. Wu, “Agentic Data Environments,” IEEE Data Engineering Bulletin, vol. 50, no. 1, 2026. Online. Available: https://arxiv.org/abs/2607.07397