Yue Zhao

Research Summary

My research develops Auditable AI Systems: methods, benchmarks, and open-source infrastructure that provide evidence for relying on AI systems when ground truth is unavailable. It asks four questions of the record a system leaves behind. Data: does the input resemble anything the system was built for? Output: does it hold up against evidence? Action: was it allowed? Effort: was the work necessary? Three questions ask whether something went wrong; the fourth asks whether the work was needed. Building the record that answers all four is the lab's contribution to AI Assurance.

1. Detection in Data
I study anomaly, outlier, and out-of-distribution detection at scale. This work asks whether an input departs from the conditions represented during system development and evaluation, especially when labels or representative examples are scarce. It includes the PyOD ecosystem, ADBench, automatic out-of-distribution detector selection, and methods for graph, multimodal, and few-shot cross-domain detection.

2. Verification of Output
I develop methods for evaluating foundation-model outputs when no reference answer is available. The work includes TrustLLM and studies of trustworthiness, causal analysis of hallucination, jailbreak detection, adversarial retrieval, and model-based evaluation.

3. Auditing of Action
I study the security and auditability of AI agents that act through tools, APIs, and shared state. The work spans pre-deployment inspection, runtime tool-call mediation, and post-run attribution and recovery. Auditable Agents organizes this area around five dimensions of auditability and three mechanism classes: detect, enforce, and recover. Related work examines agent-specific failure modes such as over-privilege and cross-user contamination.

4. Optimization of Effort
I study whether the work performed by an AI system was necessary. This direction uses execution records and dependency analysis to identify redundant or avoidable work. It includes GRADE, online model routing under changing cost and accuracy needs, dynamic workflow construction, and measurement of the autonomy cost imposed by defense training.