[In-person]GoalLog: A Goal-Directed Multi-Stage Agent for Automated Log-Based Incident Investigation

GoalLog: A Goal-Directed Multi-Stage Agent for Automated Log-Based Incident Investigation
ID:18 Submission ID:201 View Protection:ATTENDEE Updated Time:2026-10-04 23:16:35 Hits:163 In-person

Start Time:2026-10-13 11:45 (Asia/Ho_Chi_Minh)

Duration:15min

Session:[S5] Track 5: Emerging Trends of AI/ML » [S5-2] Track 5: Emerging Trends of AI/ML

Presentation File Attachment File

Tips: The file permissions under this presentation are only for participants. You have not logged in yet and cannot view it temporarily.

Abstract
Automated incident investigation requires not only anomaly detection but structured root cause analysis (RCA), calibrated confidence, and an auditable trail. We present GoalLog, a goal-directed agent that organizes the investigation as a deterministic four-state machine (detect, collect, diagnose, remedy) and fuses LLM, statistical and BM25 retrieval evidence into root-cause beliefs using Dempster–Shafer theory with an explicit ignorance mass. On 200 HDFS incidents it detects at a 1.4% false positive rate at 97% recall, against 75–86% for our reimplementations of ungated single-turn LLM approaches, and is far better calibrated (ECE 0.183 vs. 0.49–0.52). Our main result, however, is negative and we report it in full: on both corpora top-1 RCA accuracy is carried by retrieval alone. TF- IDF + logistic regression reaches 99.0% on HDFS because that taxonomy is induced from event codes the classifier reads directly, so HDFS cannot evaluate this class of system; and on BGL, whose alert categories operators wrote independently of our features, a nearest-neighbour label transfer over the same BM25 index, with no LLM call, reaches 90.0% against the pipeline’s 89.5%. What the fusion adds at equal accuracy is confidence: the ignorance mass cuts BGL calibration error from 0.098 to 0.030 and supports abstention at an absolute threshold, which no accuracy-matched baseline provides.
Keywords
log analysis,incident investigation,root cause analysis,LLM agent,Dempster–Shafer theory,anomaly detection
Speaker
Duc-Tuan Luu
Student Posts and Telecommunications Institute of Technology

Submission Author
Quang-Thanh Phan Posts and Telecommunications Institute of Technology
Duc-Tuan Luu Posts and Telecommunications Institute of Technology
Comment submit
Verification code Change another
All comments

CONTACT US

Email: asiancomnet2026@usssociety.org

Website & IT Support: hi@aconf.org

Registration Submit Paper