Session Date/Time: 18 Mar 2024 07:30

irtfopen

Summary

This meeting of the IRTF Open session at IETF 119 in Brisbane included the usual IRTF/IETF reminders, an update on IRTF activities, and the Applied Networking Research Prize award presentation by Zhong Chi Han on "Anomaly Detection in the Open World: Normality Shift Detection, Explanation, and Adaptation." The discussion focused on the challenges of applying anomaly detection in real-world networks, particularly addressing concept drift and the need for normality shift detection.

Key Discussion Points

IRTF Updates: Recent RFC publications, ongoing work on the IRTF code of conduct, and the Applied Networking Research Prize and Workshop were highlighted. Travel grants are available for future meetings.
Applied Networking Research Prize: Zhong Chi Han presented his award-winning work on open-world anomaly detection (OWD).
Normality Shift vs. Concept Drift: The presentation emphasized the distinction between concept drift (changes in anomaly distribution) and normality shift (changes in normal data distribution) and how the latter poses unique challenges for anomaly detection.
OWD Framework: The OWD framework, consisting of output calibration, shift detection, shift explanation, and shift adaptation, was described. Key to this is the need for labeling data in the context of normality shift.
Labeling Overheads: The discussion covered the challenge of labeling data for anomaly detection, including the level of expertise required. Specifically the amount of labeled data needed in practice for deployment on real networks was raised.
Initial Data Collection: The source and validity of the initial "normal" data used to bootstrap the system was discussed, including the reliance on an assumption of attack-free periods and the possibility of using the feedback loop to refine this data.
Explanation Mechanism: The nature of the explanation provided by OWD was explored, particularly whether it identifies specific applications causing anomalies or provides data samples for human analysis. The sensitivity of the OWD performance to the accuracy of the significant sample finding process was also raised.
Real-world Applicability: The speaker and the audience discussed challenges in applying anomaly detection to complex, real-world SCADA systems, particularly the issue of false positives due to the difficulty of defining "normal" traffic.
Data Availability: Availability of attack free data sets for training anomaly detection systems was discussed.

Decisions and Action Items

Action Item: Colin Perkins noted that the IETF has a new working group focused on ML-Ops and encouraged Zhong Chi Han to contribute his research.

Next Steps

Participants were encouraged to consider submitting papers to the Applied Networking Research Workshop co-located with the IETF meeting in Vancouver in July.
Interested parties are encouraged to check the IRTF website for details regarding travel grants.