I did a talk at Splunk .conf21 about how to maintain correlation searches: pdf/mp4. One of the topics is having a simple task management system that integrates with Enterprise Security and the other things I’ve made. The idea is to make it easy for analysts and developers alike to raise TODOs against correlations, optionally linking them to problematic notable events.
For the big things you will still need a proper ticketing system. But for the day to day issues such as noise, missed events, or a lack of context in an alert, it helps to have a simple in-splunk system.
It’s actually simpler than you might think. Initially I used the collect command, but I noticed sometimes it would be less than instantaneous. Then I took the plunge and learned about the KV store. It’s very easy and any change is instantaneous. The solution uses both:
the KV store keeps any pending TODOs, so that they are very easy and quick to manage
and the index keeps everything and allows to see when TODOs were created and resolved
If you want to get the source code of what you see in the talk, it’s all in the ES Choreographer app in splunkbase.
Comments