As detection researchers we are frequently asked where do our detection ideas come from (and to build a backlog for them, and when will it all be done, etc.). At some point I needed to stop referencing Demetri Martins’ stand up where he describes how his jokes are delivered by a delicate fairy from a magical shire (the AI drawing may make more sense now…or not).
It’s important to think through where these ideas come from and how they are formed so we have the time, resources and the correct mindset to continue to extract more meaning from the data.
I have outlined a high level thought process and throughout this post there are questions to ask at each phase. I’ll share some insights on why these questions are useful when it’s not extremely obvious. Part 2 of this post walks through this process using the DCSync attack example here.
The Starting Point
This is often the most difficult piece to identify since there are so many choices! We can keep it simple for now and say the starting point is an attack since that is what we’re usually attempting to detect. We can break it down like this:
Attack (X) was performed against a Service (Y) creating a disturbance (Z)
Identifying and prioritizing detection research leads is out of scope for this post but for now let’s assume we have an attack we would like to detect.
Understand the attack
If we do not know how and why the attack works we cannot build a reliable detection. Also, understanding the attack is fun!
1. What is the goal of the attack technique?
This is where you identify the specific reasons for an adversary to perform the attack. There could be many depending on the attack.
2. Why does this technique work?
This is where you identify specifically why this attack works. It could be due to a specific configuration setting, vulnerability or software feature. This analysis can shed light into how to prevent this attack too.
3. At what phase of the attack would this technique be leveraged?
This is excellent background so we know when an adversary would attempt to use this technique from their toolkit.
4. What artifacts would an adversary require to perform this attack?
An adversary requires information to perform attacks. That could be user credentials, computer certificates or even just a target IP address. Knowing this is critical when understanding how to detect their attacks but also knowing what they are capable of at certain points during an incident. This research on the attack and our own environment will become the basis of the documentation that will help educate others quickly if this alert were to trigger.
5. Does this attack work in our organization?
It’s important to ask this since if the attack doesn’t work within our organization then we can possibly be done here and focus on higher priorities.
For example if this attack requires MacOS and we don’t have any then well there are probably better things we can do with your time.
However, in some cases it may still be valuable to build a detection. In the cases where it is a no, would an adversary realize the attack will not work and still attempt it? An example could be AS-REP Roasting. Your organization may not have any AS-REP roastable accounts but if an attacker kicks off a noisy scan you may still be able to detect them. The ideal situation is that an attack fails and the attacker is detected. They don’t always have to be caught trying something that actually works!
Understand normal system and user behavior
Understanding normal is the most overlooked component of detection engineering. In fact I believe it’s the key to being a competent Detection Engineer. Life is chaotic keeping up with every attack ever and not all attack chains are publicly shared (think well funded nation states). But if you know your own environment and what normal looks like (in telemetry) then you can recognize when something is out of place. Your spidey sense will be like I don’t know what this is and that’s a concern. Then you will dig in to determine what you’re looking at. In some cases if it turns out to be nothing then you’ve just learned even more about your own environment which is always a positive.
When building detections in an isolated lab it’s not too difficult to find attacker behavior. What’s tricky is isolating that behavior from normal, or even unusual but benign user behavior in a functioning enterprise environment. To add to the complexity there will generally be data gaps within the enterprise versus a lab. To successfully build a robust detection that’s difficult to evade we must understand what normal looks like.
1. How is this service used normally? What is it used for?
Its always important to know the “boring” stuff. Which is why does this service exist, how is it intended to be used and how is it used within my environment since that can be different too.
2. Is this “exploited” feature needed in the enterprise? And why?
In some cases if a feature isn’t needed at all we can disable it. Sadly, that’s not too common but it’s a good question to ask. If at this point you realize this feature isn’t needed at all then that is covered in the “Is prevention possible” section.
3. Is this behavior outside of a standard process?
When learning what is normal it’s important to know when this activity occurs within the enterprise. A good example would be user account creation. Is there a standard process for all account creation in the enterprise? Generally, its never this simple but the goal would be to know all of the ways some action occurs so you can say anything outside of this process is either not someone following enterprise standards or it is in fact malicious. This creates process choke points attackers have to go through to potentially remain undetected. This is also a great opportunity to make pals with the Identity and Access teams. Understanding those systems and process will be very important since their mission overlaps with detection engineering!
4. Do administrators or power users perform this operation?
It’s important to think about who is authorized to do this. If it’s a small subset of the environment those are very useful details when building a detection or preventing this activity outright.
Visibility into the attack
To detect an attack we’d require visibility from a logging source. In some cases an attack will only be visible in a lab environment and not at scale in an enterprise. We do want to focus on telemetry that is accessible within an enterprise but identifying artifacts left behind (that we cannot feasibly collect at scale) is still valuable for learning or during an incident where forensic artifacts can be grabbed manually.
1. Which steps from the attack can we see within telemetry?
From part 1 (Understanding the attack) and part 2 (Understanding normal) we should have a good high level understanding on what occurs during the attack so next we can make a list of each step to attempt to identify that within the telemetry. The process diagram shows arrows going between each of the 3 parts (See picture below). During complex attacks you’ll be going back and forth learning more about the attack, then whats normal, then back to visibility and then back again until you fully understand all parts.
Keep revisiting each step to fully understand all partsThis step may also depend on what testing environments you have access to. Ideally, you can perform it in a lab environment with all logging (even more than within your organization) enabled. Then you’ll have a full picture and understanding of what occurred. At some point it would be valuable to perform the attack in the enterprise but understanding what happens in a lab would allow for you to learn the attack prior to jumping into the deep end. As we had discussed before the enterprise will have data gaps and more un-explainables to deal with. It’ll be important to understand all of this but it’s nice to not have to drink from the fire hose immediately. In addition to that it generally requires approvals and assets to test attacks in an enterprise environment which adds to the preparation time.
2. Does this telemetry exist in your organization?
If you are aware of what telemetry occurs when the attack happens you can go looking in your log analytics platform to see if it’s there and see whats missing.
- Can it be acquired at scale?
- Can it be acquired locally?
If we cannot query the logs for a detection then we cannot build a detection which is pretty straight forward. If the logs are not available it’s important to consider how they could be acquired. This research to detect an attack could be the evidence needed to get these logs ingested into a log analytics platform.
Let’s take an extreme example which would be Operating System API calls. It is highly unlikely to be able to acquire these at scale. Even monitoring them locally can cause a system to grind to a halt. Active Directory events for example are noisy but could still be justified to collect with how valuable they are.
Powershell Events would be expensive to log centrally for large organizations however if your research produces some good artifacts that would appear in powershell events there could be a push to get them enabled across the organization so they could be used if there was an incident. They can be logged and sit on an endpoint if needed. They would not be retained forever but still its better than not logging them at all.
Is prevention possible?
Detections are wonderful but stopping the attack path is better! Detections cannot be the solution for everything or we will drown in alerts. Pushing for prevention can be a long term plan but it’s important to dig into this angle as well. Enterprises have many prevention tools at their disposal so it’s very important to understand how they work and how to leverage them.
A good example of this is running a browser in debug mode. The general user population would not need to do this ever and this feature is required to steal session cookies from the browser. In this case we could check endpoint telemetry to determine if this is used within the enterprise (or Enterprise Security Standards ). It’s unlikely that its needed for the vast majority of the enterprise so this could possibly be a good candidate to disable via GPO. Anyone who needs it for testing can request it and then you’ve significantly reduced the attack surface.
This is an oversimplification of this attack but for more details refer to this awesome SpecterOps blog post.
Isolate malicious behavior
Now that we have all the pieces we can attempt to isolate the malicious behavior.
1. Is this a rare event itself?
If this is rare then this is a simpler one, we can just develop the alert looking for the rare events.
2. Is there a rare sequence of events that occurs?
This is more advanced and how we can really reduce benign events. Looking at the telemetry identify multiple rare-ish steps that’d need to occur for this attack to be successful. This would require a way to track the sequence of events. Some log analytics platforms have methods to do so. This is how you would do that with Splunk.
3. Are there multiple suspicious events that could be correlated regardless of order?
My previous post about the detection pipeline is relevant here since a risk engine would be a method to correlate and implement a risk score for multiple suspicious events. These would be events that are suspicious but the sequence does not matter.
3. Can statistics be used?
Let’s take a simple brute force password spray. We can look for a high failure ratio from a specific IP. The failure ratio is important since aggregators or morning logins by users can create spikes in fails but when looking over the activity window it really isn’t anything spectacular.
- 300 login failures, 800 login successes over a 10min time span which equates to a 27.3% failure rate— nothing special, I forget my passwords all the time too
- 300 login failures, 1 login success over 10min time span which equates to a 99.6% failure rate — this is more suspicious
4. How resilient is this detection to evasion?
This is important to understand so we can communicate how effective a detection is. As long as everyone is on the same page for the fidelity of the detection (and even better it’s tracked within Detection Metrics) then we are good. The goal is to be at the top of the pyramid but depending on the situation that isn’t always the best approach.
Let’s say we have an incident with malware in our environment and the malware uses a unique filename/location combination. We could create an alert temporarily for that pretty quickly. We would not want to tell leadership we’ll be able to Reverse Engineer the binary in 6 months then build the best detection ever. We’d first implement the basic detection to aid in response then work on the more permanent detection.
Even if we build the best detection feasible it’ll be important to understand evasion. When a detection is evade-able it may still be valuable and would still raise the bar for attackers.
Document
After all this experimenting and simulating it’ll be critical to document all of this for others to learn from and build upon. At the end of this research journey it may not even be feasible to isolate the malicious activity so a detection would not be built. In those scenarios especially it’s important to document all our research and the insights gained. A detection opportunity may come to mind later or by another teammate as the environment matures. For documentation this format can be used as well.
- Attack Details
- Attack Telemetry
- Service Background and Usage
- Prevention opportunities (explored)
- Malicious Activity
- Detection Gaps
Development
The final step if a detection concept is identified would be to proceed to development!
If you’ve made it this far I have walked through these steps in the companion post developing a Remote DCSync detection here.
Sources & Additional Reading
https://www.splunk.com/en_us/blog/security/risk-based-alerting-splunk-sequenced-events.html
https://specterops.io/blog/2025/08/27/dough-no-revisiting-cookie-theft/
https://attack.mitre.org/techniques/T1558/004/
http://www.rohitab.com/apimonitor
A Detection Researcher Mindset was originally published in Detect FYI on Medium, where people are continuing the conversation by highlighting and responding to this story.
Introduction to Malware Binary Triage (IMBT) Course
Looking to level up your skills? Get 10% off using coupon code: MWNEWS10 for any flavor.
Enroll Now and Save 10%: Coupon Code MWNEWS10
Note: Affiliate link – your enrollment helps support this platform at no extra cost to you.
Article Link: https://detect.fyi/a-detection-researcher-mindset-f2ed045480c5?source=rss----d5fd8f494f6a---4