As the landscape of cybersecurity threats evolves, protecting sensitive information stored within enterprise platforms like Microsoft 365 (M365) has become a top priority for IT and security teams. To help organizations identify and mitigate these risks, Microsoft provides a range of built-in detection mechanisms based on user activity and sign-in behavior analysis. While these tools can offer significant insights, it’s important to understand their limitations, potential false positives, and how to effectively investigate suspicious events.
-------------------------------------------------------------------------------------------------------------
Built-In Reports: Monitoring Risky Activity
Microsoft 365's built-in reporting suite provides several out-of-the-box detection features that monitor risky user behavior and sign-in activity. These include:
Risky sign-ins: Sign-ins flagged as risky due to factors like unusual IP addresses, impossible travel, or logins from unfamiliar devices.
Risky users: User accounts exhibiting abnormal behavior, such as frequent failed login attempts or multiple sign-ins from different geographies.
Risk detections: A general term referring to any identified behavior or event that deviates from normal patterns and triggers a system alert.
These alerts are largely powered by machine learning and heuristic algorithms that analyze stored log data to identify abnormal behavior patterns. The system is designed to recognize potential security risks, but it does have some caveats.
-------------------------------------------------------------------------------------------------------------
Built-In Risk Detection: Delays and False Positives
One of the most important things to understand about Microsoft’s risk detection mechanisms is that they are not instantaneous. Alerts can take up to 8 hours to be generated, meaning there is a delay between the detection of a suspicious event and the time it takes for the alert to surface. This delay is designed to allow the system to analyze events over time and avoid triggering unnecessary alerts, but it also means that organizations may not be alerted to security incidents immediately.
Another challenge is that these alerts can sometimes generate false positives. A common example is the geolocation module and its associated “impossible travel” alert. This is triggered when a user signs in from two geographically distant locations within a short time, which would be impossible under normal circumstances. However, the issue often arises from incorrect IP location data, such as when users connect to the internet via hotel networks, airplane Wi-Fi, or mobile carriers. For instance, if a user switches from airplane internet to airport Wi-Fi, the system may mistakenly flag it as an impossible travel scenario, even though the user hasn’t changed locations.
Managing False Positives
Because these false positives can clutter security dashboards, it’s important for IT teams to review and refine their alerting thresholds. Regular tuning of the system and awareness of typical user behaviors—such as frequent travelers—can help minimize the noise created by these alerts and focus on genuine threats.
-------------------------------------------------------------------------------------------------------------
Investigating and Profiling Logons
When a suspicious event is detected, one of the first steps in investigating the issue is analyzing logon data. Microsoft’s Unified Audit Logs (UAL) track over 100 types of events, including both successful and unsuccessful login attempts. Here are some key strategies for analyzing logons and identifying potential security breaches:
Tracking Successful Logins
Every successful login generates a UserLoggedIn event, which includes valuable information such as the source IP address. Investigators can use this data to identify unusual logon behavior, such as logins from unexpected geographical locations or times. Temporal or geographic outliers—such as a login from a country the user has never visited—can be red flags that warrant further investigation.
Additionally, a pattern of failed logon attempts (logged as UserLoginFailed events) followed by a successful login from a different or suspicious IP address may suggest that an attacker was trying to brute-force or guess the user’s password before successfully logging in.
Investigating Brute-Force Attacks
Brute-force attacks—where an attacker attempts to gain access by repeatedly guessing the user's credentials—leave distinctive traces in the log data. One common sign of a brute-force attack is when a user gets locked out of their account after multiple failed login attempts. In this case, you would see a sequence of UserLoginFailed events followed by a “IdsLocked” event, indicating that the account was temporarily disabled due to too many failed attempts.
Further, even if the user account doesn’t exist, the system will log the attempt with the term UserKey=“Not Available”, which can help identify instances of user enumeration—a technique used by attackers to discover valid usernames by testing different variations.
-------------------------------------------------------------------------------------------------------------
Investigating MFA-Related Events
When multi-factor authentication (MFA) is enabled, additional events are logged during the authentication process. For example:
UserStrongAuthClientAuthNRequired: Logged when a user successfully enters their username and password but is then prompted to complete MFA.
UserStrongAuthClientAuthNRequiredInterrupt: Logged if the user cancels the login attempt after being asked for the MFA token.
These events are particularly useful in detecting attempts by attackers to bypass MFA. If you notice a sudden increase in UserStrongAuthClientAuthNRequiredInterrupt events, it could indicate that attackers have obtained passwords from a compromised database and are testing accounts to find those without MFA enabled.
-------------------------------------------------------------------------------------------------------------
Investigating Mailbox Access and Delegation
Attackers who gain access to a Microsoft 365 environment often target email accounts, particularly those of key personnel. Once inside, they may attempt to read emails or set up forwarding rules to siphon off sensitive information. One tactic is to use delegate access, where one account is granted permission to access another user’s mailbox.
Delegate access is logged in UAL, and reviewing these logs can reveal when permissions are assigned or when a delegated mailbox is accessed. In addition, organizations should regularly audit their user lists to check for unauthorized accounts that may have been created by attackers. In many cases, such unauthorized users are only discovered during license reviews.
Another avenue for attackers is server-side forwarding, which can be set up through either a Transport Rule or an Inbox Rule. These forwarding mechanisms can be used to exfiltrate data, so security teams should regularly review the organization’s forwarding rules to ensure no unauthorized forwarding is taking place.
-------------------------------------------------------------------------------------------------------------
External Applications and Consent Monitoring
Microsoft 365 users can grant third-party applications access to their accounts, which poses a potential security risk. Once access is granted, the application doesn’t need further permission to interact with the account. Monitoring for the Consent to application event can help organizations detect when external applications are being granted access, particularly if the organization doesn’t typically use external apps. This was a factor in the well-documented SANS breach in 2020, where attackers exploited third-party app permissions to gain access to a user’s mailbox.
-------------------------------------------------------------------------------------------------------------
Conclusion
While Microsoft 365 offers powerful built-in tools for detecting risky behavior and investigating suspicious logon events, security teams must be aware of their limitations, particularly the potential for false positives and delayed alerts. By regularly reviewing log data, investigating unusual patterns, and keeping an eye on key events like failed login attempts, MFA interruptions, and delegation changes, organizations can better protect their environments against evolving threats. The key to effective security monitoring is a proactive approach, combining automated detection with human analysis to sift through the noise and focus on genuine risks.
Akash Patel