In cloud-based environments like Azure, maintaining comprehensive visibility over all activities is essential for securing your infrastructure and responding effectively to incidents. One of the most critical tools in your security arsenal is logging. Azure provides a variety of log sources, but not all are enabled by default. Understanding where these logs come from, how to access them, and how to store them can significantly improve your ability to investigate incidents and mitigate risks.
The Five Key Azure Log Sources
Azure collects logs from various levels of the cloud infrastructure, each serving a unique role in monitoring and security. Here are the five primary log sources you need to be aware of:
Tenant Logs
Subscription Logs
Resource Logs
Operating System Logs
Application Logs
Let’s explore each of these in more detail.
Tenant | Turned on by default | Used to detect password spray attacks or other credentian abuses. |
Subscription | Turned on by default | Used to analyze the creation, deletion, start/stop of resources in cases such as crypto mining VM incidents or mass deletion for sabotage cases. |
Resource | Turned off by default | Used to log network traffic flow, file storage access for cases such as data exfiltration. |
Operating System | Turned off by default | Used to log operating system events, which can show lateral movement. |
Application | Turned off by default | Used to create custom logs at the discretion of developers. Azure includes a log for IIS that can be used to show web servers attacks. |
-------------------------------------------------------------------------------------------------------------
Why Proper Logging Matters in Incident Response
In many cases, when an organization is called to respond to a security incident, the first challenge is discovering that key logs were never configured or stored. This leaves responders with limited information and hampers their ability to fully understand the attack.
Why is this important?
Comprehensive Monitoring: Many log sources, such as resource and OS logs, must be enabled manually. Without these logs, crucial events like unauthorized access or file manipulation might go unnoticed.
Cost of Storage: Logs must be stored in Azure, often in a Log Analytics Workspace or similar storage solution, which incurs additional costs. Without proper budgeting and planning, organizations might avoid enabling these logs due to perceived costs, leaving them vulnerable.
Log Retention: Depending on your configuration, logs might only be stored for a short period before being overwritten. Having a strategy in place for exporting and storing logs in a secure, centralized location (such as a SIEM system) is essential.
The ideal setup is to continuously export these logs to a SIEM, where they can be stored long-term and analyzed even after an incident has occurred. This prevents attackers from covering their tracks by deleting logs stored locally in Azure.
-------------------------------------------------------------------------------------------------------------
Log Analytics Workspace: Centralizing Your Logs for Efficient Analysis
Azure provides a Log Analytics Workspace as a centralized repository where logs from multiple sources, both Azure-based and non-Azure, can be aggregated and analyzed. This workspace organizes logs into tables, with each data source creating its own table.
Key benefits of using a Log Analytics Workspace include:
Scalability: The default workspace can handle up to 6GB of logs per minute and store up to 4TB of data per day. This is generally sufficient for most organizations, though custom workspaces can be created for larger log volumes.
Access Control: You can set granular permissions based on security roles, ensuring that sensitive logs are only accessible to authorized personnel.
By setting up a Log Analytics Workspace, you can automate the collection of logs from all relevant sources and integrate with Azure Monitor for real-time alerting and analysis.
-------------------------------------------------------------------------------------------------------------
Setting Up Log Analytics Workspace in Azure
A Log Analytics Workspace allows you to aggregate logs from multiple Azure services and third-party tools into one place. Here’s how to set it up:
Step-by-Step Guide to Creating a Log Analytics Workspace
Step 1: Sign in to the Azure Portal
Go to Azure Portal and sign in with your credentials.
Step 2: Search for 'Log Analytics Workspaces'
In the search bar at the top, type Log Analytics Workspaces and select the service from the list.
Step 3: Create a New Workspace
Click New to create a new workspace.
Enter the required details:
Subscription: Select your Azure subscription.
Resource Group: Choose an existing resource group or create a new one.
Workspace Name: Name your workspace (e.g., "SecurityLogsWorkspace").
Region: Choose the region where you want the workspace to reside.
Step 4: Review and Create
After entering all details, click Review + Create and then Create to deploy your Log Analytics Workspace.
This workspace will serve as a centralized location for all logs, which can be expanded to include tenant logs, subscription logs, resource logs, and more.
For more details on creating a Log Analytics workspace, visit Microsoft’s official documentation.
-------------------------------------------------------------------------------------------------------------
Tenant Logs: Overview and Access
Tenant logs provide information about operations conducted by tenant-wide services like Azure Active Directory (AAD)(Entra ID). These logs are essential for monitoring security-related events such as sign-ins, user provisioning, and audit trails.
The key AAD logs include:
Audit Logs: Track changes and configuration updates across the tenant.
Sign-in Logs: Provide detailed records of user login activity, including success, failure, and multi-factor authentication (MFA) usage.
Viewing Tenant Logs in the Azure Portal
Sign-in Logs
To quickly check sign-in activity, go to the Azure Portal and navigate to Azure Active Directory(Entra ID) > Sign-ins.
Here you can view sign-in logs for the last 30 days, showing details such as user, date, status (success, failure, interrupted), and the IP address used.
https://learn.microsoft.com/en-us/entra/fundamentals/how-to-manage-user-profile-info
Audit Logs
Similarly, go to Azure Active Directory(Entra ID) > Audit Logs to see tenant-wide changes, such as user account updates and administrative configuration changes.
However, the Azure portal limits logs to the last 30 days, making it unsuitable for long-term
forensic analysis or detailed investigations.
For comprehensive analysis and historical data retention, storing logs in a Log Analytics Workspace is a much better approach.
-------------------------------------------------------------------------------------------------------------
Exporting Azure Active Directory Logs to Log Analytics Workspace (Now AAD name have been modified to Entra ID)
To take full advantage of tenant logs, including AAD audit and sign-in logs, you should configure the logs to be stored in your Log Analytics Workspace. This allows for extended retention periods, deeper analysis, and cross-correlation with other logs.
Step-by-Step Guide to Exporting AAD Logs(Entra ID logs)
Step 1: Navigate to Azure Active Directory(Entra ID logs)
In the Azure Portal, search for and select Azure Active Directory from the services list.
Step 2: Configure Diagnostic Settings
From the AAD menu, select Diagnostic settings.
Click Add diagnostic setting to configure where the logs will be stored.
-------------------------------------------------------------------------------------------------------------
Selecting AAD Logs(Entra ID) and Setting Up Log Analytics Workspace
After setting up your Log Analytics Workspace (as described in previous steps), the next task is to configure which AAD logs you want to capture and send to the workspace. Azure provides several types of logs that you can export for analysis:
Audit Logs: Logs changes such as adding or removing users, groups, roles, policies, and applications.
Sign-in Logs: Tracks sign-in activities, including:
User sign-in: Captures direct user login events.
Non-interactive sign-in: Logs background sign-ins, such as token refreshes.
Service Principal sign-in: Logs sign-ins performed by service principals (used by applications).
Managed Identity sign-in: Captures sign-ins for managed identities.
Provisioning Logs: Tracks user, group, and role provisioning activities performed by Azure AD.
ADFS Sign-in Logs: Monitors federation sign-in events through Active Directory Federation Services (ADFS).
Identity Protection Logs: Tracks risky users and events, including RiskyUsers, UserRiskEvents, RiskyServicePrincipals, and ServicePrincipalRiskEvents.
Network Access Traffic Logs: Logs network traffic for policy and risk management, including user experience data.
To set this up:
Navigate to Diagnostic Settings:
Go to the Azure Active Directory (Entra ID) service in the Azure portal.
In the left menu, click Diagnostic settings and then select Add diagnostic setting.
Choose Logs to Export:
Select the categories of logs you want to export to the Log Analytics Workspace (e.g., AuditLogs, SignInLogs, ProvisioningLogs).
Specify the Log Analytics Workspace where these logs will be stored.
Save Settings:
Confirm the logs you’ve selected and save the diagnostic setting. Once configured, these logs will be automatically sent to the designated Log Analytics Workspace for long-term storage and analysis.
-------------------------------------------------------------------------------------------------------------
Managing Storage Costs
While it may be tempting to store all available logs, storage costs can accumulate quickly, especially for large organizations with a lot of activity. One cost-saving measure is to use Azure Storage Accounts for logs that don't require constant querying but need to be archived for compliance or later use.
For critical logs, such as sign-in and audit logs, continuous export to the Log Analytics Workspace is recommended for monitoring real-time activity and performing incident response. However, less frequently accessed logs can be stored more cost-effectively in a storage account.
-------------------------------------------------------------------------------------------------------------
Querying AAD Logs(Entra ID logs) Using Kusto Query Language (KQL)
Once AAD logs are flowing into your Log Analytics Workspace, you can use Kusto Query Language (KQL) to search, filter, and analyze log data. KQL is a powerful language for querying logs and has a syntax similar to SQL, making it approachable for those familiar with databases.
Example of a Simple KQL Query:
SigninLogs
| where TimeGenerated > ago(1d)
| where ResultType == 0SigninLogs: The first line specifies the log type you want to search.
TimeGenerated > ago(1d): Filters the query to only include logs from the past 24 hours.
ResultType == 0: This line filters for successful logins (ResultType 0 corresponds to success).
This simple query helps you identify all successful sign-in attempts in the last 24 hours. KQL also allows for more complex queries involving joins, aggregations, and visualizations, making it a robust tool for analyzing log data.
For more details on KQL, visit Microsoft’s KQL Documentation.
-------------------------------------------------------------------------------------------------------------
Using Pre-Built Queries in Log Analytics
Microsoft also provides a set of pre-built queries for common scenarios, such as analyzing sign-ins, audit events, or identifying risky behavior in your tenant. These queries serve as templates, which you can customize based on your specific investigation needs.
Pre-Built Queries: These are particularly useful when first starting with KQL, as they provide a foundation for your own queries and ensure you're asking the right questions of your data.
To use these pre-built queries:
Open your Log Analytics Workspace in the Azure portal.
Navigate to the Logs section.
Search for the desired query in the query library, or start with a template and adjust it to suit your needs.
-------------------------------------------------------------------------------------------------------------
Further we will talk in next blog, Until than stay safe and keep learning
Akash Patel
Opmerkingen