When it comes to investigating cloud environments, having the right tools can save a lot of time and effort.
Today, I’ll introduce two free, powerful tools that are absolutely fantastic for log analysis within the Microsoft cloud ecosystem: Microsoft-Extractor-Suite and Microsoft-Analyzer-Suite. These tools are easy to use, flexible, and can produce output in accessible formats like CSV and Excel, making them excellent resources for investigating business email compromises, cloud environment audits, and more.
About Microsoft-Extractor-Suite
The Microsoft-Extractor-Suite is an actively maintained PowerShell tool designed to streamline data collection from Microsoft environments, including Microsoft 365 and Azure. This toolkit provides a convenient way to gather logs and other key information for forensic analysis and cybersecurity investigations.
Supported Microsoft Data Sources
Microsoft-Extractor-Suite can pull data from numerous sources, including:
Unified Audit Log
Admin Audit Log
Mailbox Audit Log
Azure AD Sign-In Logs
Azure Activity Logs
Conditional Access Policies
MFA Status for Users
Registered OAuth Applications
This range allows investigators to get a comprehensive picture of what’s happening across an organization’s cloud resources.
----------------------------------------------------------------------------------------------------------
Installation and Setup
To get started, you’ll need to install the tool and its dependencies. Here’s a step-by-step guide:
Install Microsoft-Extractor-Suite:
Install-Module -Name Microsoft-Extractor-Suite
Install the PowerShell module Microsoft.Graph (for Graph API Beta functionalities):
Install-Module -Name Microsoft.GraphInstall ExchangeOnlineManagement (for Microsoft 365 functionalities):
Install-Module -Name ExchangeOnlineManagementInstall the Az module (for Azure Activity log functionality):
Install-Module -Name AzInstall the AzureADPreview module (for Azure Active Directory functionalities):
Install-Module -Name AzureADPreviewOnce the modules are installed, you can import them using:
Import-Module .\Microsoft-Extractor-Suite.psd1----------------------------------------------------------------------------------------------------------
Note: You will need to sign in to Microsoft 365 or Azure with appropriate permissions(Admin level access, included P1 or higher access level, or an E3/E5 license) before using Microsoft-Extractor-Suite functions.
----------------------------------------------------------------------------------------------------------
Getting Started
First, connect to your Microsoft 365 and Azure environments:
Connect-M365
Connect-Azure
Connect-AzureAZFrom here, you can specify start and end dates, user details, and other parameters to narrow down which logs to collect. The tool captures output in Excel format by default, stored in a designated output folder.
----------------------------------------------------------------------------------------------------------
Example Log I collected:
One drawback to keep in mind is that logs are collected one by one. example first u collect MFA logs second again you written command and collected Users log.
Another thing to keep in mind is if u do not provide path output will be capture under default folder where script is present.
----------------------------------------------------------------------------------------------------------
You might have question why two different suite?
Answer is because there is script name Microsoft-Analyzer-Suite developed by evild3ad. This suite offers a collection of PowerShell scripts specifically designed for analyzing Microsoft 365 and Microsoft Entra ID data, which can be extracted using the Microsoft-Extractor-Suite.
Current Analysis support by Microsoft-Analyzer-Suite is:
----------------------------------------------------------------------------------------------------------
Before I start, I will show you folder structure of both the tools:
Microsoft-Extractor-Suite
Microsoft-Analyzer-Suite-main
Analyzer-Suit allows You can also add specific IP addresses, ASNs, or applications to a whitelist by editing the whitelist folder in the Microsoft-Analyzer-Suite directory.
------------------------------------------------------------------------------------------------------------
Lets start:
I will show you two logs capture and analyzed is message trace log other one Unified audit log all collect using the script Microsoft extractor suite and than I will use Microsoft-Analyzer-Suite.
Collecting Logs with Microsoft-Extractor-Suite
Now, let’s go over collecting logs. Here’s an example command to retrieve the Unified Audit Log entries for the past 90 days for all users:
Get-UALAllAfter running this, the tool will output data in Excel format to a default folder. However, you may need to combine multiple excel file into one .csv file. Because Anlyzer suite script only run using .csv.
------------------------------------------------------------------------------------------------------------
Combining CSV Files into One Excel File
When working with large data sets, it's more efficient to combine multiple log files into a single file. Here’s how to do this in Excel:
Place all relevant CSV files in a single folder.
Open a new Excel spreadsheet and navigate to Data > Get Data > From File > From Folder.
Select the folder containing your CSV files and click “Open”.
From the Combine drop-down, choose Combine & Transform Data. This option loads your files into the Power Query Editor, where you can manipulate and arrange the data.
In the Power Query Editor, click OK to load your combined data. Edit any column formats, apply filters, or sort the data as needed.
Once done, go to Home > Close & Load
Once Done Output will be look like below:
But to ensure compatibility with Microsoft-Analyzer-Suite save the file as a .csv
Using Microsoft-Analyzer-Suite for Log Analysis
With your data collected and organized, it’s time to analyze it with Microsoft-Analyzer-Suite.
UAL-Analyzer.ps1
Before using UAL-Analyzer.ps1 script there are few dependencies u have to make sure these are installed for running script
First is creating is IPinfo account its free.
ImportExcel for Excel file handling (PowerShell Module)
Install-Module -Name ImportExcelIPinfo CLI (Standalone Binary)
xsv (Standalone Binary)
To install xsv:
Now as I had WSL (I used command git clone https://github.com/BurntSushi/xsv.git)
You can download folder (as you feel comfortable)
Once dependencies are set up, configure your IPinfo token by pasting it into the UAL-Analyzer script. To locate this in the script:
Open UAL-Analyzer.ps1 with a text editor like Notepad++, search for the token variable, and paste your token there.
Running the Analysis Script
For Unified Audit Logs, use the UAL-Analyzer script. For example:
.\UAL-Analyzer.ps1 "C:\Path\To\Your\CombinedUALLog.csv" -output "C:\Path\To\Output\"
Once script ran successfully and output collected you will get pop up
------------------------------------------------------------------------------------------------------------
Lets check the output:
As per screenshot , You can see output will be in CSV, XLSX in both format. Now question arise why there is same output in different. This is because the XLSX will contain output in coloured format, if something suspicious found it will be highlighted automatically. Where as csv will be in no highlighted format.
Example of xlsx:
Example of CSV:
Folder Suspicious Operation:
Kind note scripts are still getting updated and modified if you open GitHub you might find newer version it might work better for current this will output for me it make thing easy hope it do for you as well.
------------------------------------------------------------------------------------------------------------
Second Log we are going to talk about Message Trace logs
Command : (This will collect all logs)
Get-MessageTraceLogScreenshot of Output:
Next step is Combined all excel into one(.csv format).
Once done run MTL-Analyzer script
.\MTL-Analyzer.ps1 "C:\Path\To\Your\CombinedMTLLog.csv" -output "C:\Path\To\Output\"(Make sure before running add token details inside the script than run the script)
Conclusion
By combining Microsoft-Extractor-Suite and Microsoft-Analyzer-Suite, you can effectively streamline log collection and analysis across Microsoft 365 and Azure environments. While each suite has its own focus, together they provide an invaluable resource for incident response and cybersecurity.
Now that you have the steps, you can test and run the process on your own logs. I hope this guide makes things easier for you! See you, and take care!
Akash Patel
Comments