top of page

Identifying Malicious Software: A Guide for Incident Responders

3 days ago6 min read


One of the most critical challenges faced by incident responders is the rapid identification of suspicious and malicious software on a computer system. With thousands of files and folders present, security professionals must employ various techniques to detect threats efficiently. This process involves analyzing files, performing live response (examining a system while it's running), and conducting temporal analysis (also known as timelining) to trace malicious activity.


However, attackers constantly evolve their techniques to evade detection.

Some common methods are:


  • Deleting system logs and file metadata to erase traces of unauthorized access.

  • Obfuscating malware by renaming files or modifying execution behaviors to mimic legitimate processes.

  • Using packed executables—these files encapsulate malicious code in an encrypted or obfuscated format to evade detection.

  • Implementing custom encryption techniques to prevent analysis by traditional security tools.

To counter these techniques, incident responders can leverage

powerful tools like Sigcheck, file entropy YARA, Maldump, and Capa.

These tools facilitate rapid triage and analysis of suspicious files, helping to determine whether further investigation or reverse engineering is necessary.


-------------------------------------------------------------------------------------------------------------


Sigcheck: Analyzing Code Signing and Anomalies

Sigcheck is a widely used tool from Microsoft's Sysinternals suite designed to analyze executable files, verify digital signatures, and detect anomalies. It helps analysts quickly identify unsigned binaries, which are often associated with malware.


Why is Code Signing Important?

Most legitimate software is digitally signed, ensuring its authenticity and integrity. However, malware authors sometimes obtain stolen or fraudulent certificates to sign their malicious programs.


For example, in 2022, malicious drivers were discovered using stolen NVIDIA certificates.

Key Features of Sigcheck:

  • Identify Unsigned Binaries: Using the -u and -s options, analysts can recursively scan directories for unsigned executables.

  • Check for Suspicious Digital Signatures: Even signed malware can be detected if issued by an unfamiliar or untrusted entity.

  • Integration with VirusTotal: Sigcheck can perform hash lookups to check if a file has been flagged as malicious by security vendors.

  • Entropy Analysis: High entropy values often indicate packed or encrypted malware. Sigcheck uses Shannon's entropy scale (0 to 8) to assess randomness within files.


Example Usage:

E:\Malware discovery tool\Sigcheck>sigcheck.exe -a -c -e -h C:\Windows > Sigcheck.csv

OR

E:\Malware discovery tool\Sigcheck>sigcheck.exe -a -c -h C:\Windows >output.csv

This command scans the Windows directory for unsigned files, potentially revealing malware attempting to blend in with legitimate system files.



-------------------------------------------------------------------------------------------------------------

File Entropy

• Windows system executables average score: 4 - 6

• Packed or evasive malware average score: 6 - 8



A few things to keep in mind:

  1. Legitimate files tend to have an entropy between 4.8 and 7.2.

  2. Files with an entropy above 7.2 tend to be malicious.

  3. Nearly 30% of all of the malicious samples have an entropy near 8.0 while only 1% of legitimate samples have an entropy near 8.0.

  4. Approximately 55% of all malicious samples have a entropy of 7.2 or more versus 8% of legitimate samples.


Now if you don't want to download extra tool, you can use sigcheck itself (this tool also give entropy score

-------------------------------------------------------------------------------------------------------------


YARA: Signature-Based Detection for Malware Classification

YARA is one of the most effective tools for identifying malware based on patterns and signatures. Originally developed by VirusTotal, it has become a standard for cybersecurity professionals.


How YARA Works:

YARA rules define patterns that malware analysts use to detect threats. These patterns can be based on strings, regular expressions, or more advanced conditions.


Example YARA Rule:

rule SuspiciousPE {
    strings:
        $mz = "MZ"
        $string1 = "malicious_code"
        $string2 = "cmd.exe"
    condition:
        $mz at 0 and 2 of ($string*)
}

This rule detects executable files (MZ signature) that contain suspicious strings related to command execution.


Benefits of YARA:

  • Scalability: YARA rules can be applied across multiple systems for quick detection.

  • Flexibility: Supports regex and logical conditions for precise malware identification.

  • Automation: Can be integrated with security solutions for real-time monitoring.


Example Usage:

yara -r rules.yar C:\SuspiciousFolder
      or
E:\Malware discovery tool\yara-v4.5.2-2326-win64>yara64.exe yara\apt_apt12_malware.yar "C:\Users\Akash's\Downloads"

This command scans the folder against predefined YARA rules to detect potential threats.


-------------------------------------------------------------------------------------------------------------


Understanding Antivirus Quarantine Mechanisms

When an antivirus detects a suspicious file, it moves it to a quarantine folder. This prevents the file from executing, thereby neutralizing any potential threats. Different antivirus vendors implement their own formats and security measures to make these quarantined files inaccessible or encrypted. If you have access to the live system, the easiest way to recover these files is by using the antivirus software’s built-in recovery options. However, if you are working with a disk image or a copied folder, things can get trickier.


Security researchers have reverse-engineered many antivirus quarantine formats and developed tools to extract these files.
Extracts:

  • Avast Antivirus

  • Avira Antivirus

  • Eset NOD32

  • FortiClient

  • G Data

  • Kaspersky for Windows Server

  • Malwarebytes

  • Microsoft Defender


This is where maldump comes in handy.

Introducing maldump: A Powerful Open-Source Tool

maldump is a Python-based open-source tool designed to identify and extract quarantined files from various antivirus solutions. One of its standout features is its ability to scan a mounted disk image (such as a forensic triage image) and detect quarantined files across multiple vendors.


Key Features of maldump

  • Automatic Detection: It scans known vendor folder locations and identifies quarantined files.

  • Metadata Extraction: Provides valuable details like quarantine timestamps, file hashes, and original file names.

  • File Recovery: Extracts quarantined files for further analysis.

  • Compatibility: Works with both live file systems and mounted forensic images.


How to Use maldump

To get started, you need Python installed on your system. maldump is pre-installed, but if you need to install it manually, refer to its GitHub repository.


Basic Commands:

  • Identify Quarantine Files:

maldump -l /path/to/mounted/image

This command lists all quarantined files found in the specified directory.


  • Extract Metadata:

maldump -m /path/to/mounted/image -o metadata.csv

This saves metadata (hashes, timestamps, etc.) in a CSV file for further investigation.


  • Recover Quarantined Files:

maldump -q /path/to/mounted/image -o extracted_files/

This extracts all identified quarantine files to a specified folder.


  • Full Scan and Extraction:

maldump -a /path/to/mounted/image -o full_output/

This command performs a complete scan, extracts metadata, and recovers all quarantined files.



Important Considerations

  • Permissions: Running maldump with administrator privileges ensures access to all necessary files.

  • Folder Structure: If you only have a quarantine folder, you may need to recreate the expected directory structure for proper extraction.

  • Log Collection: Always collect logs and quarantine folders when performing forensic investigations to avoid missing critical data.


-------------------------------------------------------------------------------------------------------------


Using capa for Deeper Malware Analysis

Once you have extracted a quarantined file, the next step is to analyze its behavior.


This is where capa, an open-source malware analysis tool developed by FireEye FLARE, comes into play.


Why Use capa?

Malware analysis requires advanced skills, but capa simplifies the process by automatically identifying known patterns and techniques used in malicious software. It disassembles the file, compares it against a set of predefined rules, and generates a human-readable report.


Key Features of capa

  • Automated Static Analysis: Analyzes a binary file without executing it.

  • Comprehensive Rule Set: Uses a crowd-sourced database of malware characteristics.

  • ATT&CK Mapping: Maps detected techniques to MITRE ATT&CK for better understanding of attacker behavior.

  • Multiple Formats Supported: Works with executables, shellcode, and memory dumps.


How to Use capa

  • Basic Analysis of an Executable

E:\Malware discovery tool\capa-v9.0.0-windows>capa.exe "E:\Malware discovery tool\maldump-0.5.0\maldump\maldump.exe"

This provides an overview of the malware’s capabilities.


  • Verbose Mode for In-Depth Analysis


E:\Malware discovery tool\capa-v9.0.0-windows>capa.exe -v "E:\Malware discovery tool\maldump-0.5.0\maldump\maldump.exe"

This displays detailed rule matches within the code.


  • Shellcode Analysis

E:\Malware discovery tool\capa-v9.0.0-windows>capa.exe -f sc64  "E:\Malware discovery tool\maldump-0.5.0\maldump\maldump.exe"

This analyzes 64-bit shellcode, useful for investigating in-memory threats.



  • Export Results in JSON

E:\Malware discovery tool\capa-v9.0.0-windows>capa.exe -j "E:\Malware discovery tool\maldump-0.5.0\maldump\maldump.exe" > out.jsonson

This outputs the results in JSON format for further processing.



Output:

Understanding capa Output

The results include:

  • Capabilities (e.g., process injection, file modification, registry manipulation)

  • Malware Behavior Catalog (MBC) and ATT&CK technique references

  • Specific code locations matching known malware patterns


One key takeaway is that capa helps identify potential threats but does not provide absolute conclusions.
For example, legitimate tools like PowerShell may exhibit the same capabilities as malware. Therefore, capa’s results should be correlated with other forensic findings.

-------------------------------------------------------------------------------------------------------------


Conclusion

Mastering the extraction and analysis of quarantined antivirus files is an essential skill for malware investigators and cybersecurity professionals. With maldump, you can quickly locate and recover quarantined files from different antivirus solutions. Pairing it with capa enables deeper malware analysis, helping you understand the behavior and potential threats posed by extracted files.


--------------------------------------------------Dean---------------------------------------------------



 
 
 

Comments

bottom of page