As Mandiant consultants, we rely on FireEye Endpoint Security and Redline to collect forensic information from endpoint artifacts, known as audit data, for investigations. Both solutions offer integrated methods to review the collected audit data. However, sometimes we need to work with the data programmatically, such as performing intelligence enrichment, frequency analysis, or extracting specific information from the data in bulk.
GoAuditParser’s primary purpose is to convert XML data from FireEye Endpoint Security bulk audit requests, triages, and Redline packages into CSV format for various use cases. In CSV format, an analyst can review the data in Microsoft Excel, easily ingest the data into a preferred review platform such as a SIEM, or use any custom tools to work with the data programmatically. Just point GoAuditParser at the audit MANS file or raw XML audit data, and it will extract and/or parse the data to CSV format quickly and efficiently!
Parsing FireEye Audits from XML format to CSV
When audit data is acquired using the aforementioned FireEye tools, it is usually in XML format. Figure 1 depicts an excerpt of a sample FireEye FileItem audit, which contains an enumeration of the MFT and INDX records, in XML format.
Figure 1: Sample FireEye XML FileItem audit
GoAuditParser parses XML audit files dynamically, meaning it does not rely on a static template for retrieving all of the available information in an audit and thus will always attempt to parse any new fields or even new types of audit data sources. As long as the audit data fits the expected XML schema of FireEye audits, it can be parsed.
While the tool has many different command line arguments, it can be run easily with the command line syntax depicted in Figure 2.
|> GoAuditParser -i path/to/input -o path/to/output|
Figure 2: Simple GoAuditParser usage syntax
After parsing XML audit files with the tool, parsed CSV audit files will be found in the output directory. Figure 3 depicts the parsed output of the sample FireEye XML audit referenced in Figure 1. Please note that some of the columns in the spreadsheets within figures throughout this blog post have been manually excluded for clarity.
Figure 3: Parsed output of a FileItem audit
FireEye Audit Timelines
A key feature of GoAuditParser is the ability to create timelines from parsed audit data. Users can add the -tl flag to the command syntax depicted in Figure 2 to create a combined timeline from all the parsed audit data. Timeline files contain entries with timestamp details, a “Summary” column (more details about this column discussed further down), and a handful of specific attribute columns including “MD5”, “Size”, “User”, etc. Figure 4 depicts a sample audit timeline with multiple audit sources such as FileItem, EventLogItem, and RegistryItem.
Figure 4: A single system’s audit timeline
The timeline “Summary” column contains the important fields for each audit entry. Important audit fields are joined into this column based on the timeline configuration file which can be customized by audit source. This process is summarized for the RegistryItem audit source in Figure 5. Documentation on GoAuditParser configuration files is available on our GitHub page.
Figure 5: Summary of the timeline “Summary” process
Timelines can also be created using audit data from multiple systems. This can be used to perform a type of frequency analysis to identify anomalies, as depicted in Figure 6.
Figure 6: Using frequency analysis on a multi-system audit timeline to hunt for anomalies across multiple systems
Optimizations and Other Features
GoAuditParser can parse FireEye Endpoint Security triage package in less than a minute on average. It has been heavily optimized and has a handful of extra features implemented, including the following:
- Automatically chooses between different parsing methods to optimize speed and memory consumption
- Automatically splits large files into smaller chunks before processing to save time
- Automatically caches progress if you need to pause parsing lots of data and resume later
- Adjustable settings that make it easier to work with parsed audits in Microsoft Excel
- Documented and customizable parsing and timelining configuration files
- Can extract files in bulk from FireEye Endpoint Security file acquisitions
- Many additional small features documented on the project’s GitHub page
By making the GoAuditParser public and open source, we hope to enable those who use our tools with more versatility and customizability in their hunts for evil.