I was poring over some Windows event logs about a year ago, looking for a security breach. We had good intel that a breach occurred on this system, just not exactly what or when. I was getting ridiculously frustrated by the number of non-relevant entries I had to mentally process and thought "there has to be a better way!"
So I searched the Internet and asked colleagues in search of an application that would allow me to quickly remove lines from a text file. I wanted to be able to scroll through the file, and as I identified text that was irrelevant, remove lines from the display that contained that text. Sounds simple enough, right? But after searching for about a week, it seemed that no one knew of such a tool. Many suggested using a series of "grep -v" commands under Linux or with the Win32 Unix tools. Even though I am an avid command line user and a fan of using grep and Linux, that solution was a bit too clunky and not the sort of streamlined workflow I was looking for. A week more frustrated, I couldn't find any app like the one I was searching for, so I decided I would have to make it myself.
Over two days I wrote a very basic C# application using Microsoft Visual Studio Express. The application had a single function - load a text file into a textbox, let me select text, and remove all lines with that text from being displayed. The original file was never modified, but they weren't shown to me.
I used my new tool on a selection of the Windows event logs and immediately saw the benefit; with some files, this technique of removing lines quickly eliminated about 80-90% of the events. This let me focus closely on the remaining events, which allowed me to find evil and solve crime faster than ever!
After a little use I realized that thought it would be cool if I not only removed lines, but also found where certain strings occurred throughout a file. I started with the idea of statistical analysis on the file - generate information about each word that indicated frequency, distribution, etc. The problem with that is that I couldn't come up with any good way to represent the results. After explaining the idea to my Mandiant colleague, Lindsey Lack, he simply said "I'm a graphical person. Why don't you make a visual representation of the file and display information graphically?". GENIUS!
Our idea was to depict the file as a graphic on which we could highlight areas on the graphic that corresponded to a key word or phrase. The depiction would immediately give you a sense for frequency and distribution. So with help from one of Mandiant's Intelligent Response developers, Matt Frazier, we created a C# control that displays the file as a graphic. The graphic represents a sort of super zoomed-out version of the file. Lines from the original file are displayed as graphics lines (no text) on the screen. The lines displayed are proportional to the line lengths in the file. So you have a graphic on the screen next to the text box that proportionally represents the entire file. So, back to the Windows event logs.
I opened the log and selected a username in question identified through the previous analysis I did. I right-clicked and selected the new function - "Highlight". The graphic lit up with small red lines (highlights), indicating each exact location that username appeared in the file. I immediately noticed something odd - the red highlights appeared in a fairly regular pattern, except around a certain spot, where there were a number of red highlights that just appeared out-of-place in comparison to the rest. We made the graphic clickable, so I clicked in that area and the textbox advanced to that portion of the file. The log entries that came up were very late at night - a time when this user should not have been accessing this system. Further investigation revealed the user's account was compromised, malware was installed, and a number of other things happened that day.
Evil found. Crimes being solved.