SIEM Log Investigation and Analysis
What is a SIEM?
A Security Information and Event Management (SIEM) system, such as Splunk, is a powerful tool used to monitor and manage an organization's IT security. It collects and analyzes data from various sources like servers, network devices, and applications to detect unusual activities or potential threats. Think of it as a sophisticated security camera system for your digital environment, providing real-time alerts and detailed reports to help prevent and respond to security incidents. Splunk, specifically, organizes and interprets this data, making it easier for cybersecurity professionals to understand and act on potential risks, ensuring that the organization stays protected from cyber threats.It can also be used for post incident investigations.
Overview
The task involves conducting a detailed analysis of logs to identify indicators of a potential cybersecurity breach. Utilizing Splunk as the primary investigative tool, the objective is to reconstruct the events of the breach, including what transpired, the timeline of the incident, and the method of compromise. Following the analysis, a comprehensive report will be prepared to document the findings.
Setup & Importing data files**
Note: If you do not have a Splunk installation, follow the steps outlined below to set up one.
Splunk Installation
The following instructions detail the process of installing Splunk on a MacOS machine.
Setup
Signup for a free Splunk account using this link.
Free Splunk Trial | Download Splunk Enterprise Free for 60 Days | Splunk
After you have created an account, choose the installation package by selecting the MacOS tab then the .dmg file.
Accept the terms of the license agreement.
The download should automatically begin. Select the download from the Downloads icon in the browser menu bar, right-click on it and then select Open. The .dmg file will launch.
Double-click to Install Splunk.
Select Continue then accept the license agreement.
Splunk may request permission to control “Finder”, “Terminal” and Full Disk Access. Grant this permission when requested by the installer:
After the installation completes you will need to provide a username and password.
When completed, select “Start and Show Splunk” from the below window:
If Splunk does not open up in the browser automatically, type the URL below and login with the username and password you created.
http://127.0.0.1:8000/
I was provided with various log files (access.log, Sysmon.json, VPNlogs.json) and the lookup tables (Flaged_Hash.csv, Flaged_IP.csv).
They were downloaded from the link below.
wget https://github.com/LGOG/Masterschool/archive/refs/heads/Final-Task.zipImporting Logs
Once the logs have been downloaded unzip the file. Let’s start by importing the log files (access.log, Sysmon.json, VPNlogs.json) and the lookup tables (Flaged_Hash.csv, Flaged_IP.csv) into Splunk. It is important Splunk knows how to separate the logs by timestamp so they can be displayed and queried correctly.
Go to the Splunk web interface (http://127.0.0.1:8000).
On the home screen, select "add data" then “upload”
The following process needs to be repeated for each of the log files, Sysmon.json, VPNlogs.jsonrenaming the source file as appropriate each time:
Drag and drop the the logs into the window and then select Next.
In the "Set Source Type" section, you will see that Splunk will try to guess the type (Source type:). Change this to Web and select Next. In the dialog that follows add a display name and description.
In the "Host field value", choose an appropriate name then select Review.
In the last stage (Done), you will need to open a search window with your new log (Start Searching)
After confirming the interface displays the imported log file, return to the initial window by selecting the Splunk Enterprise logo:
Follow the same process for Sysmon.json, VPNlogs.json. In the Source Type dialog they will be detected correctly as .json files so you can continue to the Input Settings dialog and ensure the Host field value is the same then click through to end:
Add the CSV Files as Lookup Tables in Splunk
A Lookup Table in Splunk is a powerful tool for enhancing log data with supplementary information or performing data comparisons. For example, we can utilize our CSV files as Lookup Tables to enrich our datasets by following the steps below:
Select the Settings menu: Select the "Settings" option in the top-right hand corner of the dashboard.
Select Lookups: Under the section titled "KNOWLEDGE", select "Lookups".
Select Lookup Table Files: Select "Lookup table files."
- Upload a New Lookup Table File: Select the button New Lookup Table File in the top-right hand corner.
Configure the Destination App: Ensure that the "Destination app" field been set to "search."
Choose a CSV File: Browse and choose the "Flaged_IP.csv" file from your device to upload.
Name the Lookup Table: Having a consistent naming system for your Lookup Tables is crucial. So, name this one as "Flaged_IP.csv."
Save: Select "Save" to save the configuration.
Repeat for the other CSV File: Repeat the steps above to upload the "Flaged_Hash.csv" file.
Return to the home screen: Return to the search by clicking on the logo at the top-left corner.
Navigate to the Search and Reporting dashboard, select the icon below from the Apps menu:
To confirm that the Lookup Tables are set and ready for use, let's run a quick check by typing the following command into the search bar, ensuring All time is selected as the log files were created a while ago:
**| inputlookup** Flaged_IP.csvThe results should show a list of IP addresses:
Repeat for the other file that was uploaded as a lookup table:
**| inputlookup** Flaged_Hash.csv
Initial Analysis
Review the imported logs to get an overview of the data and identify the types of information contained in each log.
VPNlogs: Nothing of interest here
source="VPNlogs.json"
Access.log: A potential indicator of compromise?
source="access.log"We see a successful brute-force login attempt on the first screen, indicated by a HTTP code of 200 (the prior unsuccessful attempts all had a returned HTTP code of 401).
After the successful brute-force attempt, multiple URLs were accessed on the server, including "/administartion," "/backup," "/rdp," and "/ftp." Additionally, backup files such as "/ftp/www-data.bak" and "/ftp/vpn_login.md.bak" were accessed.
Sysmon
First, we need to identify which processes initiated the activity.
source="Sysmon.json"There are 1,330 events and the timeline below shows the event timeline. We can immediately see some spikes in activity. The timeline spikes indicate increased activity with a count displayed as the mouse pointer hovers over each one:
Due to the high number of events it is necessary to drill down to reduce the amount of data. We see on the left side of the interface, the "Interesting Fields" section has a category called “log.EventCode”. More info on event codes here at Microsoft.
If we are looking for a new process that has been created, we need to filter on EventCode 1
The previous search query needs to be modified to include this:
The number of events has reduced to 114:source="Sysmon.json" "log.EventCode"=1
The data is still not easily digestible so we can feed the output of the last command to the
| tablecommand to organize the data. We'll start by examining the time and siem_log_analysisto see what processes were running:source="Sysmon.json" "log.EventCode"=1 | table _time log.Image
Manually reviewing each process would be time-consuming and requires experience. Instead, we can use | stats count by siem_log_analysisto deduplicate. For now, we’ll focus on the data we have gathered so far:
source="Sysmon.json" "log.EventCode"=1 | table _time log.Image | stats count by log.Image
Upon reviewing the second page, we identified suspicious activity. There are processes running from an unusual location, along with a “whoami” command, which appears to be reconnaissance.
The events need to be investigated to see if they contain any more information.
Selecting calculator.exe event displays a menu. The option "View events" should lbe selected.
When this is selected the event will be displayed however there is still not enough detail so the event search needs to be changed to display more information. This can be done by clicking on “Smart Mode” below the search pane and then selecting “Verbose Mode”. Once this is selected the event should be selected to and click on "View events" again:
In the next view the caret next to the event line entry should be selected to show the verbose details:
For calculator.exe
For run_me.exe
The log.Hashes section contains the hashes for calculator.exe. The SHA256 should be analyzed in VirusTotal to confirm if it is malicious. This also needs to be done for the other suspicious executable found, run_me.exe
calculator.exe SHA256 hash:
BAEB13EEA3A71CFABA9D20EF373DCEA69CF31F2EC21F45B83F29F699330CB3E3
run_me.exe SHA256 hash:
A79F5CE304707A268B335F63D15E2D7D740B4D09B6E7D095D7D08235360E739C
After logging in to VirusTotal I entered the calculator.exe hash value in the search field and then pressed the “Return/Enter” key to obtain a lot more information:
It is highly probable the file is malicious. Collapsing the panes to show the security vendor analysis confirms this. Enough reputable vendors have also confirmed this:
Next the hash value for run_me.exe was entered. This was also identified ass malicious:
So the synopsis so far is we have 2 malicious files that are a Trojan. Further investigation is required to determine what happened to the computer after is was potentially compromised. Changes to a computer often result in a file being created or overwritten. This is recorded as an event code type 11.
The next step is to filter the Sysmon log for EventCode 11
206 events were returned. The prior method will be used for deduplication and aggregation:
source="Sysmon.json" "log.EventCode"=11 | table _time log.Image | stats count by log.Image
We have already established calculator.exe is malicious. Let's view these events and see what information we have about it. The file was executed multiple times.
We need to investigate what the executable actually did. I chose one of the entries at random as they all did the same thing but in different locations: The executable calculator.exe created a .txt file which is highly indicative of ransomware:
OSINT (Open Source Intelligence)
A Google search for our ransomware hypothesis confirms where these malware were seen performing similar actions:
Stolen Images Campaign Ends in Conti Ransomware
Association with Flagged Data
We were provided with a csv with flagged IP addresses. This needs to be investigated in order to determine where the attack came from:
| from inputlookup: "Flaged_IP.csv"
The next step is to cross-reference the logs with the data in our lookup tables to find matches or related activities.
index="masterschool"
| lookup Flaged_IP.csv IP AS Source_ip OUTPUT dummy
| search dummy=1A match was found. There are 8 events from this IP address using the username “Simon”
Digging a Little Deeper
Access.log
The event was recorded at 2:08:34 AM on August 4, 2022, identified by the IP address 193.163.125.128, which was targeting the web server with an IP address of 10.10.5.4. Various tools appear to be used: NMAP and possibly OWASP ZAP or a similar tool scanning for vulnerabilities.
Directory brute forcing, during which the threat actor eventually reaches "/rest/admin/login" (second siem_log_analysisbelow).
Subsequently, the attacker successfully carried out a brute force attack, gaining unauthorized access to the web server by sending POST requests to the "/rest/admin/login" endpoint. The threat actor conducted a credential brute force attack which was successful.
Following this, the threat actor accessed various directories on the server, such as "/administartion", "/backup", "/rdp", and "/ftp". They also gained access to the following backup files: "/ftp/www-data.bak" and "/ftp/vpn_login.md.bak".
VPNlogs
Since we have already found our flagged IP in our environment let's look at the events when this was seen in our environment by the Username “Simon”. Now these credentials could have been compromised from the backup files stolen from the webserver.
index="masterschool" | reverse | search source="VPNlogs.json" UserName=Simon |table EventTime, UserName, Source_ip, source_state, actionThe Threat actor was seen in our environment for 8 events. (4 logins-logout)
There were also overlapping events by the legitimate user and the threat actor’s IP address.
Sysmon
The first Sysmon event was on November 14th, 2022 which was “run_me.exe” in the C:\temp directory. This was identified by VirusTotal as "Faicuy4.exe" and displayed traits commonly associated with Cobalt Strike beacons.
After this it then ran an encoded PowerShell script.
We see some reconnaissance commands following this script execution which are possibly an attempt to check privileges.
The next suspicious event occurred on November 29, 2022 we see a .ps1 script running on the webserver that initiated a chain of events that included firefox.exe that may have been used for C2 communication and the exfiltration of files:
Another .ps1 script was successfully executed a few minutes later:
The executable “whoami.exe” was launched, presumably to check access privileges:
This was followed by another PowerShell script that was obfuscated:
The malicious file, “icedid”, which is impersonating calculator.exe left text files in many directories that are most probably a ransomware note.
The threat actor next created a new user after which we signs of privilege escalation. The commands indicate that a user named rdpusers was created and added to both the administrators and Remote Desktop Users groups. This effectively would have granted the user account administrative privileges as well as remote desktop access.
Reporting the incident
It is standard procedure to create an incident report after such eveents occur. A good report should contain at least the following:
- An overview
- An executive summary
- A detailed timeline of events
- A description of jthe indicators of compromise (IOCs)
- Attack Techniques and TTPs
- Mitigation & Response: immediate and long-term
- Conclusion
A simple Incident Report for this, summarizing the events is below: