We, along with the security industry and our partners, continue to investigate the extent of the Solorigate attack. While investigations are underway, we want to provide the defender community with intelligence to understand the scope, impact, remediation guidance, and product detections and protections we have built in as a result.
While the full extent of the compromise is still being investigated by the security industry as a whole, in this blog we are sharing insights into the compromised SolarWinds Orion Platform DLL that led to this sophisticated attack. The addition of a few benign-looking lines of code into a single DLL file spelled a serious threat to organizations using the affected product, a widely used IT administration software used across verticals, including government and the security industry. The discreet malicious codes inserted into the DLL called a backdoor composed of almost 4,000 lines of code that allowed the threat actor behind the attack to operate unfettered in compromised networks.
The fact that the compromised file is digitally signed suggests the attackers were able to access the company’s software development or distribution pipeline. Evidence suggests that as early as October 2019, these attackers have been testing their ability to insert code by adding empty classes.
Therefore, insertion of malicious code into the SolarWinds.Orion.Core.BusinessLayer.dll likely occurred at an early stage, before the final stages of the software build, which would include digitally signing the compiled code. As a result, the DLL containing the malicious code is also digitally signed, which enhances its ability to run privileged actions—and keep a low profile.
In many of their actions, the attackers took steps to maintain a low profile. For example, the inserted malicious code is lightweight and only has the task of running a malware-added method in a parallel thread such that the DLL’s normal operations are not altered or interrupted. This method is part of a class, which the attackers named OrionImprovementBusinessLayer to blend in with the rest of the code. The class contains all the backdoor capabilities, comprising 13 subclasses and 16 methods, with strings obfuscated to further hide malicious code.
Once loaded, the backdoor goes through an extensive list of checks to make sure it’s running in an actual enterprise network and not on an analyst’s machines. It then contacts a command-and-control (C2) server using a subdomain generated partly from information gathered from the affected device, which means a unique subdomain for each affected domain. This is another way the attackers try to evade detection.
With a lengthy list of functions and capabilities, this backdoor allows hands-on-keyboard attackers to perform a wide range of actions. As we’ve seen in past human-operated attacks, once operating inside a network, adversaries can perform reconnaissance on the network, elevate privileges, and move laterally. Attackers progressively move across the network until they can achieve their goal, whether that’s cyberespionage or financial gain.
Figure 1. Solorigate malware infection chain
The challenge in detecting these kinds of attacks means organizations should focus on solutions that can look at different facets of network operations to detect ongoing attacks already inside the network, in addition to strong preventative protection.
Where it all starts: A poisoned code library
The attackers inserted malicious code into SolarWinds.Orion.Core.BusinessLayer.dll, a code library belonging to the SolarWinds Orion Platform. The attackers had to find a suitable place in this DLL component to insert their code. Ideally, they would choose a place in a method that gets invoked periodically, ensuring both execution and persistence, so that the malicious code is guaranteed to be always up and running. Such a suitable location turns out to be a method named RefreshInternal.
Figure 2: The method infected with the bootstrapper for the backdoor
Figure 3: What the original method looks like
The modification to this function is very lightweight and could be easily overlooked—all it does is to execute the method OrionImprovementBusinessLayer.Initialize within a parallel thread, so that the normal execution flow of RefreshInternal is not altered.
Why was this method chosen rather than other ones? A quick look at the architecture of this DLL shows that RefreshInternal is part of the class SolarWinds.Orion.Core.BusinessLayer.BackgroundInventory.InventoryManager and is invoked by a sequence of methods that can be traced back to the CoreBusinessLayerPlugin class. The purpose of this class, which initiates its execution with a method named Start (likely at an early stage when the DLL is loaded), is to initialize various other components and schedule the execution of several tasks. Among those tasks is Background Inventory, which ultimately starts the malicious code.
Figure 4. The inserted malicious code runs within a parallel thread
The functionality of the backdoor resides entirely in the class OrionImprovementBusinessLayer, comprising 13 subclasses and 16 methods. Its name blends in with the rest of the legitimate code. The threat actors were savvy enough to avoid give-away terminology like “backdoor”, “keylogger”, etc., and instead opted for a more neutral jargon. At first glance, the code in this DLL looks normal and doesn’t raise suspicions, which could be part of the reason why the insertion of malicious code was undetected for months, especially if the code for this DLL was not frequently updated.
To have some minimal form of obfuscation from prying eyes, the strings in the backdoor are compressed and encoded in Base64, or their hashes are used instead.
Figure 5: Example of obfuscated strings
The Initialize method is the de facto execution entry point of the backdoor. It carries out several checks to verify that it is running in a real victim’s environment:
- It verifies that the process hosting the malicious DLL is named solarwinds.businesslayerhost.exe
- It checks that the last write-time of the malicious DLL is at least 12 to 14 days earlier
- It delays execution by random amounts of time
- It verifies that the domain name of the current device meets the following conditions:
- The domain must not contain certain strings; the check for these strings is implemented via hashes, so at this time the domain names that are block-listed are unknown
- The domain must not contain “solarwinds”
- The domain must not match the regular expression (?i)([^a-z]|^)(test)([^a-z]|$), or in simpler terms, it must not look like a test domain
- It checks that there are no running processes related to security-related software (e.g., Windbg, Autoruns, Wireshark)
- It checks that there are no drivers loaded from security-related software (e.g., groundling32.sys)
- It checks that the status of certain services belonging to security-related software meets certain conditions (e.g., windefend, sense, cavp)
- It checks that the host “api.solarwinds.com” resolves to an expected IP address
If any of these checks fail, the backdoor terminates. All these inspections are carried out to avoid exposing the malicious functionality to unwanted environments, such as test networks or machines belonging to SolarWinds.
After the extensive validation described above, the backdoor enters its main execution stage. At its core, the backdoor is a very standard one that receives instructions from the C2 server, executes those instructions, and sends back information. The type of commands that can be executed range from manipulating of registry keys, to creating processes, and deleting files, etc., effectively providing the attackers with full access to the device, especially since it’s executing from a trusted, signed binary.
In its first step, the backdoor initiates a connection to a predefined C2 server to report some basic information about the compromised system and receive the first commands. The C2 domain is composed of four different parts: three come from strings that are hardcoded in the backdoor, and one component is generated dynamically based on some unique information extracted from the device. This means that every affected device generates a different subdomain to contact (and possibly more than one). Here’s an example of a generated domain:
Figure 6: Dynamically generated C2 domain
The dynamically generated portion of the domain is the interesting part. It is computed by hashing the following data:
- The physical address of the network interface
- The domain name of the device
- The content of the MachineGuid registry value from the key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Cryptography
The backdoor also generates a pseudo-random URI that is requested on the C2 domain. Like the domain, the URI is composed using a set of hardcoded keywords and paths, which are chosen partly at random and partly based on the type of HTTP request that is being sent out. Possible URIs that can be generated follow these formats:
- pki/crl/<random components>.crl, where <random components> can be numbers and one of the following strings:
- fonts/woff/<random components>-webfont<random component>.woff2 or fonts/woff/<random components>.woff2, where the <random components> can be numbers and one or more of the following strings:
- swip/upd/<random components>, where <random components> can be one or more of the following strings:
Here are examples of final URLs generated by the backdoor:
- hxxps://3mu76044hgf7shjf[.]appsync-api[.]eu-west-1[.]avsvmcloud[.]com /swip/upd/Orion[.]Wireless[.]xml
- hxxps://3mu76044hgf7shjf[.]appsync-api[.]us-east-2[.]avsvmcloud[.]com /pki/crl/492-ca[.]crl
- hxxps://3mu76044hgf7shjf[.]appsync-api[.]us-east-1[.]avsvmcloud[.]com /fonts/woff/6047-freefont-ExtraBold[.]woff2
Finally, the backdoor composes a JSON document into which it adds the unique user ID described earlier, a session ID, and a set of other non-relevant data fields. It then sends this JSON document to the C2 server.
Figure 7: Example of data generated by the malware
If the communication is successful, the C2 responds with an encoded, compressed buffer of data containing commands for the backdoor to execute. The C2 might also respond with information about an additional C2 address to report to. The backdoor accepts the following commands:
In a nutshell, these commands allow the attackers to run, stop, and enumerate processes; read, write, and enumerate files and registry keys; collect and upload information about the device; and restart the device, wait, or exit. The command CollectSystemDescription retrieves the following information:
- Local Computer Domain name
- Administrator Account SID
- OS Version
- System Directory
- Device uptime
- Information about the network interfaces
Resulting hands-on-keyboard attack
Once backdoor access is obtained, the attackers follow the standard playbook of privilege escalation exploration, credential theft, and lateral movement hunting for high-value accounts and assets. To avoid detection, attackers renamed Windows administrative tools like adfind.exe which were then used for domain enumeration.
C:\Windows\system32\cmd.exe /C csrss.exe -h breached.contoso.com -f (name=”Domain Admins”) member -list | csrss.exe -h breached.contoso.com -f objectcategory=* > .\Mod\mod1.log
$scheduler = New-Object -ComObject (“Schedule.Service”);$scheduler.Connect($env:COMPUTERNAME);$folder = $scheduler.GetFolder(“\Microsoft\Windows\SoftwareProtectionPlatform”);$task = $folder.GetTask(“EventCacheManager”);$definition = $task.Definition;$definition.Settings.ExecutionTimeLimit = “PT0S”;$folder.RegisterTaskDefinition($task.Name,$definition,6,”System”,$null,5);echo “Done” C:\Windows\system32\cmd.exe /C schtasks /create /F /tn “\Microsoft\Windows\SoftwareProtectionPlatform\EventCacheManager” /tr “C:\Windows\SoftwareDistribution\EventCacheManager.exe” /sc ONSTART /ru system /S [machine_name]
Persistence is achieved via backdoors deployed via various techniques:
Powershell -nop -exec bypass -EncodedCommand
The –EncodedCommand, once decoded, would resemble:
Invoke-WMIMethod win32_process -name create -argumentlist ‘rundll32 c:\windows\idmu\common\ypprop.dll _XInitImageFuncPtrs’ -ComputerName WORKSTATION
C:\Windows\System32\rundll32.exe C:\Windows\Microsoft.NET\Framework64\[malicious .dll file], [various exports]
With Rundll32, each compromised device receives a unique binary hash, unique local filesystem path, pseudo-unique export, and unique C2 domain.
The backdoor also allows the attackers to deliver second-stage payloads, which are part of the Cobalt Strike software suite. We continue to investigate these payloads, which are detected as Trojan:Win32/Solorigate.A!dha, as the situation continues to unfold.
Endpoint product(s) and hardening guidance
Supply chain compromise continues to be a growing concern in the security industry. The Solorigate incident is a grave reminder that these kinds of attacks can achieve the harmful combination of widespread impact and deep consequences for successfully compromised networks. We continue to urge customers to:
- Isolate and investigate devices where these malicious binaries have been detected
- Identify accounts that have been used on the affected device and consider them compromised
- Investigate how those endpoints might have been compromised
- Investigate the timeline of device compromise for indications of lateral movement
Hardening networks by reducing attack surfaces and building strong preventative protection are baseline requirements for defending organizations. On top of that, comprehensive visibility into system and network activities drive the early detection of anomalous behaviors and potential signs of compromise. More importantly, the ability to correlate signals through AI could surface more evasive attacker activity.
Comprehensive detection coverage via endpoints would have covered the realm that the Solorigate attack chain utilized. These detections raise alerts that inform security operations teams about the presence of activities and artifacts related to this incident. Given that this attack involves the compromise of legitimate software, automatic remediation is not enabled to prevent service interruption. The detections, however, provide visibility into the attack activity. Analysts can then use investigation and remediation tools in their respective Endpoint software to perform deep investigation and additional hunting.
The vast majority of end point security provides visibility beyond endpoints by consolidating threat data from across domains – identities, data, cloud apps, as well as endpoints – delivering coordinated defense against this threat. This cross-domain visibility allows Microsoft 365 Defender to correlate signals and comprehensively resolve whole attack chains. Security operations teams can then hunt using this rich threat data and gain insights for hardening networks from compromise.
Figure 8. Microsoft Defender for Endpoint detections across the Solorigate attack chain
Several Microsoft Defender for Endpoint capabilities are relevant to the Solorigate attack:
Next generation protection
The default antimalware solution on Windows 10, detects and blocks the malicious DLL and its behaviors. It quarantines malware, even if the process is running.
Detection for backdoored SolarWinds.Orion.Core.BusinessLayer.dll files:
Detection for Cobalt Strike fragments in process memory and stops the process:
Detection for the second-stage payload, a cobalt strike beacon that might connect to infinitysoftwares[.]com.
Detection for the PowerShell payload that grabs hashes and SolarWinds passwords from the database along with machine information:
Figure 9. Microsoft Defender for Endpoint prevented malicious binaries
Endpoint detection and response (EDR)
Alerts with the following titles in the Microsoft Defender Security Center and Microsoft 365 security center can indicate threat activity on your network:
- SolarWinds Malicious binaries associated with a supply chain attack
- SolarWinds Compromised binaries associated with a supply chain attack
- Network traffic to domains associated with a supply chain attack
Alerts with the following titles in the Microsoft Defender Security Center and Microsoft 365 security center can indicate the possibility that the threat activity in this report occurred or might occur later. These alerts can also be associated with other malicious threats.
- ADFS private key extraction attempt
- Masquerading Active Directory exploration tool
- Suspicious mailbox export or access modification
- Possible attempt to access ADFS key material
- Suspicious ADFS adapter process created
Figure 10. Microsoft Defender for Endpoint detections of suspicious LDAP query being launched and attempted ADFS private key extraction
Figure 11. Microsoft Defender for Endpoint alert description and recommended actions for possible attempt to access ADFS key material
Our ability to deliver these protections through our security technologies is backed by our security experts who immediately investigated this attack and continue to look into the incident as it develops. Careful monitoring by experts is critical in this case because we’re dealing with a highly motivated and highly sophisticated threat actor. In the same way that our products integrate with each other to consolidate and correlate signals, security experts and threat researchers across Microsoft are working together to address this advanced attack and ensure our customers are protected.
Threat analytics report
We published a comprehensive threat analytics report on this incident. Threat analytics reports provide technical information, detection details, and recommended mitigations designed to empower defenders to understand attacks, assess its impact, and review defenses.
Figure 12. Threat analytics report on the Solorigate attack
MITRE ATT&CK techniques observed
This threat makes use of attacker techniques documented in the MITRE ATT&CK framework.
- Initial Access
- Command and Control
- Defense Evasion
Additional malware discovered
In an interesting turn of events, the investigation of the whole SolarWinds compromise led to the discovery of an additional malware that also affects the SolarWinds Orion product but has been determined to be likely unrelated to this compromise and used by a different threat actor. The malware consists of a small persistence backdoor in the form of a DLL file named App_Web_logoimagehandler.ashx.b6031896.dll, which is programmed to allow remote code execution through SolarWinds web application server when installed in the folder “inetpub\SolarWinds\bin\”. Unlike Solorigate, this malicious DLL does not have a digital signature, which suggests that this may be unrelated to the supply chain compromise. Nonetheless, the infected DLL contains just one method (named DynamicRun), that can receive a C# script from a web request, compile it on the fly, and execute it.
Figure 13: Original DLL
Figure 14: The malicious addition that calls the DynamicRun method
This code provides an attacker the ability to send and execute any arbitrary C# program on the victim’s device. Antivirus detects this compromised DLL as Trojan:MSIL/Solorigate.G!dha.