- Cisco has identified active exploitation of a previously unknown vulnerability in the Web User Interface (Web UI) feature of Cisco IOS XE software (CVE-2023-20198) when exposed to the internet or untrusted networks. This affects both physical and virtual devices running Cisco IOS XE software that also have the HTTP or HTTPS Server feature enabled.
- Successful exploitation of this vulnerability allows an attacker to create an account on the affected device with privilege level 15 access, effectively granting them full control of the compromised device and allowing possible subsequent unauthorized activity.
- The recommendation that Cisco has provided in its security advisory to disable the HTTP server feature on internet-facing systems is consistent with not only best practices but also guidance the U.S. government has provided in the past on mitigating risk from internet-exposed management interfaces.
- Cisco support centers collaborated with the security team after using methods and procedures to correlate similar indicators in a very small number of cases out of our normal substantial daily case volume.
- This is a critical vulnerability, and we strongly recommend affected entities immediately implement the steps outlined in Cisco’s PSIRT advisory.
We discovered early evidence of potentially malicious activity on September 28, 2023, when a case was opened with Cisco's Technical Assistance Center (TAC) that identified unusual behavior on a customer device. Upon further investigation, we observed what we have determined to be related activity as early as September 18. The activity included an authorized user creating a local user account under the username “cisco_tac_admin” from a suspicious IP address (5.149.249[.]74). Instances of this activity ended on October 1, and we did not observe any other associated behavior at that time other than the suspicious account creation.
On October 12, Cisco Talos Incident Response (Talos IR) and TAC detected what we later determined to be an additional cluster of related activity that began on that same day. In this cluster, an unauthorized user was observed creating a local user account under the name “cisco_support” from a second suspicious IP address (154.53.56[.]231). Unlike the September case, this October activity included several subsequent actions, including the deployment of an implant consisting of a configuration file (“cisco_service.conf”). The configuration file defines the new web server endpoint (URI path) used to interact with the implant. That endpoint receives certain parameters, described in more detail below, that allows the actor to execute arbitrary commands at the system level or IOS level. For the implant to become active, the web server must be restarted; in at least one observed case the server was not restarted so the implant never became active despite being installed.
The implant is saved under the file path “/usr/binos/conf/nginx-conf/cisco_service.conf” that contains two variable strings made up of hexadecimal characters. The implant is not persistent—meaning a device reboot will remove it—but the newly created local user accounts remain active even after system reboots. The new user accounts have level 15 privileges, meaning they have full administrator access to the device. This privileged access to the devices and subsequent creation of new users is tracked as CVE-2023-20198.
We assess that these clusters of activity were likely carried out by the same actor. Both clusters appeared close together, with the October activity appearing to build off the September activity. The first cluster was possibly the actor’s initial attempt and testing their code, while the October activity seems to show the actor expanding their operation to include establishing persistent access via deployment of the implant.
The new CVE-2023-20198 vulnerability received the highest Common Vulnerability Scoring System (CVSS) score (10/critical). Successful exploitation would grant an attacker full administrator privileges, allowing them to effectively take full control of the affected router and allowing possible subsequent unauthorized activity.
Leveraging existing detections, we observed the actor exploiting CVE-2021-1435, for which Cisco provided a patch in 2021, to install the implant after gaining access to the device. We have also seen devices fully patched against CVE-2021-1435 getting the implant successfully installed through an as of yet undetermined mechanism.
The implant is based on the Lua programming language and consists of 29 lines of code that facilitates the arbitrary command execution. The attacker must create an HTTP POST request to the device, which delivers the following three functions (Figure 1):
- The first function is dictated by the “menu” parameter, which must exist and must be non-empty. This returns a string of numbers surrounded by forward-slashes, which we suspect might represent the implant’s version or installation date.
- The second function is dictated by the “logon_hash” parameter, which must be set to “1”. This returns an 18-character hexadecimal string that is hardcoded into the implant.
- The third function is also dictated by the “logon_hash” parameter, which checks to see if the parameter matches a 40-character hexadecimal string that is hardcoded into the implant. A second parameter used here is “common_type”, which must be non-empty, and whose value determines whether the code is executed at the system level or IOS level. If the code is executed at the system level, this parameter must be set to “subsystem”, and if it is executed at the IOS level, the parameter must be “iox”. The IOX commands are executed at privilege level 15.
Figure 1: Implant code
In most instances we have observed of this implant being installed, both the 18-character hexadecimal string in the second function and the 40-character hexadecimal string in the third function are unique, although in some cases, these strings were the same across different devices. This suggests there is a way for the actor to compute the value used in the third function from the value returned by the second function, acting as a form of authentication required for the arbitrary command execution provided in the third function.
We strongly recommend organizations that may be affected by this activity immediately implement the guidance outlined in Cisco’s Product Security Incident Response Team (PSIRT) advisory.
Organizations should look for unexplained or newly created users on devices as evidence of potentially malicious activity relating to this threat. One method to identify if the implant is present is to run the following command against the device, where the "DEVICEIP” portion is a placeholder for the IP address of the device to check:
curl -k -X POST "https[:]//DEVICEIP/webui/logoutconfirm.html?logon_hash=1"
Note: The above check should use the HTTP scheme if the device is only configured for an insecure web interface.
This will execute a request to the device’s Web UI to see if the implant is present. If the request returns a hexadecimal string, similar to what was outlined in the third function above, the implant is present. We note this will only work as an indication of compromise if the web server was restarted by the actor after the implant was installed.
We also have the following Snort coverage to address this threat. The first Snort ID is an older rule that covers CVE-2021-1435, and the last three alert if interaction with the implant occurs:
- 3:50118:2
- 3:62527:1
- 3:62528:1
- 3:62529:1
The recommendation that Cisco has provided in its security advisory to disable the HTTP/S server feature on internet-facing systems is consistent with best practices and also guidance the U.S. government has provided in the past on mitigating risk from internet-exposed management interfaces. This is also in line with Cisco’s ongoing work with industry partners as part of the Network Resilience Coalition.
Cisco support centers collaborated with the security team after using methods and procedures to correlate similar indicators in a very small number of cases out of our normal substantial daily case volume.
IOCs
5.149.249[.]74
154.53.56[.]231
Usernames:
cisco_tac_admin
cisco_support
In addition to the curl command referenced above, perform the following checks to determine whether a device may have been compromised:
- Check the system logs for the presence of any of the following log messages where “user” could be “cisco_tac_admin”, “cisco_support” or any configured, local user that is unknown to the network administrator:
%SYS-5-CONFIG_P: Configured programmatically by process SEP_webui_wsma_http from console as user on line
%SEC_LOGIN-5-WEBLOGIN_SUCCESS: Login Success [user: user] [Source: source_IP_address] at 03:42:13 UTC Wed Oct 11 2023
Note: The %SYS-5-CONFIG_P message will be present for each instance that a user has accessed the web UI. The indicator to look for is new or unknown usernames present in the message.
- Check the system logs for the following message where filename is an unknown filename that does not correlate with an expected file installation action:
%WEBUI-6-INSTALL_OPERATION_INFO: User: username, Install Operation: ADD filename