Search Provider Algolia Discloses Security Incident

AlgoliaImage: Algolia

Search service Algolia said it suffered a security breach over the weekend after hackers exploited a well-known vulnerability in the Salt server configuration software to gain access to its infrastructure.

The company said the hackers installed a backdoor and a cryptocurrency miner on a small number of its servers, but that the incident did not impact its operations in any significant way.

Algolia, which provides an on-demand search function for large-scale websites (such as Twitch, Hacker News, or Stripe), said it detected the incident almost immediately after it happened because server alerts notified staff that search and indexing functions were going down for a number of customers.

The company said engineers intervened to remove the malware, shut down impacted servers, and restore service to customers, most of which suffered downtimes that lasted no longer than 10 minutes.

  • 15 clusters out of over 700 (~2%) were impacted by a search downtime longer than 5 minutes.
  • 6 clusters (less than 1%) were impacted by a search downtime longer than 10 minutes.

“While recovering the servers one by one, our main concern was to accurately evaluate what was the exact scale of the attack. […] Analyzing the payloads executed by the malware, we concluded that the only goal of the attack was to mine crypto-currencies, and not to collect, alter, destroy or damage data,” Julien Lemoine, Co-founder & CTO at Algolia.

Attacks on Salt servers continue

In a post-mortem report published on Tuesday, Algolia said the hack took place on Sunday, May 3, at 3:12am, Paris time.

The time of the attack matches with other security breaches reported by LineageOS, Ghost, Digicert, Xen Orchestra, and many other smaller companies (via this GitHub thread).

The attack on Algolia is believed to have been carried out by the operators of the Kinsing cryptocurrency mining botnet, believed to be behind all the aforementioned incidents.

A source in the cyber-security community told ZDNet on Sunday that the operators of the Kinsing botnet were the first threat actor to weaponize two vulnerabilities disclosed last week in Salt, a remote server configuration tool used in data centers, large corporate networks, and cloud setups.

The two vulnerabilities — CVE-2020-11651 (an authentication bypass) and CVE-2020-11652 (a directory traversal) — allowed the Kinsing (H2Miner) botnet crew to automate attacks on a massive scale and take over large server clusters.

In particular, Kinsing operators exploited the authentication bypass bug to gain access to Salt master servers left exposed online, where they installed a backdoor, and then deployed cryptocurrency-mining malware on connected slave servers.

The Kinsing attacks started on Saturday, and they are still ongoing, although new threat actors have also joined in the attacks.

Attacks are expected to amplify in the coming weeks as easily accessible proof-of-concept code for exploiting the authentication bypass issue (CVE-2020-11651) has been published on GitHub by multiple users[1, 2, 3, 4], lowering the bar even further for new attackers to join the fold and launch attacks.

SaltStack, the company behind both the commercial and open-source versions of the Salt software, released patches last week and also did its best to warn customers on the necessity to install the patches as soon as possible.

This week, the security community has backported the patches to older End-of-Life Salt versions, and security researchers also published a detection script that verifies if Salt servers have been patched and if the patches have installed correctly.

Last week, F-Secure, the company who identified and reported the two Salt vulnerabilities, said that a scan revealed more than 6,000 Salt master servers left available on the internet, susceptible to attacks. SaltStack and F-Secure recommdend that companies either move these systems on local networks or at least put them behind a firewall with strict access control rules.