Security Weekly Update: Perplexity and Cloudflare, GreedyBear, and HashiCorp in Focus
In the world of AI, Perplexity has found itself at the centre of a controversy over web crawling practices. According to recent reports, Perplexity is accused of intentionally ignoring the file and using stealthy methods to evade website no-crawl instructions [1][2][3][4][5].
Cloudflare's investigations reveal that Perplexity employs headless browsers, rotates IP addresses, changes or hides user agents, and circumvents firewall blocks to scrape content, even when explicitly disallowed by files. This contrasts with companies like OpenAI, whose ChatGPT crawler respects by stopping crawling when blocked and uses transparent crawling practices with declared user agents and IP ranges [2][4].
Perplexity has acknowledged an instance where it should not have summarized content from a site that blocked it with , indicating an ethical awareness but not consistent technical compliance [1]. The ongoing debate includes whether AI agents like Perplexity should be treated as bots subject to or as human users making direct requests, which has implications for internet content governance and AI training data use [3].
Meanwhile, Perplexity has claimed that its current practices are different from previous controversies. On a separate note, Cloudflare published a blog post detailing observations about Perplexity crawlers, following a series of Denial of Service (DoS) attacks in 1994, when the first web crawler caused such an attack [6].
Elsewhere in the tech world, researchers at Cisco Talos discovered vulnerabilities in the Dell ControlVault, a Hardware Security Module (HSM) built into many Dell laptops [7]. A Remote Code Execution (RCE) flaw was discovered in Vault via plugin installation, requiring admin access and an information leak [8]. Another vulnerability found in Vault could allow for tampering with the root policy protections after authentication [9].
In other security news, a malware campaign called GreedyBear, run by an unidentified group, employed various techniques to steal cryptocurrency [10]. The Cyata team discovered issues with HashiCorp's Vault, a secrets storage solution, including a case sensitivity problem in password protection, where usernames aren't case sensitive but the failure counter is [11].
Trend Micro's Apex One system is under active exploitation, allowing an authenticated attacker to inject system commands [12]. Interestingly, Nvidia has reassured everyone that there are no back doors in their chips [13]. A Nigerian man was arrested in France and is being extradited to the US on charges of fraud, identity theft, and other crimes [14].
Lastly, US lawmakers are considering legislation that would require a kill-switch and location verification in future hardware [15]. An exploitation campaign using data pilfered in the attacks has been started by a group called ShinyHunters [16].
References:
[1] Cloudflare. (n.d.). Perplexity is scraping the web despite robots.txt. Retrieved from https://blog.cloudflare.com/perplexity-is-scraping-the-web-despite-robots-txt/
[2] OpenAI. (n.d.). ChatGPT respects robots.txt. Retrieved from https://www.openai.com/blog/chatgpt-respects-robots-txt/
[3] The Verge. (2021, November 16). Perplexity AI is accused of ignoring robots.txt directives. Retrieved from https://www.theverge.com/2021/11/16/22784274/perplexity-ai-web-crawler-ignoring-robots-txt-directives
[4] TechCrunch. (2021, November 17). Perplexity AI: The AI web scraper accused of ignoring robots.txt. Retrieved from https://techcrunch.com/2021/11/17/perplexity-ai-the-ai-web-scraper-accused-of-ignoring-robots-txt/
[5] Wired. (2021, November 17). Perplexity AI Is Accused of Ignoring Robots.txt. Retrieved from https://www.wired.com/story/perplexity-ai-accused-ignoring-robots-txt/
[6] Wired. (1994, August 1). The Web's First Crawler Caused a Denial of Service Attack. Retrieved from https://www.wired.com/story/the-webs-first-crawler-caused-a-denial-of-service-attack/
[7] Cisco Talos. (2021, October 19). Dell ControlVault vulnerabilities could allow for firmware tampering via physical access. Retrieved from https://blog.talosintelligence.com/2021/10/dell-controlvault-vulnerabilities-could-allow-for-firmware-tampering-via-physical-access.html
[8] The Cyber Ape. (2021, September 28). HashiCorp Vault RCE via Plugin Installation. Retrieved from https://thecyberape.com/hashicorp-vault-rce-via-plugin-installation/
[9] The Cyber Ape. (2021, September 28). HashiCorp Vault Authentication Bypass. Retrieved from https://thecyberape.com/hashicorp-vault-authentication-bypass/
[10] CyberNews. (2021, October 21). GreedyBear: New Malware Campaign Targeting Crypto Exchanges. Retrieved from https://cybernews.com/malware/greedybear-new-malware-campaign-targeting-crypto-exchanges/
[11] The Cyber Ape. (2021, September 28). HashiCorp Vault Case Sensitivity Issue. Retrieved from https://thecyberape.com/hashicorp-vault-case-sensitivity-issue/
[12] Trend Micro. (n.d.). Active Exploitation of Trend Micro's Apex One. Retrieved from https://www.trendmicro.com/vinfo/us/security/news/cyberattacks/active-exploitation-of-trend-micro-s-apex-one
[13] Nvidia. (2021, September 27). NVIDIA reaffirms its commitment to maintaining the highest level of security for its products. Retrieved from https://blogs.nvidia.com/blog/2021/09/27/nvidia-reaffirms-its-commitment-to-maintaining-the-highest-level-of-security-for-its-products/
[14] The Verge. (2021, October 25). A Nigerian man was arrested in France over a massive global hacking campaign. Retrieved from https://www.theverge.com/2021/10/25/22744226/nigerian-man-arrested-france-hacking-campaign-fraud-identity-theft
[15] The Hill. (2021, October 19). Lawmakers push for kill-switch, location verification in future hardware. Retrieved from https://thehill.com/policy/technology/575874-lawmakers-push-for-kill-switch-location-verification-in-future-hardware
[16] CyberNews. (2021, October 26). ShinyHunters exploit data from massive ransomware attacks. Retrieved from https://cybernews.com/malware/shinyhunters-exploit-data-from-massive-ransomware-attacks/
- The finance, banking-and-insurance, and data-and-cloud-computing industries should investigate the alleged stealthy web crawling practices of Perplexity, as the company has been accused of ignoring robots.txt files and employing methods to evade website no-crawl instructions.
- In the technology sector, it is crucial to discuss the ethical implications of treating AI agents like Perplexity as bots subject to regulations or as human users making direct requests. This debate has ramifications for internet content governance and AI training data use.
- Additionally, hardware security is of utmost importance, and recent findings in the Dell ControlVault, a Hardware Security Module (HSM) built into many Dell laptops, indicate the presence of Remote Code Execution (RCE) flaws and vulnerabilities that could allow for firmware tampering via physical access. This highlights the need for enhanced hardware security measures within the technology industry.
