LLMjacking: Stolen Cloud Credentials Used in New AI Attack

AI 4周前 admin
49 0 0

LLMjacking: Stolen Cloud Credentials Used in New AI Attack

The Sysdig Threat Research Team (TRT) recently observed a new attack that leveraged stolen cloud credentials in order to target ten cloud-hosted large language model (LLM) services, known as LLMjacking. The credentials were obtained from a popular target, a system running a vulnerable version of Laravel (CVE-2021-3129). Attacks against LLM-based Artificial Intelligence (AI) systems have been discussed often, but mostly around prompt abuse and altering training data. In this case, attackers intend to sell LLM access to other cybercriminals while the cloud account owner pays the bill.
Sysdig 威胁研究团队 (TRT) 最近观察到一种新的攻击,该攻击利用被盗的云凭据来针对十个云托管的大型语言模型 (LLM) 服务,称为 LLMjacking。这些凭据是从流行的目标获取的,该目标运行易受攻击的 Laravel 版本 (CVE-2021-3129)。针对LLM基于人工智能 (AI) 系统的攻击经常被讨论,但主要是围绕及时滥用和更改训练数据。在这种情况下,攻击者打算将访问权限出售给LLM其他网络犯罪分子,而云帐户所有者则支付账单。

Once initial access was obtained, they exfiltrated cloud credentials and gained access to the cloud environment, where they attempted to access local LLM models hosted by cloud providers: in this instance, a local Claude (v2/v3) LLM model from Anthropic was targeted. If undiscovered, this type of attack could result in over $46,000 of LLM consumption costs per day for the victim.
获得初始访问权限后,他们泄露了云凭据并获得了对云环境的访问权限,在那里他们试图访问由云提供商托管的本地LLM模型:在本例中,来自 Anthropic 的本地 Claude (v2/v3) LLM 模型成为目标。如果未被发现,这种类型的攻击可能会导致受害者每天花费超过 46,000 美元的LLM消费成本。

Sysdig researchers discovered evidence of a reverse proxy for LLMs being used to provide access to the compromised accounts, suggesting a financial motivation.  However, another possible motivation is to extract LLM training data. 
Sysdig 研究人员发现了反向代理LLMs用于提供对受感染帐户的访问的证据,这表明存在经济动机。 但是,另一个可能的动机是提取LLM训练数据。

LLMjacking: Stolen Cloud Credentials Used in New AI Attack

Breadth of Targets 目标的广度

We were able to discover the tools that were generating the requests used to invoke the models during the attack. This revealed a broader script that was able to check credentials for ten different AI services in order to determine which were useful for their purposes. These services include:
我们能够发现在攻击期间生成用于调用模型的请求的工具。这揭示了一个更广泛的脚本,该脚本能够检查十种不同 AI 服务的凭据,以确定哪些服务对其目的有用。这些服务包括:

  • AI21 Labs, Anthropic, AWS Bedrock, Azure, ElevenLabs, MakerSuite, Mistral, OpenAI, OpenRouter, and GCP Vertex AI
    AI21 Labs、Anthropic、AWS Bedrock、Azure、ElevenLabs、MakerSuite、Mistral、OpenAI、OpenRouter 和 GCP Vertex AI

The attackers are looking to gain access to a large amount of LLM models across different services. No legitimate LLM queries were actually run during the verification phase. Instead, just enough was done to figure out what the credentials were capable of and any quotas. In addition, logging settings are also queried where possible. This is done to avoid detection when using the compromised credentials to run their prompts.

Background 背景

Hosted LLM Models 托管LLM模型

All major cloud providers, including Azure Machine Learning, GCP’s Vertex AI, and AWS Bedrock, now host large language model (LLM) services. These platforms provide developers with easy access to various popular models used in LLM-based AI. As illustrated in the screenshot below, the user interface is designed for simplicity, enabling developers to start building applications quickly.
所有主要的云提供商(包括 Azure 机器学习、GCP 的 Vertex AI 和 AWS Bedrock)现在都托管大型语言模型 (LLM) 服务。这些平台使开发人员能够轻松访问基于 AI 的各种LLM流行模型。如下面的屏幕截图所示,用户界面旨在简化设计,使开发人员能够快速开始构建应用程序。

LLMjacking: Stolen Cloud Credentials Used in New AI Attack

These models, however, are not enabled by default. Instead, a request needs to be submitted to the cloud vendor in order to run them. For some models, it is an automatic approval; for others, like third-party models, a small form must be filled out. Once a request is made, the cloud vendor usually enables access pretty quickly. The requirement to make a request is often more of a speed bump for attackers rather than a blocker, and shouldn’t be considered a security mechanism. 

Cloud vendors have simplified the process of interacting with hosted cloud-based language models by using straightforward CLI commands. Once the necessary configurations and permissions are in place, you can easily engage with the model using a command similar to this:
云供应商通过使用简单的 CLI 命令简化了与托管的基于云的语言模型交互的过程。一旦必要的配置和权限到位,你就可以使用类似于以下内容的命令轻松地与模型进行交互:

aws bedrock-runtime invoke-model –model-id anthropic.claude-v2 –body ‘{“prompt”: “\n\nHuman: story of two dogs\n\nAssistant:”, “max_tokens_to_sample” : 300}’  –cli-binary-format raw-in-base64-out  invoke-model-output.txt
aws bedrock-runtime invoke-model –model-id anthropic.claude-v2 –body ‘{“prompt”: “\n\n人类:两只狗的故事\n\nAssistant:”, “max_tokens_to_sample” : 300}’ –cli-binary-format raw-in-base64-out invoke-model-output.txt

LLM Reverse Proxy LLM反向代理

The key checking code that verifies if credentials are able to use targeted LLMs also makes reference to another project: OAI Reverse Proxy. This open source project acts as a reverse proxy for LLM services. Using software such as this would allow an attacker to centrally manage access to multiple LLM accounts while not exposing the underlying credentials, or in this case, the underlying pool of compromised credentials. During the attack using the compromised cloud credentials, a user-agent that matches OAI Reverse Proxy was seen attempting to use LLM models.
用于验证凭据是否能够使用目标LLMs的密钥检查代码还引用了另一个项目:OAI 反向代理。这个开源项目充当服务的反LLM向代理。使用此类软件将允许攻击者集中管理对多个LLM帐户的访问,同时不暴露基础凭据,或者在这种情况下,暴露泄露凭据的基础池。在使用泄露的云凭据的攻击期间,发现与 OAI 反向代理匹配的用户代理试图使用LLM模型。

LLMjacking: Stolen Cloud Credentials Used in New AI Attack

The image above is an example of an OAI Reverse Proxy we found running on the Internet. There is no evidence that this instance is tied to this attack in any way, but it does show the kind of information it collects and displays. Of special note are the token counts (“tookens”), costs, and keys which are potentially logging.
上图是我们在 Internet 上发现的运行的 OAI 反向代理示例。没有证据表明此实例以任何方式与此攻击有关,但它确实显示了它收集和显示的信息类型。特别需要注意的是令牌计数(“tookens”)、成本和可能被记录的密钥。

LLMjacking: Stolen Cloud Credentials Used in New AI Attack

This example shows an OAI reverse proxy instance, which is setup to use multiple types of LLMs. There is no evidence that this instance is involved with the attack. 
此示例显示了一个 OAI 反向代理实例,该实例设置为使用多种类型的 LLMs.没有证据表明此实例与攻击有关。

If the attackers were gathering an inventory of useful credentials and wanted to sell access to the available LLM models, a reverse proxy like this could allow them to monetize their efforts.

LLMjacking: Stolen Cloud Credentials Used in New AI Attack

Technical Analysis 技术分析

In this technical breakdown, we explore how the attackers navigated a cloud environment to carry out their intrusion. By employing seemingly legitimate API requests within the cloud environment, they cleverly tested the boundaries of their access without immediately triggering alarms. The example below demonstrates a strategic use of the InvokeModel API call logged by CloudTrail. Although the attackers issued a valid request, they intentionally set the max_tokens_to_sample parameter to -1. This unusual parameter, typically expected to trigger an error, instead served a dual purpose. It confirmed not only the existence of access to the LLMs but also that these services were active, as indicated by the resulting ValidationException. A different outcome, such as an AccessDenied error, would have suggested restricted access. This subtle probing reveals a calculated approach to uncover what actions their stolen credentials permitted within the cloud account.
在此技术细分中,我们将探讨攻击者如何驾驭云环境进行入侵。通过在云环境中使用看似合法的 API 请求,他们巧妙地测试了其访问边界,而不会立即触发警报。以下示例演示了对 CloudTrail 记录的 InvokeModel API 调用的策略性使用。尽管攻击者发出了有效的请求,但他们故意将 max_tokens_to_sample 参数设置为 -1。这个不寻常的参数通常会导致错误,但实际上具有双重目的。它不仅确认了对 的LLMs访问的存在,而且还确认了这些服务处于活动状态,如生成的 ValidationException 所示。如果出现不同的结果(如 AccessDenied 错误),则建议访问受限。这种微妙的探测揭示了一种经过计算的方法,可以发现其被盗凭据在云帐户中允许的操作。


The InvokeModel call is logged by CloudTrail and an example malicious event can be seen below. They sent a legitimate request but specified “max_tokens_to_sample” to be -1. This is an invalid error which causes the “ValidationException” error, but it is useful information for the attacker to have because it tells them the credentials have access to the LLMs and they have been enabled. Otherwise, they would have received an “AccessDenied” error.
CloudTrail 会记录 InvokeModel 调用,下面可以看到一个恶意事件示例。他们发送了一个合法的请求,但将 “ max_tokens_to_sample ” 指定为 -1。这是一个导致“ValidationException”错误的无效错误,但它对攻击者来说是有用的信息,因为它告诉他们凭据可以访问并且LLMs它们已被启用。否则,他们将收到“AccessDenied”错误。


    "eventVersion": "1.09",

    "userIdentity": {

        "type": "IAMUser",

        "principalId": "[REDACTED]",

        "arn": "[REDACTED]",

        "accountId": "[REDACTED]",

        "accessKeyId": "[REDACTED]",

        "userName": "[REDACTED]"


    "eventTime": "[REDACTED]",

    "eventSource": "bedrock.amazonaws.com",

    "eventName": "InvokeModel",

    "awsRegion": "us-east-1",

    "sourceIPAddress": "",

    "userAgent": "Boto3/1.29.7 md/Botocore#1.32.7 ua/2.0 os/windows#10 md/arch#amd64 lang/python#3.12.1 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.32.7",

    "errorCode": "ValidationException",

    "errorMessage": "max_tokens_to_sample: range: 1..1,000,000",

    "requestParameters": {

        "modelId": "anthropic.claude-v2"


    "responseElements": null,

    "requestID": "d4dced7e-25c8-4e8e-a893-38c61e888d91",

    "eventID": "419e15ca-2097-4190-a233-678415ed9a4f",

    "readOnly": true,

    "eventType": "AwsApiCall",

    "managementEvent": true,

    "recipientAccountId": "[REDACTED]",

    "eventCategory": "Management",

    "tlsDetails": {

        "tlsVersion": "TLSv1.3",

        "cipherSuite": "TLS_AES_128_GCM_SHA256",

        "clientProvidedHostHeader": "bedrock-runtime.us-east-1.amazonaws.com"


}Code language: Perl (perl)

Example Cloudtrail log 示例 Cloudtrail 日志

AWS Bedrock is not supported in all regions so the attackers called “InvokeModel” only in the supported regions. At this time, Bedrock is supported in us-east-1, us-west-2, ap-southeast-1, ap-northeast-1, eu-central-1, eu-west-3, and us-gov-west-1, as shown here. Different models are available depending on the region; here is the list of models supported by AWS Region.
并非所有区域都支持 AWS Bedrock,因此攻击者仅在支持的区域中调用“InvokeModel”。目前,us-east-1、us-west-2、ap-southeast-1、ap-northeast-1、eu-central-1、eu-west-3 和 us-gov-west-1 支持 Bedrock,如下所示。根据地区不同,有不同的型号可供选择;以下是 AWS 区域支持的模型列表。


Interestingly, the attackers showed interest in how the service was configured. This can be done by calling “GetModelInvocationLoggingConfiguration,” which returns S3 and Cloudwatch logging configuration if enabled. In our setup, we used both S3 and Cloudwatch to gather as much data about the attack as possible. 
有趣的是,攻击者对服务的配置方式表现出兴趣。这可以通过调用“GetModelInvocationLoggingConfiguration”来完成,如果启用,它将返回 S3 和 Cloudwatch 日志记录配置。在我们的设置中,我们同时使用 S3 和 Cloudwatch 来收集尽可能多的攻击数据。


    "loggingConfig": {

        "cloudWatchConfig": {

            "logGroupName": "[REDACTED]",

            "roleArn": "[REDACTED]",

            "largeDataDeliveryS3Config": {

                "bucketName": "[REDACTED]",

                "keyPrefix": "[REDACTED]"



        "s3Config": {

            "bucketName": "[REDACTED]",

            "keyPrefix": ""


        "textDataDeliveryEnabled": true,

        "imageDataDeliveryEnabled": true,

        "embeddingDataDeliveryEnabled": true


}Code language: Perl (perl)

Example GetModelInvocationLoggingConfiguration response
示例 GetModelInvocationLoggingConfiguration 响应

Information about the prompts being run and their results are not stored in Cloudtrail. Instead, additional configuration needs to be done to send that information to Cloudwatch and S3. This check is done to hide the details of their activities from any detailed observations. OAI Reverse Proxy states it will not use any AWS key that has logging enabled for the sake of “privacy.” This makes it impossible to inspect the prompts and responses if they are using the AWS Bedrock vector.
有关正在运行的提示及其结果的信息不会存储在 Cloudtrail 中。相反,需要执行其他配置才能将该信息发送到 Cloudwatch 和 S3。进行此检查是为了从任何详细观察中隐藏其活动的详细信息。OAI 反向代理声明,为了“隐私”,它不会使用任何启用了日志记录的 AWS 密钥。这使得无法检查提示和响应(如果它们使用的是 AWS Bedrock 向量)。

Impact 冲击

In an LLMjacking attack, the damage comes in the form of increased costs to the victim. It shouldn’t be surprising to learn that using an LLM isn’t cheap and that cost can add up very quickly. Considering the worst-case scenario where an attacker abuses Anthropic Claude 2.x and reaches the quota limit in multiple regions, the cost to the victim can be over $46,000 per day.
在 LLMjacking 攻击中,损害以受害者成本增加的形式出现。得知使用 an LLM 并不便宜并且成本会很快增加也就不足为奇了。考虑到最坏的情况,即攻击者滥用 Anthropic Claude 2.x 并在多个区域达到配额限制,受害者每天的成本可能超过 46,000 美元。

According to the pricing and the initial quota limit for Claude 2:
根据 Claude 2 的定价和初始配额限制:

  • 1000 input tokens cost $0.008, 1000 output tokens cost $0.024.
    1000 个输入代币的成本为 0.008 美元,1000 个输出代币的成本为 0.024 美元。
  • Max 500,000 input and output tokens can be processed per minute according to AWS Bedrock. We can consider the average cost between input and output tokens, which is $0.016 for 1000 tokens.
    根据 AWS Bedrock,每分钟最多可以处理 500000 个输入和输出令牌。我们可以考虑输入和输出代币之间的平均成本,即 0.016 个代币为 1000 美元。

Leading to the total cost: (500K tokens/1000 * $0.016) * 60 minutes * 24 hours * 4 regions = $46,080 / day
导致总成本:(500K 代币/1000 * 0.016 美元)* 60 分钟 * 24 小时 * 4 个地区 = 46,080 美元/天

By maximizing the quota limits, attackers can also block the compromised organization from using models legitimately, disrupting business operations.

Detection 检波

The ability to detect and respond swiftly to potential threats can make all the difference in maintaining a robust defense. Drawing insights from recent feedback and industry best practices, we’ve distilled key strategies to elevate your detection capabilities:

  • Cloud Logs Detections: Tools like Falco, Sysdig Secure, and CloudWatch Alerts are indispensable allies. Organizations can proactively identify suspicious behavior by monitoring runtime activity and analyzing cloud logs, including reconnaissance tactics such as those employed within AWS Bedrock. 
    云日志检测:Falco、Sysdig Secure 和 CloudWatch Alerts 等工具是不可或缺的盟友。组织可以通过监控运行时活动和分析云日志(包括侦察策略(例如在 AWS Bedrock 中采用的策略)来主动识别可疑行为。
  • Detailed Logging: Comprehensive logging, including verbose logging, offers invaluable visibility into the inner workings of your cloud environment. Verbose information about model invocations and other critical activities gives organizations a nuanced understanding about activity in their cloud environments. 

Cloud Log Detections 云日志检测

Monitoring cloud logs can reveal suspicious or unauthorized activity. Using Falco or Sysdig Secure, the reconnaissance methods used during the attack can be detected, and a response can be started. For Sysdig Secure customers, this rule can be found in the Sysdig AWS Notable Events policy.
监控云日志可能会发现可疑或未经授权的活动。使用 Falco 或 Sysdig Secure,可以检测攻击期间使用的侦察方法,并启动响应。对于 Sysdig Secure 客户,可以在 Sysdig AWS 值得注意的事件策略中找到此规则。

Falco rule: 法尔科规则:

- rule: Bedrock Model Recon Activity

  desc: Detect reconaissance attempts to check if Amazon Bedrock is enabled, based on the error code. Attackers can leverage this to discover the status of Bedrock, and then abuse it if enabled.

    condition: jevt.value[/eventSource]="bedrock.amazonaws.com" and jevt.value[/eventName]="InvokeModel" and jevt.value[/errorCode]="ValidationException"

    output: A reconaissance attempt on Amazon Bedrock has been made (requesting user=%aws.user, requesting IP=%aws.sourceIP, AWS region=%aws.region, arn=%jevt.value[/userIdentity/arn], userAgent=%jevt.value[/userAgent], modelId=%jevt.value[/requestParameters/modelId])

    priority: WARNINGCode language: Perl (perl)

In addition, CloudWatch alerts can be configured to handle suspicious behaviors. Several runtime metrics for Bedrock can be monitored to trigger alerts.
此外,还可以将 CloudWatch 警报配置为处理可疑行为。可以监视 Bedrock 的多个运行时指标以触发警报。

Detailed Logging 详细日志记录

Monitoring your organization’s use of language model (LLM) services is crucial, and various cloud vendors provide facilities to streamline this process. This typically involves setting up mechanisms to log and store data about model invocations.
监视组织对语言模型 (LLM) 服务的使用至关重要,各种云供应商提供了简化此过程的工具。这通常涉及设置机制来记录和存储有关模型调用的数据。

For AWS Bedrock specifically, users can leverage CloudWatch and S3 for enhanced monitoring capabilities. CloudWatch can be set up by creating a log group and assigning a role with the necessary permissions. Similarly, to log into S3, a designated bucket is required as a destination. It is important to note that the CloudTrail log of the InvokeModel command does not capture details about the prompt input and output. However, Bedrock settings allow for easy activation of model invocation logging. Additionally, for model input or output data larger than 100kb or in binary format, users must explicitly specify an S3 destination to handle large data delivery. This includes input and output images, which are stored in the logs as Base64 strings. Such comprehensive logging mechanisms ensure that all aspects of model usage are monitored and archived for further analysis and compliance.
具体而言,对于 AWS Bedrock,用户可以利用 CloudWatch 和 S3 来增强监控功能。可以通过创建日志组并分配具有必要权限的角色来设置 CloudWatch。同样,要登录 S3,需要指定存储桶作为目标。请务必注意,InvokeModel 命令的 CloudTrail 日志不会捕获有关提示输入和输出的详细信息。但是,基岩设置允许轻松激活模型调用日志记录。此外,对于大于 100kb 或二进制格式的模型输入或输出数据,用户必须明确指定 S3 目标以处理大型数据传输。这包括输入和输出图像,这些图像作为 Base64 字符串存储在日志中。这种全面的日志记录机制可确保对模型使用的所有方面进行监控和存档,以便进一步分析和合规性。

The logs contain additional information about the tokens processed, as shown in the following example:


    "schemaType": "ModelInvocationLog",

    "schemaVersion": "1.0",

    "timestamp": "[REDACTED]",

    "accountId": "[REDACTED]",

    "identity": {

        "arn": "[REDACTED]"


    "region": "us-east-1",

    "requestId": "bea9d003-f7df-4558-8823-367349de75f2",

    "operation": "InvokeModel",

    "modelId": "anthropic.claude-v2",

    "input": {

        "inputContentType": "application/json",

        "inputBodyJson": {

            "prompt": "\n\nHuman: Write a story of a young wizard\n\nAssistant:",

            "max_tokens_to_sample": 300


        "inputTokenCount": 16


    "output": {

        "outputContentType": "application/json",

        "outputBodyJson": {

            "completion": " Here is a story about a young wizard:\n\nMartin was an ordinary boy living in a small village. He helped his parents around their modest farm, tending to the animals and working in the fields. [...] Martin's favorite subject was transfiguration, the art of transforming objects from one thing to another. He mastered the subject quickly, amazing his professors by turning mice into goblets and stones into fluttering birds.\n\nMartin",

            "stop_reason": "max_tokens",

            "stop": null


        "outputTokenCount": 300


}Code language: Perl (perl)

Example S3 log 示例 S3 日志

Recommendations 建议

This attack could have been prevented in a number of ways, including:

  • Vulnerability management to prevent initial access.
  • Secrets management to ensure credentials are not stored in the clear where they can be stolen.
  • CSPM/CIEM to ensure the abused account had the least amount of permissions it needed.

As highlighted by recent research, cloud vendors offer a range of tools and best practices designed to mitigate the risks of cloud attacks. These tools help organizations build and maintain a secure cloud environment from the outset.

For instance, AWS provides several robust security measures. The AWS Security Reference Architecture outlines best practices for securely constructing your cloud environment. Additionally, AWS recommends using Service Control Policies (SCP) to centrally manage permissions, which helps minimize the risk associated with over-permissioned accounts that could potentially be abused. These guidelines and tools are part of AWS’s commitment to enhancing security and providing customers with the resources to protect their cloud infrastructure effectively. Other cloud vendors offer similar frameworks and tools, ensuring that users have access to essential security measures to safeguard their data and services regardless of the platform.
例如,AWS 提供了多种强大的安全措施。AWS 安全参考架构概述了安全构建云环境的最佳实践。此外,AWS 建议使用服务控制策略 (SCP) 来集中管理权限,这有助于最大限度地降低与可能被滥用的过度权限账户相关的风险。这些准则和工具是 AWS 致力于增强安全性并为客户提供有效保护其云基础设施的资源的一部分。其他云供应商提供类似的框架和工具,确保用户能够访问必要的安全措施,以保护他们的数据和服务,无论平台如何。

Conclusion 结论

Stolen cloud and SaaS credentials continue to be a common attack vector. This trend will only increase in popularity as attackers learn all of the ways they can leverage their new access for financial gain. The use of LLM services can be expensive, depending on the model and the amount of tokens being fed to it. Normally, this would cause a developer to try and be efficient — sadly, attackers do not have the same incentive. Detection and response is critical to deal with any issues quickly. 
被盗的云和 SaaS 凭据仍然是常见的攻击媒介。随着攻击者了解他们可以利用新访问权限获得经济利益的所有方法,这种趋势只会越来越受欢迎。LLM服务的使用可能很昂贵,具体取决于模型和提供给它的代币数量。通常,这会导致开发人员尝试提高效率——可悲的是,攻击者没有同样的动机。检测和响应对于快速处理任何问题至关重要。

IoCs 国际奥委会

IP Addresses IP 地址

原文始发于ALESSANDRO BRUCATO :LLMjacking: Stolen Cloud Credentials Used in New AI Attack

版权声明:admin 发表于 2024年5月18日 上午9:59。
转载请注明:LLMjacking: Stolen Cloud Credentials Used in New AI Attack | CTF导航