securing-agentic-ai-tool-invocation

これはレビュー時に保存された抜粋です。完全で最新の内容は外部ソースを確認してください。

---
name: securing-agentic-ai-tool-invocation
description: Implements defense-in-depth controls at an AI agent's tool-invocation
  boundary using tool allowlisting, least-privilege identity binding, NeMo Guardrails
  policy enforcement, human-in-the-loop approval, and audit logging. Use when hardening
  an agent that calls tools with real side effects (email, payments, file writes,
  code execution), mapping OWASP Agentic AI Top 10 controls, or bounding prompt-injection
  blast radius.
domain: cybersecurity
subdomain: ai-security
tags:
- ai-security
- agentic-ai
- least-privilege
- tool-allowlisting
- human-in-the-loop
- nemo-guardrails
- identity-binding
- owasp-agentic
version: '1.0'
author: mahipal
license: Apache-2.0
nist_ai_rmf:
- GOVERN-1.3
atlas_techniques:
- AML.T0053
---
# Securing Agentic AI Tool Invocation

> **Authorized-use-only notice:** This is a defensive skill. The controls below govern how an AI agent invokes tools/plugins. Deploy them on systems you own or operate. Test guardrail bypasses only against your own agent in a non-production environment.

## Overview

Autonomous (agentic) AI systems decide *which tool to call, with what arguments, and when*, based on model reasoning over untrusted inputs. That makes the tool-invocation boundary the highest-risk control point in an agent: a single successful prompt injection or a poisoned tool can turn the agent into a confused deputy that deletes data, sends money, or pivots into connected systems. The relevant threat is MITRE ATLAS **AML.T0053 (LLM Plugin Compromise)** and the OWASP **Agentic AI Top 10** classes for *Tool Misuse*, *Excessive Agency*, and *Privilege Compromise*.

The defense is layered, defense-in-depth governance of tool calls: (1) a strict **allowlist** of which tools the agent may call and with which argument shapes; (2) **least-privilege identity binding** so each tool call runs with scoped, short-lived credentials tied to the acting user/session — not a single god-mode service account; (3) **policy enforcement** at the call boundary (NVIDIA **NeMo Guardrails** dialog/flow rails and `tool` guardrails, or a deterministic policy wrapper); (4) **human-in-the-loop (HITL)** approval for high-impact actions; and (5) **audit logging** of every invocation for detection. This skill implements all five with verified, runnable patterns using NeMo Guardrails and a framework-agnostic Python policy wrapper.

## When to Use

- When building or hardening an agent that can call tools with real-world side effects (email, payments, file writes, infra changes, code execution).
- When mapping OWASP Agentic AI Top 10 controls onto an existing agent framework.
- When you need to bound the blast radius of prompt injection / tool poisoning.
- When a compliance or governance requirement mandates approvals and audit trails for autonomous actions.
- During an architecture review of an agent's tool layer.

## Prerequisites

- Python 3.10+ and a virtual environment.
- An agent/LLM framework you control.
- Install the tooling:

```bash
python -m venv .venv && source .venv/bin/activate

# NVIDIA NeMo Guardrails — programmable rails incl. tool/flow controls
pip install nemoguardrails

# JSON schema validation for tool argument allowlisting
pip install jsonschema

# (Optional) cloud SDK for scoped credential issuance, e.g. AWS STS
pip install boto3
```

## Objectives

- Define an explicit tool allowlist with per-tool argument schemas (deny-by-default).
- Bind each tool call to a scoped, short-lived identity instead of a shared service account.
- Enforce a policy decision (allow / require-approval / deny) before every invocation.
- Insert human-in-the-loop approval gates for high-impact tools.
- Wrap an agent's tools with NeMo Guardrails and/or a deterministic policy wrapper.
- Produce a tamper-evident audit log of all tool calls mapped to ATLAS AML.T0053.

## MITRE ATT&CK Mapping

| ID | Official Name | Relevance |
|----|---------------|-----------|
| AML.T0053 | LLM Plugin Co