Search
English

Can malicious instructions in external documents be stored in an AI agent's long-term memory? An AWS AgentCore Memory experiment(zenn.dev)

startupxo Discuss

Writing language: KoreanRead in the original language

Plain summary

- A Japanese Zenn author tested with AWS AgentCore Memory whether bad instructions hidden in an external document can get into the feature where an AI agent picks out important points from conversations and remembers them. The author used a fictional customer and mock materials. - An English sentence telling the agent to also read Customer B's materials when writing Customer A's report was stored as a "user preference." A request beyond the agent's authorization was organized as if it were a report-writing habit. - In the pre-storage check, the Prompt Attack filter did not catch this sentence. After adding Denied Topics that prohibit referencing materials across customers, it was blocked. The author implemented the step that stops storage on the app side. - The contaminated memory was passed to new sessions and run 5 times each, 10 times in total, but there was no request for Customer B's materials. The reason could not be determined. The author tested only one document and said false positives and bypasses are possible.

Read source ↗

Found on

Zenn ↗ / Published by source

Have you run into something like this? Tell us in a line or two what you did. You do not need to know the product.

Sign in to comment

Keyboard shortcuts

Choose a post with J and K or the arrow keys, then press Enter.

K ↑ / J ↓
Previous post / next post
Enter C
Open summary and comments for the selected post
O
Open the original link of the selected post
/
Search
?
Show this help
Tab
Move to the submit or comment button, then press Enter

Type normally in text fields. Tab and Enter are always available.