Goodfire Releases a Low-Cost Tool That Monitors AI Agents by Reading Their Internal Signals(techcrunch.com)
Writing language: KoreanRead in the original language
Plain summary
- Usually, whether an AI agent is misbehaving is checked by having another AI reread its output. Goodfire says it has introduced a method that reads signals the model computes internally while it works, using small detectors called probes.
- In the company's tests, monitoring about 1,500 sessions of the Kimi K3 model cost about 51 dollars. The company said the same work would cost 233 dollars with a cheap AI model that checks every step, and about $10,000 with a top-tier model.
- In the same tests, the method caught 94% of malicious hacking sessions and sent 8.7% of harmless sessions for additional review. The company explained that even when 4 probes run at the same time, the time before the model starts responding increased by less than 2%.
- Baseten customers can choose what to monitor from hacking, biochemical weapon misuse, and reward hacking. When something is found, they can decide the response from logging, human review, or refusing the request.
Found on
TechCrunch ↗ / Published by source
Have you run into something like this? Tell us in a line or two what you did. You do not need to know the product.