Search
English

Goodfire Releases a Low-Cost Tool That Monitors AI Agents by Reading Their Internal Signals(techcrunch.com)

startupxo Discuss

Writing language: KoreanRead in the original language

Plain summary

- Usually, whether an AI agent is misbehaving is checked by having another AI reread its output. Goodfire says it has introduced a method that reads signals the model computes internally while it works, using small detectors called probes. - In the company's tests, monitoring about 1,500 sessions of the Kimi K3 model cost about 51 dollars. The company said the same work would cost 233 dollars with a cheap AI model that checks every step, and about $10,000 with a top-tier model. - In the same tests, the method caught 94% of malicious hacking sessions and sent 8.7% of harmless sessions for additional review. The company explained that even when 4 probes run at the same time, the time before the model starts responding increased by less than 2%. - Baseten customers can choose what to monitor from hacking, biochemical weapon misuse, and reward hacking. When something is found, they can decide the response from logging, human review, or refusing the request.

Read source ↗

Found on

TechCrunch ↗ / Published by source

Have you run into something like this? Tell us in a line or two what you did. You do not need to know the product.

Sign in to comment

Keyboard shortcuts

Choose a post with J and K or the arrow keys, then press Enter.

K ↑ / J ↓
Previous post / next post
Enter C
Open summary and comments for the selected post
O
Open the original link of the selected post
/
Search
?
Show this help
Tab
Move to the submit or comment button, then press Enter

Type normally in text fields. Tab and Enter are always available.