Goodfire says its new ‘inside-out’ monitors catch rogue AI agents at a fraction of the cost

Goodfire unveiled a new monitoring system that inspects internal model computations to flag potentially dangerous actions by AI agents. Rather than having a separate model re-read every token an agent produces, Goodfire's probes tap into the intermediate activations computed during the model's forward pass and run lightweight classifiers on those signals. When a probe detects a concerning pattern, a separate model or human review is invoked. The monitors are being offered to customers of Baseten, the hosting platform that recently announced a safety partnership with Goodfire and Hugging Face. Baseten users can select which risk categories to watch for — including offensive hacking, chemical and biological weapon misuse, and reward hacking — and pick automated responses such as logging, human review, or outright refusal of a request. Goodfire reported cost and performance figures from tests on Kimi K3: monitoring roughly 1,500 sessions with its probes cost about $51, versus $233 for a less expensive external monitor and roughly $10,000 for a premium model-checking system. In those tests the probes detected 94% of malicious hacking sessions while forwarding 8.7% of benign sessions for further inspection. Running four probes concurrently added less than 2% to the model's startup latency, according to the company. The product targets open models, which developers can download and modify and which generally lack the built-in safeguards of closed systems. Goodfire's founders say internal monitoring can catch problematic behavior earlier in evaluation or training and that this technique complements longer-term research goals to trace behaviors back to their origins in training data and dynamics. The company noted broader industry precedent for the approach, pointing to misuse-detection probes informed by Google DeepMind research.