Fewer tools, more decisions: how I filter AI news

The real problem isn't information

Every morning, my inbox brings a new model, a new MCP server, a new agent that "changes everything". Newsletters pile up, and so do links. Reading all of it is impossible. Testing all of it would be worse.

I eventually understood that the scarce resource wasn't information. It was decisions. What do I adopt, what do I let pass, and what do I remove?

I work every day with several coding agents (Claude Code, Copilot CLI) plugged into my tools: tickets, tasks, email, NAS. This setup changes all the time. To keep it useful, I needed a simple filter, not a tech-watch program.

Two ways to fail

The first is adopting too much. Every tool you keep becomes a permanent surface: a config that drifts, a token to renew, one more thing to debug at the worst possible moment. You pay the cost every month. The benefit, often, only once.

The second is adopting nothing. The setup then rots silently. Pinned model IDs get retired by the vendor, defaults no longer match the original intent, and nothing throws an error.

Both failures are expensive. The second one is sneakier, because you don't see it.

What "better" means

Before judging a tool, you need to know what you are looking for. Here are my criteria, in order of priority. When two criteria conflict, the higher one wins.

1. Correctness. An agent that is wrong faster is not better.
2. Security and privacy. What leaves the machine, and to whom.
3. Cost per completed task, not per token. A cheaper model that needs three attempts is not cheaper.
4. Reversibility. How long does it take to roll back?
5. Complexity. The number of moving parts, configs and secrets.
6. Speed. Real, but the least important of the six.

Look at vendor announcements: they mostly talk about cost and speed. Those are precisely the two criteria that matter least here.

The four-question filter

Everything that comes in goes through the same filter: a link, a tool, a model, a workflow idea. We stop at the first no. And the default answer is no: it is up to the new thing to prove itself.

1. Is there a specific pain? A real incident, a recurring friction, an hour lost. "Looks interesting" deserves a bookmark, not a change.
2. Does it replace something, or does it add to the pile? A replacement starts on equal footing. An addition has to beat the cost of carrying it forever.
3. Can we roll back in under an hour? If undoing it requires a migration, it is no longer a trial. It is a commitment.
4. Can we test it in a single session? If the evaluation is a project in itself, it becomes a ticket. We don't start it in the middle of triage.

What fails the filter doesn't go in the bin. It goes on a reading list, or becomes a ticket. It simply changes nothing today.

Not all evidence is equal

The four questions stay the same. The level of evidence, however, depends on the type of change.

• Switching models (moving to the next version): low bar. Check price and availability; it is reversible in one line.
• Changing a prompt or a skill: medium bar. A before/after on a real task, not on a made-up example.
• Changing a workflow: medium bar. One complete end-to-end run, then write the rule down so it doesn't get debated again.
• Adding a tool, an MCP server, a dependency: high bar. A pain that nothing existing covers, and a removal plan.

It is that last category that piles up. Adding a tool to work around a problem created by another tool: that is the most common way to degrade a setup, even though each decision, taken alone, looked reasonable.

Two failures nobody noticed

In September, while rereading my own launch scripts, I found two bugs. Neither threw an error.

The first: my Claude Code launcher defaulted to "opusplan" mode. The intent was "use the best model". In practice, Opus only ran in planning mode, and a smaller model did everything else. For months. I replaced the value with the "opus" alias, which now follows each new release without me touching it.

The second: my Copilot CLI launcher pointed to a model ID the vendor had already retired. The default value was dead. I only found out by testing the IDs one by one.

The lesson is not "check your model names more often". It is that a pinned value with no expiry date is a bug waiting for its deadline.

Removing is part of the job

Adoption gets all the attention. Removal, almost none. Yet that is where breakage hides.

My rule: prefer an alias that tracks the latest version over a pinned number. When no alias exists, the pin must be tested at regular intervals. I only pin where reproducibility truly requires it.

The same goes for documentation. A version number written in a context file becomes wrong at the next update, and nobody checks it again. Better to name the tool and say how to ask it for its version.

And a pin rarely lives in just one place. The day I unplugged a memory MCP server, I archived it cleanly… but an installation guide still listed it. Since then, removing a tool starts with searching for its name across the whole repository, and every reference gets fixed before saying "done".

The traps I avoid

• Chasing benchmarks. They measure someone else's workload. If I don't measure on my own tasks, I don't call it an improvement.
• Switching tools to avoid learning the one already installed.
• Pinning a version by reflex when an alias exists.
• Adopting without a removal plan. Without an answer to "how do we roll back?", it is not a trial.
• Re-judging every month a decision already made. A recorded decision stands until new evidence arrives.

No new ritual

This filter doesn't add a meeting or a dashboard. It plugs into what already runs. My email and task triage already sort incoming items: AI news is just one more category. My regular audit of packages and containers now also checks for outdated models and tools.

Once a quarter, or with each new generation of models, I do a short review. I test every pinned ID, I check that recent changes respected the order of criteria, and I close or shrink the open tickets on the topic.

Continuous improvement of an AI agent setup, in the end, looks like that of any system. The hard part isn't adding. It is knowing how to say no, and remembering to remove.