Trust sheet: AI Signal
AI Signal is the daily news tool we run on this site. We use it as the worked example of a trust sheet because it is ours to describe honestly, including the parts that are not proven yet.
Trust sheet
AI Signal: daily draft posts from the AI news
Every morning it reads about thirty public news sources (the frontier labs, the open-source ecosystem, industry press, the governance beat, and research trade press), scores what it finds, and drafts a small set of LinkedIn post options grounded in the most relevant items. Each option connects a few facts and ends with the open questions they leave. It saves the reading, and it is not a substitute for choosing what to say.
- A person reads the options, chooses one, edits it and posts it. Nothing is posted automatically.
- The editorial line is set by us: what counts as a usable fact, which stories matter, what the register should be. The tool has no view of its own to add.
- Every draft can be rated up or down. Ratings are reviewed after a few weeks and used to change the instructions.
- Opinions and interpretation are ours. The drafting instructions tell the tool to connect facts and pose questions, not to assert conclusions.
- Each draft goes through a second AI pass that is separate from the drafting pass. It fetches the pages the draft cites and checks every factual claim against the text of those pages only. Its own knowledge is not allowed to count as evidence.
- Claims the sources do not support are flagged and the wording is corrected. The verdict for each claim is shown next to the draft.
- A draft that could not be checked is labelled "Not fact-checked" rather than shown as if it had passed.
- In testing, the check caught an unsupported statement about what a vendor had shipped, and an example that overstated its source.
Not yet measured: how often the checker itself misses an error, or flags a claim that was in fact supported. Until we have measured that against a reference set, "no issues found" means the check found nothing, not that the draft is verified. That is why this is amber.
The draft text, links to every source it used, a verdict on each claim, a short note on why that option was suggested, and the open questions it raises. Past drafts stay in an archive. The raw news stream behind them is visible on the AI Signal page.
- It processes public news headlines, summaries and article text. It does not process personal data about customers or readers.
- Article text and drafts are sent to Anthropic's API (the Claude models) to draft and to check.
- Drafts, source links, verdicts and ratings are stored on Cloudflare (Workers KV). They are kept until deleted; there is currently no automatic expiry.
- Rating and refresh actions are protected by private tokens. Reading the drafts is public.
- The check can only test a draft against what the cited page says. It cannot tell whether the page is right.
- If a page cannot be fetched (a paywall, or a site that blocks automated access), the check falls back to the feed summary, which is weaker. Some trade publications cannot be reached at all, so coverage leans towards what we can read.
- Ranking is rule-based and has missed important stories before. A cheap open model release once scored below routine product announcements. We corrected the scoring, and the daily set is a shortlist, not a census.
- Each claim is judged once by the checking pass, not by several independent runs, so there is no vote to show where it is unsure. Where the checker is wrong, nothing else catches it except the person reading.
- It is not for publishing without a person reading it, and nothing it produces is legal advice.
Why we publish this: it is the standard we would apply to a client's automation, applied to our own. Read the full approach, or see the tool itself at AI Signal.