Automating a weekly digital-policy monitoring report

Problem

Emergent Public Affairs runs government policy monitoring for the Asia Tech Alliance, an industry association whose members include AWS, Apple, Spotify, and Microsoft. The headline deliverable is a weekly monitoring report covering digital-policy developments across 15 Asia-Pacific markets: platform regulation, AI governance, online safety, data privacy, and adjacent areas.

Producing it was manual and expensive. Each week, individual consultants wrote country updates from third-party vendor feeds and their own research, and a project lead compiled the country sections and regional highlights into one report. Three people spent about a day each on it, every week. A typical edition ran 5,000 to 6,000 words, drawn from 50 to 60 source articles, and followed a strict house template: theme tags per market, a short implication line per entry, and a regional highlights narrative.

The constraints shaped everything:

• A two-week turnaround. This was a spike to test feasibility, not a product build. The only question that mattered was whether the output could reach publishable quality.
• 100% article coverage. These are high-value client documents. Every submitted source had to be extracted and accounted for. Silently dropping a paywalled article was not acceptable.
• Messy, multilingual sources. Around a third of articles were not in English. Individual government sources ran as long as 10,000 words and needed standardising before they could feed a report.
• Editorial control and transparency. The client had to be able to see and steer what passed from one stage to the next. The target was to remove the mechanical work, summarisation, first-level analysis, writing, and formatting, without removing the analyst’s judgment.

Approach

The job to be done was narrow: get a consultant from a raw URL list to a house-style report without doing the summarising and formatting by hand, while keeping editorial sign-off.

I scoped the pipeline to three stages with a human checkpoint in the middle: fetch and summarise every source into a reviewable content file, present an outline for analyst confirmation, then generate the branded Word document. I deliberately left out scheduling, source discovery, and any user interface. It runs as a Claude skill from the terminal. At spike stage, a rough operator experience was an acceptable price for answering the only question that mattered, which was whether the output was good enough.

Key decisions and tradeoffs

Summarise on ingest rather than carry full text. Fifty to sixty sources, some of them 10,000 words of government prose, would overwhelm the context window long before report writing began. I chose to standardise each article down to a 300-word summary written to a content file, keep only a lightweight record in active context, and re-read the file when writing each entry. The second-order effect mattered more than the token saving: the content file became the audit record and the editorial-control surface the client wanted. Transparency fell out of a context-management decision rather than being bolted on.

A human checkpoint before any report is written. The pipeline stops after building the outline and waits for the analyst to confirm, move, exclude, or add entries. Running end to end would have been faster and would have defeated the purpose. Because these are high-value documents and the client wanted to own selection and framing, the checkpoint is where their judgment enters. It also catches errors against the source before they propagate into 5,000 words of finished report.

A separate compliance pass for editorial voice. The hard part was never summarising. It was the “What this means” line, the analyst’s read on each development. The model jumped to implications and reached for alarmist framing. One real example: for a proposed social-media identity rule in Indonesia, the generated line read that the move “signals a significant escalation” that “would create new account verification obligations for platforms.” The human version simply noted that the proposal was expected to go through public consultation as the government continued exploring identity-linked accountability measures. In the broader regulatory context it was not a significant escalation, and it was going to consultation regardless. Governments talk constantly; the value of this report is tracking the direction of that conversation so members can respond when consultations actually open, not treating every signal as a deadline.

Inline instructions got me part of the way. The structural fix was to add a dedicated compliance-review step that checks the draft against house rules separately from writing it. That single change, introduced after the third iteration, improved instruction-following across the whole document, not just the implication line. The voice took seven iterations to settle.

Architecture

Three stages, with a markdown content file as the spine.

Stage 1, fetch. For 100% coverage I layered three extraction methods: a web_fetch pass first, a Firecrawl pass for what failed, and on-screen browser control as the last resort, since driving a real browser bypasses most bot detection. Each result is written to the content file immediately, marked fetched or inaccessible. Foreign-language sources are translated at summary time. Extraction is the brittle part of the system, which is exactly why coverage is engineered with fallbacks rather than trusted to one method.

Stage 2, confirm. The skill produces an outline organised by market in a fixed regional order, with proposed category tags and entry titles, and hands over the content file. It then waits.

Stage 3, generate. It re-reads each confirmed source from the content file rather than trusting Stage 1 context, writes the entries and the regional highlights, then renders everything into the branded Word template through direct OOXML manipulation so the named styles, logo, and header survive. Building from scratch would have lost the branding; writing into the template was the only way to get a correctly formatted output.

The leverage point is the content file: source of truth, audit trail, and review surface in one artifact. The known failure mode is extraction. That is also the part with the shortest shelf life. The plan is to retire the scraping layer once this connects directly to Emergent’s monitoring database, at which point the sources arrive structured.

Results

I validated the skill against five historical reports, one per month across five months, by running the same sources through the pipeline and comparing the output against what the team had originally published. Scoring was pass or fail per paragraph. The bar was not a word-for-word recreation; it was an entry that faithfully reflected the source and matched house editorial judgment.

This eval is what caught the over-implication and alarmist-framing problem, and it is what told me when the voice was close enough to ship. After the output cleared review in a single round, we ran the AI process in tandem with the manual one for two weeks, both producing the report side by side, to build consultant trust before anything depended on it.

The skill is expected to save about 10 hours a week of consultant time and was approved for production.

I built the end-to-end pipeline: the staged workflow, the content-file design, the compliance pass, the editorial tuning, and the document generation. The Emergent analysts owned the editorial standard, reviewed the output, and made the call that it was good enough to take forward.

What I’d do differently

I spent too long at the start trying to solve extraction inside the skill from day one. It is a scraping problem with all the usual bot-detection complexity, and it is the exact part being replaced by the database integration. The value lived in output quality and the editorial calibration. Next time I would box extraction behind a simple interface early, accept imperfect coverage during development, and spend the freed time on the eval loop sooner, since the eval was the thing that actually moved quality.

Tech stack: Claude skill Claude in Chrome web_fetch Firecrawl Python (lxml / OOXML) .docx templating markdown content + audit file pass/fail paragraph eval against historical reports