CCF Observatory
A lighthouse on Canada’s climate coverage
0 %
Lighting the lighthouse…
Chapter 06 · Database & method

One article, followed
from ingestion to the observatory.

The same path every article takes: ingested with its metadata, read sentence by sentence, annotated and validated, compared with the rest of the corpus, enriched in both languages, then published.

6steps
65categories
2languages
4×a day
Chapter 06 · One article, end to end

The same six steps, four times a day

An article enters the corpus from our press archive databases, is read sentence by sentence, annotated, checked against human coding, compared with the rest of the corpus, summarised in both languages, then served to the observatory. Each step opens the chapter that documents it.

01 Ingested with its metadata The article arrives from our press archive databases with its outlet, date, byline and full text. Four collection passes a day cover all 22 outlets; when two of our databases deliver the same piece, a fuzzy title and body comparison keeps one of them. Chapter 02 · The sources 02 Read sentence by sentence The text is cut into two-sentence units, and every unit is scored on the 65 annotation categories. People, organisations and places are extracted in the same pass. The corpus holds 10,031,575 such units. Chapter 03 · The method 03 Checked twice by people The models are measured against a gold standard coded by hand and kept out of training; a second annotator then re-codes that standard blind. Each category keeps its own reliability tier, including the ones we publish as fragile. Chapter 05 · Validation · macro F1 = 0.866 04 Compared with the rest of the corpus Sentence embeddings place the article beside the ones it resembles: clustering turns them into events, then into the cascades that carry them from one outlet to the next. The four most recent days are served as provisional and settle as they age. Chapter 04 · What we extract 05 Titled and summarised in both languages A language model running on our own servers writes an English and a French title and summary for the article, its event and its cascade. It also names the locality, which a gazetteer places on the map. Real-time observatory 06 Served to the observatory The article joins the day’s thread, the media pulse, the cards of the entities it quotes, the Climate Journal, the daily newsletter and the Bluesky thread. All 287,227 articles in the corpus stay queryable through the API. The observatory
Inside the article

What the models attribute to one sentence

The bench isolates a single sentence so its annotations stay legible. In production the same pass covers every sentence of the article, before corpus comparison, event and cascade detection, bilingual enrichment and publication.

Classification examples Four sentences · recorded annotations
Input sentence

Categories attributed
Named entities

The classification bench — one sentence, its annotations. Four sentences from English-language coverage, with the categories and entities the models attributed to them. The four passes light up in the order the chain runs them: tokenising, embeddings, classifiers, entity extraction. The annotations shown are recorded rather than recomputed when the page opens; the word underlined in green is the one the entity extractor kept. Click a number to see another sentence.

The last step is public

Everything the six steps produce is served on the observatory: the day’s article thread and its summaries, the events and cascades detected, the pulse of the frames, the cards of the entities quoted. Chapter 04 lists the sixty-odd annotation categories; chapter 05 gives the validation figures, category by category.

Open the observatory ✦ · All 65 categories · Validation

Final step The live observatory
Go to the observatory