feat: organize news packages into verified event timelines

Add optional events, evidence-backed timelines, versioned package corrections and bounded historical reprocessing. Keep independent packages readable and isolate retries by failed stage. Configure reasoning effort per model stage and refresh default news Skill drafts without publishing.

deploy/README.md

@@ -46,15 +46,29 @@ Only resources labeled for the current CI attempt are cleaned; there is no globa
 
 ## Three-stage news research
 
-The `news-risk` service runs three independent loops. Original news and revisions remain in their existing tables; AI packages, package versions, memberships and research evidence are stored separately.
+The `news-risk` service runs three model stages with separate durable queues for grouping, package review, event timelines, reading and explicit reprocessing. Original news and revisions remain in their existing tables; AI packages, package versions, memberships and research evidence are stored separately.
 
 1. **Grouping:** the selected Luna model processes every domain and content type, including political news, statements, data, explanations and opinions. Reposts of the same concrete fact share a package with original links and normalized tags. New developments and opinions form separate packages; topic similarity alone is insufficient. One article can contribute to multiple packages.
 2. **Package review:** a separately selected model receives the package fact, title, category, tags and content type alongside its original articles, so separate facts within one roundup retain distinct review scopes. It records facts, timeline, claims, verification verdicts, counter-evidence, related packages, opinions and watchpoints. Reading importance (1–5) is independent of risk severity (0–5), so major positive developments can enter the important reading list. Each attempt permits four local searches, eight local reads, two external searches and four external reads, across at most eight tool rounds followed by submission. Context planning reserves the research tool definitions before extracting or compacting long articles. News/package/calendar search and SearXNG are read-only. Search snippets and other AI summaries are leads, not independent verification. Quotes must occur in a source actually supplied or read; insufficient evidence must remain explicit. Tool responses, failed attempts and previously successful versions are retained. A manual retry can reuse saved successful tool responses; old failures cannot requeue a revision that already succeeded.
-3. **Reading and trends:** the independently selected advanced model ranks every successful package of importance 3–5 within the last 72 hours. Large inputs are processed in batches; truncated ranking outputs split into smaller batches, while a truncated single-package output fails explicitly. The complete reading list is retained and paginated, with strict package/version coverage checks. Up to three lower-importance packages from different domains provide expansion reading. Synthesis distinguishes facts from inference and gives market implications, 24–72-hour and 1–4-week trends, conditions, counter-evidence and watchpoints. Ordinary changes aggregate for 15 minutes; importance-5 developments can trigger earlier, with at least 60 seconds between starts. Citation-only changes do not repeat advanced-model calls; changed facts, ratings or verification verdicts still trigger updates. Reposts and analysis execution alone do not advance package source-progress timestamps. Window expiry also triggers an update. Failed refreshes retain the previous successful briefing.
+3. **Reading and trends:** the independently selected advanced model ranks every successful event or independent package of importance 3–5 within the last 72 hours. Large inputs are processed in batches; truncated ranking outputs split into smaller batches, while a truncated single-package output fails explicitly. The complete reading list is retained and paginated, with strict package/version coverage checks. Up to three lower-importance packages from different domains provide expansion reading. Synthesis distinguishes facts from inference and gives market implications, 24–72-hour and 1–4-week trends, conditions, counter-evidence and watchpoints. Ordinary changes aggregate for 15 minutes; importance-5 developments can trigger earlier, with at least 60 seconds between starts. Citation-only changes do not repeat advanced-model calls; changed facts, ratings or verification verdicts still trigger updates. Reposts and analysis execution alone do not advance package source-progress timestamps. Window expiry also triggers an update. Failed refreshes retain the previous successful briefing.
+
+An event is optional and requires at least two distinct packages describing the same developing incident or process. A standalone package remains readable and can join an event later. Review must actually read the other packages before proposing membership. Each package has at most one main event; cross-event links remain separate. Event timelines preserve known, partial and unknown occurrence times, reporting times, verification status, original quotes and exact package versions. Unknown occurrence times are never replaced by publication times. A later event refresh cannot publish over changed member versions. Successful events aggregate their represented packages into one reading entry; typed `(kind, id, revision)` references distinguish events from packages.
+
+Package review can request atomic split/merge corrections. Corrections lock and verify input revisions, require all current source assignments and literal evidence, retain old IDs/history, and preserve the earliest ID on merges. Repeating a correction with the same source hashes and Skill rules is rejected to prevent loops. Corrections are versioned separately; superseded package URLs link to replacements. A pure repost only adds a source; new facts, independent evidence and corrections request another review. Original news rows are never rewritten by AI.
+
+The homepage offers a publication-range reprocessing preview followed by explicit cost confirmation. Starting captures the latest source hashes, three model/effort configurations and published Skill contents; it does not reconstruct a historical information state. Jobs yield to pending realtime work, persist position and failure reasons, and resume after restart. Cancellation stops subsequent articles; an in-flight call and already queued package/event reviews may finish. Pause stops new work. Completion waits for package/event review and a reading snapshot; failures retain prior successful outputs. Failures identify their stage and object. Retrying grouping failures reruns only those news items; package review, event timeline and reading failures resume their own stages without repeating successful grouping. Retries can incur additional model costs. Pausing before a grouping call leaves the job checkpoint unchanged, even when an older successful analysis exists. Source revisions invalidate stale job items; select a new range to include the new revision. Shared packages/events may update even when some of their articles fall outside the selected range.
 
 The first successful enable operation records a permanent start timestamp. Only source publications or source updates after that timestamp enter the pipeline; ingestion of older history does not trigger analysis. Pause/resume retains the timestamp. Historical records remain available to research tools. No trading actions are triggered.
 
-Create the three default news Skill drafts in homepage settings, edit and publish them in the Skills page's News tab, then bind the grouping, package-analysis and reading Skills. Templates never overwrite existing rules or publish themselves. News-scoped Skills never enter Chat/trading prompts. Each attempt retains the effective rules and model identity. All three model selections support provider model lists and manual IDs. Test-and-enable makes real validation calls; saving paused settings makes none. Publishing a Skill does not enqueue a historical reanalysis.
+During package review, long-article slices with no facts relevant to the package return empty facts and evidence together and are omitted from compaction. Relevant slices must retain literal source quotes; if every slice is irrelevant, review fails explicitly. Grouping extraction, compaction and final package summaries still require nonempty evidence.
+
+Create or update the three default news Skill drafts in homepage settings, review and publish them in the Skills page's News tab, then bind the grouping, package-analysis and reading Skills. Updating appends a new draft only when the latest content differs; it preserves node IDs, previous versions and published versions. Templates never publish themselves. `POST /api/risk/skill-templates` without a body, or with `refresh_existing=false`, only creates missing templates; the homepage explicitly sends `refresh_existing=true` to prepare updated drafts. News-scoped Skills never enter Chat/trading prompts. Publishing a Skill does not enqueue a historical reanalysis.
+
+All three stages independently select a model and reasoning effort (`low`, `medium`, `high`, `xhigh`, `max`). Model selections support provider model lists and manual IDs. Clearing a stage's effort inherits its provider setting; if both are unset, no effort parameter is sent. Overrides do not change the shared provider. Each attempt retains the selected override, effective effort, rules and model identity; changing either the override or effective settings changes its configuration fingerprint. Test-and-enable makes real validation calls and reports unsupported settings without silently lowering effort; saving paused settings makes no model calls.
+
+The additive `c3d5e7f90123` migration follows `b2c4d6e8f012` and adds three nullable stage-effort columns. Existing settings inherit provider effort. Apply it only with separate migration authorization through the normal deployment lock; it does not update live Skill contents, publish drafts, enable AI or reprocess history.
+
+The subsequent `d4e6f8012345` migration adds events, membership and cross-event links, grouping versions, correction records, reprocessing jobs and active article assignments. It seeds grouping history from existing analyses and pauses AI while leaving raw news and prior successful summaries intact. Both pending migrations require separate authorization under the deployment lock. Update and review the default Skill drafts before enabling the new output contracts; no historical job starts automatically. CI exercises migration and pipeline behavior using synthetic models and a disposable database, including cancelled/failed reprocessing, reposts and correction history.
 
 ### Search and subscriptions