Commit Graph
11 Commits
Author SHA1 Message Date
eezd 89c729092c Add resumable US policy news collectors and provenance
Development CI / check-and-build (push) Successful in 2m54s
GitHub Git mirror / mirror (push) Successful in 9s
Development CI / deploy-development (push) Failing after 4m24s
Collect official crypto and US policy documents with deterministic screening, full bodies and attachments, bounded retries, and durable history cursors. Preserve publication precision and source identity while keeping new collectors disabled.

Add real Congress, GovInfo, DOJ and Federal Register fixtures, offline capture replay, isolated database validation, and rollout instructions. Validate 982 backend tests, frontend checks, and 57 deployment control tests.
2026-11-04 21:41:04 +08:00
eezd e2157443b7 Bound CLS cleanup time and support stopped-instance retries
Development CI / check-and-build (push) Successful in 1m51s
GitHub Git mirror / mirror (push) Successful in 6s
Development CI / deploy-development (push) Failing after 6s
2026-11-04 16:59:08 +08:00
eezd fcd1cfc344 Remove CLS collection and support a full news AI reset
Development CI / check-and-build (push) Successful in 1m49s
GitHub Git mirror / mirror (push) Successful in 5s
Development CI / deploy-development (push) Failing after 7s
2026-11-04 16:47:29 +08:00
eezd 80d266dba7 Add three-stage cross-domain news research and reading briefings
Group original news into tagged fact packages, review each package with evidence and read-only research, and build complete 72-hour reading lists with independent model selections.

Add paused RSS/Atom subscriptions, news-scoped Skills, versioned analysis records, and locked deployment support for SearXNG. Preserve original news and keep live migration and legacy AI cleanup separately authorized.

Validate with 604 backend tests, 49 frontend tests, 19 CI control tests, lint, type checks, production build, and disposable PostgreSQL migration and three-stage smoke checks.
2026-10-31 22:08:59 +08:00
eezd e8ca4daa0e Add official US and European collection and calendar storage
Collect official RSS and Atom summaries, preserve historical captures, and poll macro releases with bounded cadence and Retry-After handling. Separate news collectors from the economic calendar worker.

Track statistical periods, publication stages, first observations, revisions, and current official series. Include explicit migration files and isolated archive, parsing, and import tests; no migration or import is executed by this commit.

Keep dependency declarations and lockfile together for PDF archives and the upcoming captcha integration. Verify the staged snapshot with 90 focused tests.
2026-10-26 12:03:00 +08:00
eezd 5fa1e37fd4 Preserve public summaries for unavailable historical paid articles
Validate Wallstreetcn business responses and retain list provenance for historical paid articles whose detail API returns code 50000. Keep other failures explicit and cover the boundary with eight isolated tests.
2026-10-26 12:01:29 +08:00
eezd 77b33e5e77 Refresh news metadata independently of content revisions 2026-10-20 18:51:39 +08:00
eezd 285b3952d7 Support selectable demo exchanges and configurable analysis limits
Persist manual requests until exchange confirmation, preserve stop protection, and reconcile fills before settling plans. Tighten model stream, crawler, skill, research accounting, and frontend race handling, with regression coverage. Keep queued analyses pending until a fresh account snapshot is available.
2026-10-20 18:45:00 +08:00
eezd 5c8d773ce1 Add resumable news history backfill
Let the crawler console set a backfill target per source. Each scheduled
run pages backwards from a checkpoint for at most 20s so real-time
crawling is not blocked. The checkpoint is saved only after a successful
run and is also kept in process memory, because the next trigger may be
queued before the database write lands. cls_depth is excluded since its
list API cannot page into history.
2026-10-13 14:58:48 +08:00
eezd 554d224be2 Harden background loops and API edge cases from code review
- crawler manager: retry transient errors inside each loop, never let
  log writes break supervision, run tasks in a TaskGroup so any escaped
  error restarts the container instead of leaving a half-dead process
- market worker: tolerate symbols deleted mid-backfill, retry task
  selection and symbol queries, run loops in a TaskGroup
- market WS: validate subscription messages and close the connection
  when either the receive or forward side ends
- news pipeline: make first insert concurrency-safe with ON CONFLICT
- market API: only proxy ticker/1s klines for added symbols, map Binance
  rate limits to 503 and upstream errors to 502 without blocking
- news API: escape LIKE wildcards in search, return 422 for bad cursors
- market page: cancel pending reconnect timer on unmount
2026-10-13 13:51:00 +08:00
eezd aabc81ba5b Initial AlphaPulse implementation: news crawler and Binance klines
- Backend (FastAPI, SQLAlchemy 2, Alembic): single-admin cookie auth,
  news/crawler/market APIs with SSE and WebSocket streaming
- Crawler (Scrapy + Scrapy-Redis): wscn_news, wscn_live, cls_depth,
  cls_telegraph spiders, idempotent upserts with revisions, raw response
  archiving, per-source subprocess supervision, restart and backoff
- Market worker: full-history Binance spot/USD-M kline backfill,
  realtime WS stream, hourly and on-reconnect catch-up
- Frontend (Vue 3, Element Plus, UnoCSS, KLineChart): news feed,
  crawler console with live logs, Binance-style chart page
- Deploy: docker compose, nginx site, migrate and daily backup scripts
2026-10-13 13:32:55 +08:00