Commit Graph
3 Commits
Author SHA1 Message Date
eezd fcd1cfc344 Remove CLS collection and support a full news AI reset
Development CI / check-and-build (push) Successful in 1m49s
GitHub Git mirror / mirror (push) Successful in 5s
Development CI / deploy-development (push) Failing after 7s
2026-11-04 16:47:29 +08:00
eezd 5c8d773ce1 Add resumable news history backfill
Let the crawler console set a backfill target per source. Each scheduled
run pages backwards from a checkpoint for at most 20s so real-time
crawling is not blocked. The checkpoint is saved only after a successful
run and is also kept in process memory, because the next trigger may be
queued before the database write lands. cls_depth is excluded since its
list API cannot page into history.
2026-10-13 14:58:48 +08:00
eezd aabc81ba5b Initial AlphaPulse implementation: news crawler and Binance klines
- Backend (FastAPI, SQLAlchemy 2, Alembic): single-admin cookie auth,
  news/crawler/market APIs with SSE and WebSocket streaming
- Crawler (Scrapy + Scrapy-Redis): wscn_news, wscn_live, cls_depth,
  cls_telegraph spiders, idempotent upserts with revisions, raw response
  archiving, per-source subprocess supervision, restart and backoff
- Market worker: full-history Binance spot/USD-M kline backfill,
  realtime WS stream, hourly and on-reconnect catch-up
- Frontend (Vue 3, Element Plus, UnoCSS, KLineChart): news feed,
  crawler console with live logs, Binance-style chart page
- Deploy: docker compose, nginx site, migrate and daily backup scripts
2026-10-13 13:32:55 +08:00