5c8d773ce1943f9501f3a02b69b1361b69ba717b
Let the crawler console set a backfill target per source. Each scheduled run pages backwards from a checkpoint for at most 20s so real-time crawling is not blocked. The checkpoint is saved only after a successful run and is also kept in process memory, because the next trigger may be queued before the database write lands. cls_depth is excluded since its list API cannot page into history.
Languages
Python
69.8%
Vue
18%
TypeScript
11.3%
CSS
0.6%
Shell
0.1%
Other
0.1%