Daily Crawl Statistics

30 days · 2026-08-22 → 2026-09-20 (SGT) · rendered 2026-09-20 19:46 SGT
1279
New SWE postings (7d)
158540
Postings archived (7d)
10379
Active postings
0
Enrich backlog

Last run: enrich · 2026-09-20 03:12 SGT · success · unmapped tech terms: 9295

1. New SWE postings per day

319008-22: 31908-2208-23: 5208-24: 4808-25: 29508-26: 31508-2608-27: 25908-28: 30508-29: 27708-30: 4408-3008-31: 3309-01: 25609-02: 26709-03: 24409-0309-04: 27709-05: 21109-06: 3109-07: 3409-0709-08: 28909-09: 22009-10: 24509-11: 26109-1109-12: 26209-13: 2609-14: 2809-15: 23309-1509-16: 24509-17: 26709-18: 25709-19: 21409-20: 3509-20

2. Daily crawl detail

Date (SGT)RunsStatusIngest timePages ArchivedNewSWEUpdated ClosedErrorsLLM calls/hits
2026-09-20 reconcileenrich success 27m48s 928 82440 69 35 10169 2672 0 36 / 0
2026-09-19 increnrich success 4m24s 147 14700 424 214 1230 0 0 185 / 29
2026-09-18 increnrich success 4m31s 151 15100 466 257 1230 0 0 218 / 39
2026-09-17 increnrich success 4m43s 159 15900 511 267 1187 0 0 225 / 42
2026-09-16 increnrich success 4m01s 137 13700 474 245 780 0 0 213 / 33
2026-09-15 increnrich success 2m30s 86 8600 425 233 269 0 0 195 / 37
2026-09-14 increnrichreport success 2m26s 81 8100 64 28 664 0 0 12 / 16
2026-09-13 reconcileenrich success 28m04s 940 83730 53 26 10116 2317 0 23 / 5
2026-09-12 increnrich success 4m27s 147 14700 479 262 1141 0 0 230 / 33
2026-09-11 increnrich success 4m30s 150 15000 480 261 1074 0 0 233 / 28
2026-09-10 increnrich success 4m36s 154 15400 433 245 1218 0 0 215 / 30
2026-09-09 increnrich success 3m36s 121 12100 418 220 860 0 0 200 / 21
2026-09-08 increnrich success 2m25s 83 8300 526 289 300 0 0 265 / 24
2026-09-07 increnrichreport success 2m16s 76 7600 78 34 587 0 0 26 / 8
2026-09-06 reconcileenrich success 27m28s 922 82197 60 31 9834 2483 0 31 / 2
2026-09-05 increnrich success 4m29s 149 14900 426 211 1205 0 0 196 / 17
2026-09-04 increnrich partial 4m40s 157 15700 527 277 1180 0 26 294 / 11
2026-09-03 increnrich partial 4m43s 158 15800 473 244 1197 0 3 242 / 2
2026-09-02 increnrich partial 3m40s 124 12400 511 267 731 0 3 207 / 63
2026-09-01 increnrich success 2m38s 91 9100 453 256 300 0 0 178 / 79
2026-08-31 increnrichreport success 2m43s 92 9200 81 33 750 0 0 21 / 12
2026-08-30 reconcileenrich success 27m11s 910 81089 84 44 9762 2207 0 23 / 21
2026-08-29 increnrich success 4m36s 155 15500 543 277 1289 0 0 170 / 107
2026-08-28 increnrich success 4m33s 154 15400 562 305 1311 0 0 247 / 60
2026-08-27 increnrich success 4m59s 168 16800 525 259 1456 0 0 202 / 57
2026-08-26 increnrich success 3m56s 134 13400 614 315 910 0 0 258 / 56
2026-08-25 increnrich success 2m49s 97 9700 585 295 357 0 0 230 / 67
2026-08-24 increnrichreport success 2m32s 87 8700 85 48 865 0 0 38 / 10
2026-08-23 reconcileenrich success 25m26s 867 77647 116 52 8910 2402 0 42 / 11
2026-08-22 increnrich success 4m27s 150 15000 619 319 1166 0 0 233 / 86

Archived = every posting written to raw/ (all categories, before filtering). New = candidate postings first stored that day; SWE = the subset classified as software engineering. Closed = postings whose closed_at landed that day.

3. Tech stack in postings crawled (last 7 days)

python404sql340aws337azure273java261javascript252git201docker194kubernetes189linux180react163machine-learning159google-cloud149typescript135llm117
Counters come from ingest_run; stored/closed counts from job, bucketed by SGT calendar day. Numbers are computed by SQL from public MyCareersFuture data. Methodology: docs/03-data-model.md · Compliance: public job and company data only, no personal data.