Praxis

Praxis Validation Notebooks

Audit security-ML results. Decide how much of a published score you believe—and defend that judgement with evidence you produced yourself.

36teaching notebooks

1doctoral exemplar

34schema cards

10rubric criteria

2 mindefence

1canonical defense atlas

What this page offers

Jump to the part you need. Long guidance is optional—open a section only when you want detail.

1 · Orientation Start here Rules of the game in one screen. 2 · Assignment What you do Two-minute defence · validation steps · timing. 3 · Scoring Review rubric 10 criteria · bands · Part B probe cell. 4 · Positive control Doctoral 30/30 Repaired protocol vs teaching nb04 · HTML/PDF/ipynb. 5 · After class Answer sheets One canonical 767-page defense atlas. 6 · Catalogue 36 teaching notebooks Jump index + full cards (HTML · PDF · .ipynb).

Start here

Some notebooks are wrong on purpose. Others are sound. You are not told which.
Grade the audit, not the headline score. An honest “F on the data” can be strong work.
False positives cost more than missed faults—do not invent defects that are not there.
One assigned notebook → run probes → two-minute verdict with the rubric.
Answer sheets and the 30/30 exemplar come after your own audit—not instead of it.

The assignment

Read · run · interrogate one notebook · present a two-minute verdict. Score with 10 criteria (Part A by reading, Part B by probes you run).

Open rubric Two-minute script How to validate Pick a notebook

Part A vs Part B (detail)

Part A (6 points) is answerable by reading: provenance, task definition, baseline comparison, whether the result rests on a single column, whether the conclusion's numbers appear in the output, and whether the scope is stated.

Part B (4 points) cannot be answered by reading. It requires running a probe whose four results appear nowhere in any notebook: reproduction, grade stability under reseeding, sampling artifacts, and how deep the shortcut goes.

At least one of the two criteria you present must come from Part B.

Criterion 1 (every notebook): who published the file the loader downloads, and is that the party the notebook cites? Each card shows uploader vs credited authors.

How to validate (6 steps)

Establish provenance. Confirm the corpus matches the prose and loader claim.
Reproduce. Run every cell. Mismatch is your first finding.
Attack the headline. Part B probe: re-seed, full corpus vs sample, contamination window, drop top features.
Separate the axes. Shortcut vs contaminated split are different failures.
Score and band. Total /10 → quote / quote with caveats / do not quote / not evidence yet.
Name what you could not settle. End with the next experiment, not a number alone.

Your two minutes (say this)

Time	What you say
0:00–0:20	Claim, corpus, headline number.
0:20–0:45	Total /10 and verdict band.
0:45–1:30	Two criteria that moved the score most (with numbers). ≥1 from Part B.
1:30–2:00	The single next experiment that would settle the rest.

Slides optional. The clock is not.

Doctoral exemplar (30/30)

Positive control for teaching nb04 on the same CIC-IDS2018 day. Protocol repair only—not a different corpus. Inline tables document every delta that moves doctoral axes 7 → 30.

Read HTML (vs-nb04 diffs) PDF .ipynb Full catalogue card ↓

Canonical answer sheet

Post-assignment reference. Use after your own audit.

Combined doctoral defense atlas (PDF)

The canonical 767-page student edition: rulings, axes, faults, repairs, and proof locators for all 36 notebooks.

Open PDF Download PDF

Data policy (short)

No dataset rows are hosted here—only schema cards (columns, types, label mix). Notebooks fetch from original sources on first run. SWaT (nb33) is request-gated by iTrust.

Why (licensing detail)

Third-party research datasets. Most require accepting terms on Kaggle before download; re-hosting would bypass licensor consent. SWaT requires a signed access agreement.

Notebook jump index

Click a code to jump to its card. Every card has HTML · PDF · .ipynb.

Exemplar

30/30

Teaching

nb01 nb02 nb03 nb04 nb05 nb06 nb07 nb08 nb09 nb10 nb11 nb12 nb13 nb14 nb15 nb16 nb17 nb18 nb19 nb20 nb21 nb22 nb23 nb24 nb25 nb26 nb27 nb28 nb29 nb30 nb31 nb32 nb33 nb34 nb35 nb36

Catalogue — full notebook cards

1 doctoral exemplar + 36 teaching studies. Expand schema on a card if you need column names before downloading data.

Doctoral exemplar

30/30 Doctoral-Grade Protocol: CIC-IDS2018 Web Attacks (30/30)

Web application attacks · doctoral exemplar 1,048,575 rows x 67 features (Dst Port excluded); attack rate 0.00054; train/val/locked-test with attack-aware temporal group split.

Corpus. Same CSE-CIC-IDS2018 Friday 2018-02-23 CICFlowMeter day as nb04, repaired evaluation protocol (not a different corpus). no credentials required

Provenance chain. Fetched from AWS Open Data s3://cse-cic-ids2018/ (same day CSV as nb04) — the authoritative distribution.

What it teaches

What a fully repaired evaluation looks like versus the teaching nb04 template: train/val/locked test, temporal+group-disjoint splits, AP as primary metric, FPR-budget threshold, class weights, bootstrap CI, full family ledger, dataset hash. Inline tables document every delta that moves doctoral axes from 7/30 to 30/30.

How it works

Attack-aware temporal + 6-decimal group-disjoint partitions; models selected on validation Average Precision only; threshold chosen on validation for FPR≤1e-3; locked test evaluated once. Class weights / scale_pos_weight from train only. Bootstrap 95% CI on test AP. Per-family ledger prints every label with Wilson intervals.

Why it matters

The series shows how easy it is to publish a high AUC under a broken protocol. This notebook is the positive control: same data, protocol that clears doctoral axes D1–D6 at 5/5 each (30/30). It still does not claim production WAF detection.

Relevance and limits

Reference implementation for doctoral evaluation standards. Use it as the template when repairing any of the 36 teaching notebooks.

What you are looking for. Nothing is planted as a hidden defect. The honest residual is stated outright: single CIC day, CICFlowMeter feature limits (Engelen/Rosay), no cross-day transfer, and operating-point precision can be low even when ranking AP looks excellent. That is the point of reporting both.

Check first: Open the opening section “What makes this 30/30 — differences from teaching nb04” and the per-section vs teaching callouts. Compare selection surface (val AP vs test ROC), Dst Port presence, family ledger policy, and doctoral total 7→30 against teaching nb04 on the same SHA-256 cache.

Read notebook PDF .ipynb ↑ Index

Teaching notebooks

nb01 Network Intrusion Detection on the KDD Cup 1999 Corpus

Network intrusion 4,898,431 rows x 40 features, attack rate 0.8014 — attack is the majority class here, not the minority.

Corpus. Per-connection TCP/IP summary records synthesised from the 1998 DARPA military testbed, fetched through scikit-learn's mirror of the UCI KDD archive. no credentials required

Provenance chain. Fetched from scikit-learn, mirroring the UCI KDD Archive — the authoritative distribution.

What it teaches

How to distinguish a saturated benchmark from a working detector. You run an ablation that strips each suspected artifact, then read the result when the score refuses to move. That null result is the finding, not a failed experiment.

How it works

Four learners — logistic regression, random forest, XGBoost, LightGBM — fit a 120,000-row stratified subsample and score a 1,224,608-row held-out split. The audit computes each feature's univariate ROC-AUC, the corpus duplicate rate, and the fraction of held-out rows that are exact duplicates of training rows. The ablation re-splits and refits the winner twice: once on the de-duplicated corpus, once with the strongest single feature removed. A per-family recall panel then decomposes the binary score by attack family, worst first.

Why it matters

KDD99 still anchors published NIDS comparisons, so its failure modes propagate into results people cite today. Removing 78% duplicate rows and dropping the top feature leaves the AUC at 0.999993 or above. The separability lives in the simulator, not in the detector, and no amount of cleaning recovers a deployment estimate.

Relevance and limits

This stands in for every benchmark whose traffic was generated rather than captured, and for the habit of reporting one aggregate AUC on one random split. It cannot stand in for deployment evidence: no temporal split, no cross-distribution test, and not the corrected NSL-KDD protocol. The 1998 capture predates ubiquitous TLS, cloud tenancy, and encrypted C2.

What you are looking for. Train/test exact-row contamination is 0.6885, so most of the held-out split was already memorised, and the notebook grades its own data F for it. The subtler defect sits in the per-family recall table: several attack families fall below a hard count-of-5 floor and are dropped with no printed warning. The ones that vanish are the rare R2L/U2R classes the conclusion nominates as the honest lead numbers.

Check first: Count the families plotted in the recall chart against the families present in the labels, and ask whether the conclusion survives the ones that vanished.

Read notebook PDF .ipynb ↑ Index

nb02 CIC-IDS2018 Case Study — FTP/SSH Brute Force

Credential brute force 1,048,575 rows x 68 features, attack rate 0.3633.

Corpus. One capture day (2018-02-14) of CSE-CIC-IDS2018, as CICFlowMeter bidirectional flow records, streamed from the public AWS Open Data bucket. no credentials required

Provenance chain. Fetched from AWS Open Data, the CIC's own distribution — the authoritative distribution.

What it teaches

How to spot a feature that names the attack instead of describing it. The day's attacks are FTP on port 21 and SSH on port 22, and `Dst Port` stays in the matrix. You learn to separate a shortcut with physical justification from one that merely encodes the capture schedule.

How it works

Four learners fit a 120,000-row stratified subsample; RandomForest wins on a 262,144-row held-out split at ROC-AUC 1.000000. The audit computes univariate ROC-AUC per feature (`Fwd Seg Size Min`, 0.9984), the duplicate rate 0.356, and train/test exact-row overlap sampled from 50,000 held-out rows. The ablation refits the winner on the de-duplicated corpus and again with that top feature dropped. Recall is reported separately for `FTP-BruteForce` and `SSH-Bruteforce`.

Why it matters

Brute force is the attack class operators actually see daily, and the class most often reported at AUC 1.000000. Both ablations leave the score at 1.000000, so no single removable artifact explains it. A score that survives every correction you know how to apply is a harder problem than leakage.

Relevance and limits

IDS2018 is the reference corpus for supervised NIDS work published since 2018, and Engelen et al. (2021) documented defects in the extractor that produced these columns. It cannot stand in for a modern key-based or MFA-protected estate, and it says nothing about slow, distributed credential stuffing.

What you are looking for. Train/test exact-row contamination is 0.3288 and the notebook grades its data D for it. The provenance section calls `Dst Port` a name for the attack class rather than a description of behaviour, then promises section 10 will price it. Section 10 surfaces `Fwd Seg Size Min` instead, and the port column is never tested or removed.

Check first: Refit with `Dst Port` and `Protocol` dropped and see whether ROC-AUC 1.000000 survives — the notebook flags the column and then measures a different one.

Schema — 80 columns, label “Label”

Column names and types only. No rows of this corpus are hosted here.

Label distribution (proportions from the first 2,000 rows, not the full corpus): FTP-BruteForce 95.3% · Benign 4.8%

Dst Portint64Protocolint64TimestampobjectFlow Durationint64Tot Fwd Pktsint64Tot Bwd Pktsint64TotLen Fwd Pktsint64TotLen Bwd Pktsint64Fwd Pkt Len Maxint64Fwd Pkt Len Minint64Fwd Pkt Len Meanfloat64Fwd Pkt Len Stdfloat64Bwd Pkt Len Maxint64Bwd Pkt Len Minint64Bwd Pkt Len Meanfloat64Bwd Pkt Len Stdfloat64Flow Byts/sfloat64Flow Pkts/sfloat64Flow IAT Meanfloat64Flow IAT Stdfloat64Flow IAT Maxint64Flow IAT Minint64Fwd IAT Totint64Fwd IAT Meanfloat64Fwd IAT Stdfloat64Fwd IAT Maxint64Fwd IAT Minint64Bwd IAT Totint64Bwd IAT Meanfloat64Bwd IAT Stdfloat64Bwd IAT Maxint64Bwd IAT Minint64Fwd PSH Flagsint64Bwd PSH Flagsint64Fwd URG Flagsint64Bwd URG Flagsint64Fwd Header Lenint64Bwd Header Lenint64Fwd Pkts/sfloat64Bwd Pkts/sfloat64Pkt Len Minint64Pkt Len Maxint64Pkt Len Meanfloat64Pkt Len Stdfloat64Pkt Len Varfloat64FIN Flag Cntint64SYN Flag Cntint64RST Flag Cntint64PSH Flag Cntint64ACK Flag Cntint64URG Flag Cntint64CWE Flag Countint64ECE Flag Cntint64Down/Up Ratioint64Pkt Size Avgfloat64Fwd Seg Size Avgfloat64Bwd Seg Size Avgfloat64Fwd Byts/b Avgint64Fwd Pkts/b Avgint64Fwd Blk Rate Avgint64

… and 20 further columns.

Read notebook PDF .ipynb ↑ Index

nb03 CIC-IDS2018 Case Study — DoS (GoldenEye / Slowloris)

Denial of service 1,048,575 rows x 68 features, attack rate 0.0501.

Corpus. One capture day (2018-02-15) of CSE-CIC-IDS2018 as CICFlowMeter flow records, streamed without credentials from the public AWS Open Data bucket. no credentials required

Provenance chain. Fetched from AWS Open Data, the CIC's own distribution — the authoritative distribution.

What it teaches

How to audit the provenance of a file, not only the model fitted to it. The question is whether the rows you loaded are the population you believe you sampled. A row count is itself a measurement, and here it is the most informative one printed.

How it works

Four learners fit a 120,000-row stratified subsample of the training half, then score the whole 262,144-row held-out split. Every figure is therefore a full-holdout measurement of a subsample-trained model, and RandomForest wins it at ROC-AUC 1.000000. The audit prints the strongest univariate feature (`Fwd Seg Size Min`, AUC 0.9852), duplicate rate 0.156, and train/test overlap 0.1042, yielding trust grade C. The ablation refits after de-duplication, which changes nothing, and after dropping that feature, which moves the AUC to 0.999769.

Why it matters

GoldenEye and Slowloris perturb flow timing, so a detector leaning on volume or port has learned the schedule rather than the attack. The notebook states outright that `Dst Port` encodes CIC's published attack windows, and that it does not quantify the effect. Judging whether a named but unmeasured confound invalidates a result is the reviewing skill this exercises.

Relevance and limits

This day is the standard low-and-slow DoS slice in the IDS2018 literature and underlies many published timing-based detection claims. It cannot stand in for volumetric DDoS at carrier scale, nor for application-layer abuse against a load-balanced or CDN-fronted service.

What you are looking for. The provenance table checks the one ceiling it controls, declares the loader's 1.2M-line cap non-binding, and stops there. But 1,048,575 rows plus one header is exactly 2^20, the spreadsheet worksheet ceiling, and two other days here stop at the same count. Three different capture days cannot coincidentally hold identical flow counts, so the printed attack rate 0.0501 describes whatever the export kept.

Check first: Print the first and last `Timestamp` in the loaded frame and compare them against CIC's published 2018-02-15 schedule.

Schema — 80 columns, label “Label”

Column names and types only. No rows of this corpus are hosted here.

Label distribution (proportions from the first 2,000 rows, not the full corpus): DoS attacks-GoldenEye 97.8% · Benign 2.3%

… and 20 further columns.

Read notebook PDF .ipynb ↑ Index

nb04 CIC-IDS2018 Case Study — Web Attacks (Brute Force / XSS / SQLi)

Web application attacks 1,048,575 rows x 68 features, attack rate 0.0005.

Corpus. One capture day (2018-02-23) of CSE-CIC-IDS2018 as CICFlowMeter flow records from the AWS Open Data bucket, carrying three distinct web-attack labels. no credentials required

Provenance chain. Fetched from AWS Open Data, the CIC's own distribution — the authoritative distribution.

What it teaches

How much of a metric is real when the positive class numbers in the hundreds. You learn to read an interval into a point estimate, and to notice when one learner in a comparison has silently failed rather than merely lost.

How it works

Four learners fit a 120,000-row stratified subsample and are scored on a 262,144-row held-out split; XGBoost wins at ROC-AUC 0.996349. The audit prints the strongest univariate feature (`Bwd Pkts/s`, AUC 0.7457), duplicate rate 0.132 and train/test overlap 0.0884, for a data-trust grade of B. The ablation refits after de-duplication, reaching 0.999049, and after dropping that feature, reaching 0.994340. Recall is broken out across `Brute Force -Web`, `Brute Force -XSS` and `SQL Injection`, and 3-fold CV returns 0.9746 +/- 0.0352.

Why it matters

Web attacks are the rarest and highest-consequence class in most enterprise captures, and rarity is what breaks evaluation rather than modelling. This model scores ROC-AUC 0.996349 while recalling 0.556 of SQL injections. That gap is exactly where aggregate metrics and incident response stop agreeing.

Relevance and limits

This is the canonical test of whether flow-level features can see application-layer attacks at all, at a base rate close to a real enterprise. It cannot stand in for WAF or HTTP-log detection, which read the payload that flow features discard by construction.

What you are looking for. Nothing leaks here: single-feature AUC 0.7457 grades A, contamination 0.0884 grades B, and the data-trust grade is B. The fragility is statistical: 262,002 of the 262,144 held-out rows are benign, so 142 positives carry recalls of 0.536, 0.556 and 0.852. LightGBM meanwhile lands at ROC-AUC 0.514258, effectively random, and sits in the ranking table without comment.

Check first: Ask what a recall of 0.556 means across tens of events, then read the CV spread of +/- 0.0352 against the headline point estimate 0.996349.

Schema — 80 columns, label “Label”

Column names and types only. No rows of this corpus are hosted here.

Label distribution (proportions from the first 2,000 rows, not the full corpus): Benign 71.9% · Brute Force -Web 18.1% · Brute Force -XSS 7.5% · SQL Injection 2.5%

… and 20 further columns.

Read notebook PDF .ipynb ↑ Index

nb05 Intrusion Detection on a 5G Network: the 5G-NIDD Dataset

5G network intrusion 1,215,890 rows x 46 features, positive rate 0.6071.

Corpus. Labelled flow records captured on a functional 5G test network (Samarakoon et al., 2022), distributed as a single combined CSV via Kaggle. Kaggle: humera11/5g-nidd-dataset

Provenance chain. Fetched from the Kaggle account humera11. The corpus is credited to Samarakoon, S. et al. Criterion 1 asks you to establish whether those are the same party, and what follows if they are not.

What it teaches

How to read an ablation that moves the wrong way. De-duplication is meant to remove contamination and lower the score; here it does the opposite. You learn to demand a printed measurement for the explanation, not a plausible sentence.

How it works

Four learners fit a 120,000-row stratified subsample and score a 303,973-row held-out split; LightGBM wins at ROC-AUC 0.874514, with 3-fold CV 0.8742 +/- 0.0031. The loader drops `Attack Type` and `Attack Tool` before the split to prevent trivial label leakage. The audit prints the strongest univariate feature (`SrcTCPBase`, AUC 0.6657, grade A), duplicate rate 0.692, and train/test exact-row overlap 0.6008, grade F. The ablation refits the winner on the de-duplicated corpus and again with `SrcTCPBase` removed.

Why it matters

Very few public captures come from real 5G infrastructure rather than a simulator, so this is the closest thing to ground truth for cellular NIDS. It is also the only sub-0.9 headline in this group, which makes it look honest and therefore easy to over-trust. What breaks it is not the model.

Relevance and limits

It stands in for detection inside a mobile core, where the operator observes flows but not endpoints, and it is the reference corpus for 5G-specific IDS work. It cannot stand in for RAN-layer or signalling-plane attacks, and it remains one testbed with one traffic mix.

What you are looking for. De-duplication — the single run that removes the contamination the F grade rests on — raises the held-out AUC from 0.874514 to 1.000000. The conclusion explains this by asserting that identical feature vectors carried conflicting labels and were collapsed to one, making the de-duplicated task easier. No count of conflicting-label duplicates is printed anywhere in the notebook, so the explanation is claimed rather than measured.

Check first: The printed false-positive rate at threshold 0.5 is 0.4663 — 55,697 of 119,434 benign flows flagged — a detector no analyst could run.

Schema — 52 columns, label “Label”

Column names and types only. No rows of this corpus are hosted here.

Label distribution (proportions from the first 2,000 rows, not the full corpus): Malicious 97.1% · Benign 2.9%

Unnamed: 0int64Seqint64Durfloat64RunTimefloat64Meanfloat64Sumfloat64Minfloat64Maxfloat64ProtoobjectsTosfloat64dTosfloat64sDSbobjectdDSbobjectsTtlfloat64dTtlfloat64sHopsfloat64dHopsfloat64CauseobjectTotPktsint64SrcPktsint64DstPktsint64TotBytesint64SrcBytesint64DstBytesint64Offsetint64sMeanPktSzfloat64dMeanPktSzfloat64Loadfloat64SrcLoadfloat64DstLoadfloat64Lossint64SrcLossint64DstLossint64pLossfloat64SrcGapfloat64DstGapfloat64Ratefloat64SrcRatefloat64DstRatefloat64StateobjectSrcWinfloat64DstWinfloat64sVidfloat64dVidfloat64SrcTCPBasefloat64DstTCPBasefloat64TcpRttfloat64SynAckfloat64AckDatfloat64LabelobjectAttack TypeobjectAttack Toolobject

Read notebook PDF .ipynb ↑ Index

nb06 IoT/IIoT Intrusion Detection: the Edge-IIoTset Dataset

IoT / IIoT intrusion 2,219,201 rows x 32 features, positive rate 0.2720.

Corpus. Flow and protocol-field records from a layered IoT/IIoT testbed spanning MQTT, Modbus and HTTP with 14 attack types (Ferrag et al., 2022), obtained via Kaggle. Kaggle: sibasispradhan/edge-iiotset-dataset

Provenance chain. Fetched from the Kaggle account sibasispradhan. The corpus is credited to Ferrag, M.A. et al. Criterion 1 asks you to establish whether those are the same party, and what follows if they are not.

What it teaches

What a sound notebook looks like, so the failures elsewhere in the series become legible by contrast. You also learn that a clean validity audit does not certify a usable detector. Here the audit passes and the per-family recall is still where the operational damage sits.

How it works

Four learners fit a 120,000-row stratified subsample and score a 554,801-row held-out split; LightGBM wins at ROC-AUC 0.993167, 3-fold CV 0.9929 +/- 0.0002. The loader drops `Attack_type`, the endpoint-identity columns `ip.src_host`, `ip.dst_host` and the ARP IPv4 fields, plus the HTTP/DNS/MQTT string payloads, before anything is split. The audit prints the strongest univariate feature (`tcp_seq`, AUC 0.7229) and train/test exact-row overlap 0.0326, both grade A. The ablation refits after de-duplication, giving 0.994870, and with `tcp_seq` removed, giving 0.991590; recall is computed for all 14 families.

Why it matters

IIoT devices are constrained and stay in service for a decade, so a detector must hold against traffic it never trained on. This notebook earns its number: no shortcut, no meaningful contamination, and the identity columns are removed deliberately rather than caught by a generic high-cardinality filter. What remains is a genuine detection gap, not an evaluation artifact.

Relevance and limits

It stands in for protocol-aware monitoring of a heterogeneous device estate, where one sensor speaks Modbus and the next speaks MQTT. It cannot stand in for a production OT network. This is a testbed, the split is random rather than temporal, and every device role is fixed.

What you are looking for. Nothing is planted here, and reporting 'nothing found' is the correct answer. Contamination is 0.0326, the strongest single feature reaches only 0.7229 alone, dropping it costs 0.993167 → 0.991590, and an independent two-feature ablation barely moves it either. The real weakness is disclosed rather than hidden: MITM recall is 0.339, so the model misses two thirds of a family it claims to cover.

Check first: MITM recall 0.339 against an aggregate ROC-AUC of 0.993167 — decide which of the two you would put in front of an operator.

Schema — 63 columns

Column names and types only. No rows of this corpus are hosted here.

frame.timeobjectip.src_hostobjectip.dst_hostobjectarp.dst.proto_ipv4objectarp.opcodefloat64arp.hw.sizefloat64arp.src.proto_ipv4objecticmp.checksumfloat64icmp.seq_lefloat64icmp.transmit_timestampfloat64icmp.unusedfloat64http.file_datafloat64http.content_lengthfloat64http.request.uri.queryfloat64http.request.methodfloat64http.refererfloat64http.request.full_urifloat64http.request.versionfloat64http.responsefloat64http.tls_portfloat64tcp.ackfloat64tcp.ack_rawfloat64tcp.checksumfloat64tcp.connection.finfloat64tcp.connection.rstfloat64tcp.connection.synfloat64tcp.connection.synackfloat64tcp.dstportfloat64tcp.flagsfloat64tcp.flags.ackfloat64tcp.lenfloat64tcp.optionsobjecttcp.payloadobjecttcp.seqfloat64tcp.srcportfloat64udp.portfloat64udp.streamfloat64udp.time_deltafloat64dns.qry.namefloat64dns.qry.name.lenint64dns.qry.qufloat64dns.qry.typefloat64dns.retransmissionfloat64dns.retransmit_requestfloat64dns.retransmit_request_infloat64mqtt.conack.flagsobjectmqtt.conflag.cleansessfloat64mqtt.conflagsfloat64mqtt.hdrflagsfloat64mqtt.lenfloat64mqtt.msg_decoded_asfloat64mqtt.msgobjectmqtt.msgtypefloat64mqtt.proto_lenfloat64mqtt.protonameobjectmqtt.topicobjectmqtt.topic_lenfloat64mqtt.verfloat64mbtcp.lenfloat64mbtcp.trans_idfloat64

… and 3 further columns.

Read notebook PDF .ipynb ↑ Index

nb07 Network Intrusion Detection on CIC-IDS2017

Enterprise network intrusion 2,830,743 flows x 70 features, attack rate 0.1970.

Corpus. Eight daily CICFlowMeter captures from a week of profiled enterprise traffic and a staged attack schedule, combined from the Kaggle mirror `chethuhn/network-intrusion-dataset`. Kaggle: chethuhn/network-intrusion-dataset

Provenance chain. Fetched from the Kaggle account chethuhn. The corpus is credited to Sharafaldin, I., Lashkari, A.H. & Ghorbani, A.A. Criterion 1 asks you to establish whether those are the same party, and what follows if they are not.

What it teaches

How to read a near-perfect aggregate score against per-family recall, and why the two can disagree completely. It also teaches ablation as a refutation test: refit after removing the suspected artifact, then see what the score does. A headline that survives is not vindicated, only unexplained by that artifact.

How it works

Four learners train on a stratified 120,000-row subsample and are ranked by ROC-AUC on a 707,686-row held-out split. The majority-class baseline accuracy 0.8030 prints inside the ranking table. The audit scores every feature alone on a 60,000-row sample, computes the whole-corpus exact-duplicate rate, and measures train/test exact-row overlap on the first 50,000 held-out rows; the trust grade is the worse of those two concerns. The ablation then refits the winner twice: once on the de-duplicated corpus, once with the strongest single feature dropped.

Why it matters

CIC-IDS2017 is the most-cited flow IDS benchmark and a documented cautionary tale. This notebook loads the original Kaggle CSVs, not the corrected re-releases of Engelen et al. (2021) or Rosay et al. (2022); those defects are present in every number. XGBoost reaches 0.999948 held-out AUC at a false-positive rate of 0.0008, while Infiltration recall is 0.000.

Relevance and limits

It stands in for enterprise flow NIDS evaluated against a full week of scheduled attacks, the setting most published IDS results occupy. It cannot stand in for deployment. The split is random rather than temporal, so training and test flows come from the same days.

What you are looking for. The per-family recall block skips any family with fewer than five positives in the held-out split, via a bare `if cnt < 5: continue` that prints nothing. Fourteen attack families reach that split; Heartbleed falls under the floor and disappears without a line of output. The chart then plots only the worst nine and best three of the thirteen that remain.

Check first: Count the attack families in the label column, then count the bars in the per-family recall chart.

Schema — 79 columns, label “ Label”

Column names and types only. No rows of this corpus are hosted here.

Label distribution (proportions from the first 2,000 rows, not the full corpus): BENIGN 100.0%

Destination Portint64 Flow Durationint64 Total Fwd Packetsint64 Total Backward Packetsint64Total Length of Fwd Packetsint64 Total Length of Bwd Packetsint64 Fwd Packet Length Maxint64 Fwd Packet Length Minint64 Fwd Packet Length Meanfloat64 Fwd Packet Length Stdfloat64Bwd Packet Length Maxint64 Bwd Packet Length Minint64 Bwd Packet Length Meanfloat64 Bwd Packet Length Stdfloat64Flow Bytes/sfloat64 Flow Packets/sfloat64 Flow IAT Meanfloat64 Flow IAT Stdfloat64 Flow IAT Maxint64 Flow IAT Minint64Fwd IAT Totalint64 Fwd IAT Meanfloat64 Fwd IAT Stdfloat64 Fwd IAT Maxint64 Fwd IAT Minint64Bwd IAT Totalint64 Bwd IAT Meanfloat64 Bwd IAT Stdfloat64 Bwd IAT Maxint64 Bwd IAT Minint64Fwd PSH Flagsint64 Bwd PSH Flagsint64 Fwd URG Flagsint64 Bwd URG Flagsint64 Fwd Header Lengthint64 Bwd Header Lengthint64Fwd Packets/sfloat64 Bwd Packets/sfloat64 Min Packet Lengthint64 Max Packet Lengthint64 Packet Length Meanfloat64 Packet Length Stdfloat64 Packet Length Variancefloat64FIN Flag Countint64 SYN Flag Countint64 RST Flag Countint64 PSH Flag Countint64 ACK Flag Countint64 URG Flag Countint64 CWE Flag Countint64 ECE Flag Countint64 Down/Up Ratioint64 Average Packet Sizefloat64 Avg Fwd Segment Sizefloat64 Avg Bwd Segment Sizefloat64 Fwd Header Length.1int64Fwd Avg Bytes/Bulkint64 Fwd Avg Packets/Bulkint64 Fwd Avg Bulk Rateint64 Bwd Avg Bytes/Bulkint64

… and 19 further columns.

Read notebook PDF .ipynb ↑ Index

nb08 IoT/IIoT Network-Flow Intrusion Detection: CIC-ToN-IoT

IoT/IIoT network flows 4,847,499 flow records x 68 features, positive rate 0.5619.

Corpus. A CICFlowMeter reprocessing of the UNSW ToN_IoT network packet captures, read from the Kaggle mirror `dhoogla/cictoniot`, with no telemetry or OS-log view touched. Kaggle: dhoogla/cictoniot

Provenance chain. Fetched from the Kaggle account dhoogla. The corpus is credited to Moustafa, N. Criterion 1 asks you to establish whether those are the same party, and what follows if they are not.

What it teaches

That a duplicate rate is a property of a matrix, not of a capture, and the two diverge once preprocessing rewrites values. It teaches you to trace every audit statistic back to the object it was computed on. It also teaches how to read a near-balanced corpus, where the base rate flatters nothing.

How it works

Four learners train on a stratified 120,000-row subsample and are ranked on a 1,211,875-row held-out split, against a majority-class baseline accuracy of 0.5619. LightGBM wins at ROC-AUC 0.996751, and the strongest single feature is `Bwd_IAT_Min` at AUC 0.8132. The exact-duplicate rate is 0.307 and train/test exact-row overlap is 0.114, measured on the first 50,000 held-out rows. The ablation refits on the de-duplicated corpus, to 0.995080, and with `Bwd_IAT_Min` dropped, to 0.996740.

Why it matters

The corpus is near-balanced, so the usual base-rate excuse for a high score does not apply and the number needs another explanation. The operational gap is per-family: mitm recall 0.602 at a false-positive rate of 0.0183 across 530,967 benign flows. A flow-only detector also says nothing about whether device telemetry would have caught the same attacks.

Relevance and limits

It stands in for flow-level detection on a single IIoT testbed with the CICFlowMeter feature family, the dominant practice in IoT-NIDS work. It cannot stand in for telemetry-based or host-log detection, and it cannot support a cross-testbed claim. One testbed, one extractor, one random split.

What you are looking for. The loader ends with `X.clip(-1e15, 1e15)`, and ToN_IoT's CICFlowMeter Idle Mean/Std/Max/Min fields sit partly above that bound. Flows that differed only there arrive at the audit already byte-identical, and Idle Max is flattened to a constant and dropped. So the printed 0.307 and 0.114 are upper bounds containing duplication the pipeline made, and the pre-clip rate is never measured.

Check first: Whether the 0.307 duplicate rate is a property of CIC-ToN-IoT or of the clip that runs immediately before it.

Schema — 79 columns, label “Label”

Column names and types only. No rows of this corpus are hosted here.

Label distribution (proportions from the first 2,000 rows, not the full corpus): 0 96.0% · 1 4.0%

Protocolint8Flow Durationint64Total Fwd Packetsint32Total Backward Packetsint32Fwd Packets Length Totalfloat32Bwd Packets Length Totalfloat32Fwd Packet Length Maxfloat32Fwd Packet Length Minfloat32Fwd Packet Length Meanfloat32Fwd Packet Length Stdfloat32Bwd Packet Length Maxfloat32Bwd Packet Length Minfloat32Bwd Packet Length Meanfloat32Bwd Packet Length Stdfloat32Flow Bytes/sfloat32Flow Packets/sfloat64Flow IAT Meanfloat32Flow IAT Stdfloat32Flow IAT Maxfloat32Flow IAT Minfloat32Fwd IAT Totalfloat32Fwd IAT Meanfloat32Fwd IAT Stdfloat32Fwd IAT Maxfloat32Fwd IAT Minfloat32Bwd IAT Totalfloat32Bwd IAT Meanfloat32Bwd IAT Stdfloat32Bwd IAT Maxfloat32Bwd IAT Minfloat32Fwd PSH Flagsint8Bwd PSH Flagsint8Fwd URG Flagsint8Bwd URG Flagsint8Fwd Header Lengthint32Bwd Header Lengthint32Fwd Packets/sfloat32Bwd Packets/sfloat32Packet Length Minfloat32Packet Length Maxfloat32Packet Length Meanfloat32Packet Length Stdfloat32Packet Length Variancefloat32FIN Flag Countint8SYN Flag Countint32RST Flag Countint32PSH Flag Countint32ACK Flag Countint32URG Flag Countint8CWE Flag Countint8ECE Flag Countint8Down/Up Ratiofloat32Avg Packet Sizefloat32Avg Fwd Segment Sizefloat32Avg Bwd Segment Sizefloat32Fwd Avg Bytes/Bulkint8Fwd Avg Packets/Bulkint8Fwd Avg Bulk Rateint8Bwd Avg Bytes/Bulkint32Bwd Avg Packets/Bulkint32

… and 19 further columns.

Read notebook PDF .ipynb ↑ Index

nb09 Network Intrusion Detection on UNSW-NB15

Hybrid-traffic network intrusion 2,540,047 records x 41 features, attack rate 0.1265.

Corpus. The four headerless UNSW-NB15_1..4 record files from the ACCS testbed, real benign background traffic mixed with synthesised attacks, via Kaggle `mrwellsdavid/unsw-nb15`. Kaggle: mrwellsdavid/unsw-nb15

Provenance chain. Fetched from the Kaggle account mrwellsdavid. The corpus is credited to Moustafa, N. & Slay, J. Criterion 1 asks you to establish whether those are the same party, and what follows if they are not.

What it teaches

How a near-sufficient single feature survives its own ablation when collinear copies of it remain in the matrix. Dropping one column is a weak refutation test whenever the shortcut lives in the traffic rather than in that column. The skill is deciding which of the two the evidence supports.

How it works

The loader names 49 columns and drops eight: `Label`, `attack_cat`, `srcip`, `dstip`, `sport`, `dsport`, `Stime`, `Ltime`. The printed feature count is 41, so the constant-column filter removed nothing and every other named column reaches the model. Four learners train on a stratified 120,000-row subsample and are ranked on a 635,012-row held-out split; the baseline accuracy is 0.8735 and LightGBM wins at ROC-AUC 0.999671. The audit finds `ct_state_ttl` alone at AUC 0.9918, duplicate rate 0.206, overlap 0.133; the ablation refits to 0.998731 de-duplicated and 0.999659 with that column dropped.

Why it matters

UNSW-NB15 was built to fix KDD99's unrealistic distributions, and it still yields one column that separates the classes almost perfectly. That the attack traffic carries a distinguishable TTL signature is a property of how the attacks were generated, not of network physics. No deployed sensor inherits it, and Fuzzers recall sits at 0.638 at a false-positive rate of 0.0037.

Relevance and limits

It pairs directly with nb13, the same underlying UNSW-NB15 capture re-featured into the 43-field NetFlow v2 schema by Sarhan et al. (2022). There the strongest single feature is `MIN_TTL` at AUC 0.9932, so the shortcut survives a complete change of feature extractor. Read together, the pair locates the artifact in the packets and shows that re-featurisation does not clean a corpus.

What you are looking for. The feature matrix retains `sttl` and `dttl` alongside `ct_state_ttl`. The ablation therefore removes one TTL-derived column and leaves two behind, and the AUC moves 0.999671 to 0.999659. The prose reads that as evidence the separability is multi-feature; decide whether that inference is available given what is still in X.

Check first: Whether dropping `ct_state_ttl` removes the TTL shortcut at all, given that `sttl` and `dttl` remain in the matrix.

Schema — 49 columns

Column names and types only. No rows of this corpus are hosted here.

59.166.0.0object1390int64149.171.126.6object53objectudpobjectCONobject0.001055float64132int64164int6431int6429int640int640.1int64dnsobject500473.9375float64621800.9375float642int642.1int640.2int640.3int640.4int640.5int6466int6482int640.6int640.7int640.8float640.9float641421927414int641421927414.1int640.017float640.013float640.10float640.11float640.12float640.13int640.14int640.15int640.16int640.17int643int647int641int643.1int641.1int641.2int641.3int64Unnamed: 47object0.18int64

Read notebook PDF .ipynb ↑ Index

nb10 Botnet Detection on the CTU-13 Dataset

Botnet detection, real malware 1,572,056 flows x 9 features, botnet rate 0.1736 (a bounded-sample rate, not the natural base rate).

Corpus. Thirteen Stratosphere IPS captures of real malware executed in a lab alongside real background and normal traffic, read as Argus .binetflow records by direct download. no credentials required

Provenance chain. Fetched from Stratosphere IPS, the authors' own site — the authoritative distribution.

What it teaches

What a false-positive rate means when the negative class was never verified to be clean. It also teaches how feature-space coarseness interacts with a duplicate-row audit. With nine columns, distinct flows can collapse to identical rows for reasons unrelated to repeated traffic.

How it works

The loader keeps every botnet flow within the first 1,500,000 rows of each scenario, then subsamples up to 100,000 non-botnet flows per scenario. Family comes from the scenario-to-malware map in Garcia et al. (2014), and `scenario` is dropped because it maps one-to-one onto family. Four learners train on 120,000 rows and are ranked on a 393,014-row held-out split; the baseline accuracy is 0.8264 and XGBoost wins at 0.989405. The audit reports `State` alone at AUC 0.8271, duplicate rate 0.419, overlap 0.353; the ablation refits to 0.972796 de-duplicated and 0.988168 without `State`.

Why it matters

CTU-13 is one of the few corpora where the malicious traffic is real malware rather than a scripted generator. Sogou recall is 0.188 and NSIS.ay is 0.284: two families the model mostly misses while the headline reads 0.989405. De-duplication costs almost two points of AUC, so repeated rows carry part of that headline.

Relevance and limits

It stands in for flow-level botnet detection against real command-and-control traffic, the setting Garcia et al. evaluated per-scenario. It cannot stand in for cross-scenario generalisation. The split is random over pooled scenarios, so a family's training and test flows come from the same capture.

What you are looking for. Class 0 means 'not labelled botnet', not 'known benign'. It mixes verified Normal flows with Background traffic the CTU authors define as traffic whose purpose they do not know. So the printed false-positive rate of 0.0162 counts alerts on unverified traffic, and the same noisy negatives trained the model.

Check first: Whether a 0.353 train/test overlap on a nine-column matrix is repeated traffic, or a feature space too coarse to separate flows.

Read notebook PDF .ipynb ↑ Index

nb11 NetFlow-Standardized Intrusion Detection: NF-BoT-IoT-V2

Standardised NetFlow: Bot-IoT 30,420,086 rows x 41 features, positive rate 0.9957.

Corpus. Bot-IoT re-featured into the 43-field NetFlow v2 schema of Sarhan et al. (2022), read as parquet from the Kaggle mirror `dhoogla/nfbotiotv2`. Kaggle: dhoogla/nfbotiotv2

Provenance chain. Fetched from the Kaggle account dhoogla. The corpus is credited to Sarhan, M., Layeghy, S. & Portmann, M. Criterion 1 asks you to establish whether those are the same party, and what follows if they are not.

What it teaches

How to distinguish an audit that passed from an audit that had nothing to bite on. When the class balance is inverted and the negatives are a thin sliver, validity checks lose their power before they run. It also teaches you to read the whole comparison table, not the winning row.

How it works

Identifier-like object columns are dropped, the rest label-encoded, values clipped to plus/minus 1e15, and constants removed. Four learners train on a stratified 120,000-row subsample and are ranked on a 7,605,022-row held-out split. The majority-class baseline accuracy is 0.9957 and the majority class is attack, not benign; XGBoost wins at ROC-AUC 0.999902. The audit finds `SRC_TO_DST_SECOND_BYTES` alone at AUC 0.8823, duplicate rate 0.000, overlap 0.000, and neither ablation variant moves the score.

Why it matters

Attack is the majority class and benign is the rare one, which reverses every intuition about what accuracy means. The false-positive rate of 0.0341 is computed on 32,359 benign rows out of 7,605,022 held out, so the denominator that matters is small. All four attack families score recall at or above 0.982, so the per-family panel resolves nothing.

Relevance and limits

The NetFlow v2 schema exists so a model trained on one corpus can be tested on another. Layeghy and Portmann (2023) found no model that generalised across four such corpora. This notebook runs no transfer test and makes no transfer claim; notebook 36 runs it, and an in-distribution score cannot answer that question.

What you are looking for. Two things are worth judging. The exact-duplicate rate is exactly 0.000 across 30,420,086 rows, which is not plausible for a raw capture; decide whether that is a finding about the collection or about the distribution channel, because the notebook does not ask. Separately, LightGBM lands at ROC-AUC 0.720484 on the split where XGBoost reaches 0.999902, a collapse printed but never explained.

Check first: How many benign rows the printed false-positive rate of 0.0341 is actually computed on, relative to the held-out split.

Schema — 43 columns, label “Label”

Column names and types only. No rows of this corpus are hosted here.

Label distribution (proportions from the first 2,000 rows, not the full corpus): 0 0.4% · 1 99.6%

L4_SRC_PORTint16L4_DST_PORTint16PROTOCOLint8L7_PROTOfloat32IN_BYTESint32IN_PKTSint32OUT_BYTESint32OUT_PKTSint32TCP_FLAGSint16CLIENT_TCP_FLAGSint16SERVER_TCP_FLAGSint16FLOW_DURATION_MILLISECONDSint32DURATION_INint16DURATION_OUTint16MIN_TTLint16MAX_TTLint16LONGEST_FLOW_PKTint32SHORTEST_FLOW_PKTint16MIN_IP_PKT_LENint16MAX_IP_PKT_LENint32SRC_TO_DST_SECOND_BYTESfloat32DST_TO_SRC_SECOND_BYTESfloat32RETRANSMITTED_IN_BYTESint32RETRANSMITTED_IN_PKTSint16RETRANSMITTED_OUT_BYTESint16RETRANSMITTED_OUT_PKTSint8SRC_TO_DST_AVG_THROUGHPUTint64DST_TO_SRC_AVG_THROUGHPUTint64NUM_PKTS_UP_TO_128_BYTESint16NUM_PKTS_128_TO_256_BYTESint16NUM_PKTS_256_TO_512_BYTESint16NUM_PKTS_512_TO_1024_BYTESint32NUM_PKTS_1024_TO_1514_BYTESint32TCP_WIN_MAX_INint32TCP_WIN_MAX_OUTint32ICMP_TYPEint32ICMP_IPV4_TYPEint16DNS_QUERY_IDint32DNS_QUERY_TYPEint16DNS_TTL_ANSWERint32FTP_COMMAND_RET_CODEfloat32Labelint8Attackobject

Read notebook PDF .ipynb ↑ Index

nb12 NetFlow-Standardized Intrusion Detection: NF-ToN-IoT-V2

Standardised NetFlow: ToN-IoT 13,135,881 rows x 41 features, positive rate 0.7258.

Corpus. The ToN-IoT network captures of Moustafa (2021), re-featured into the 43-field NetFlow v2 schema of Sarhan et al. (2022), from the Kaggle mirror `dhoogla/nftoniotv2`. Kaggle: dhoogla/nftoniotv2

What it teaches

What to do when every check passes. A clean audit bounds exactly two failure modes and is silent on the rest, so the analytic move is to name what the checks cannot see. A passed check and an inapplicable check look identical in the output.

How it works

Identifier-like object columns are dropped, the remainder label-encoded, values clipped and constants removed. Four learners train on a stratified 120,000-row subsample and are ranked on a 3,283,971-row held-out split, against a baseline accuracy of 0.7258. XGBoost wins at ROC-AUC 0.998863, LightGBM 0.998699, RandomForest 0.997670, LogisticRegression 0.839443. The audit finds `LONGEST_FLOW_PKT` alone at AUC 0.7637, duplicate rate 0.000, overlap 0.000; both ablation variants return 0.998863, unchanged.

Why it matters

This is the control case in the set: a near-perfect score that the two standard explanations do not account for. That makes the residual question sharper, not softer. The weakest group is ransomware at recall 0.904, and the false-positive rate is 0.0231 across 900,321 benign flows, a large absolute alert volume.

Relevance and limits

It stands in for in-distribution detection on a standardised NetFlow schema, the format the field adopted to enable cross-corpus comparison. It cannot stand in for that comparison. Layeghy and Portmann (2023) found no model generalising across four NetFlow-standardised corpora, and no transfer test is run here.

What you are looking for. By the two checks this notebook runs, nothing is found, and reporting 'nothing found' is the correct answer. But an exact-duplicate rate of exactly 0.000 across 13,135,881 flows is not plausible for a raw capture, and the notebook never asks why. The remaining risks, a random rather than temporal split and untested transfer, are unmeasured rather than absent.

Check first: Whether a 0.000 duplicate rate over 13.1 million flows is evidence of clean data, or of de-duplication that happened before the notebook.

Schema — 43 columns, label “Label”

Column names and types only. No rows of this corpus are hosted here.

Label distribution (proportions from the first 2,000 rows, not the full corpus): 0 97.4% · 1 2.6%

L4_SRC_PORTint32L4_DST_PORTint32PROTOCOLint8L7_PROTOfloat32IN_BYTESint32IN_PKTSint32OUT_BYTESint32OUT_PKTSint32TCP_FLAGSint16CLIENT_TCP_FLAGSint16SERVER_TCP_FLAGSint16FLOW_DURATION_MILLISECONDSint32DURATION_INint32DURATION_OUTint32MIN_TTLint16MAX_TTLint16LONGEST_FLOW_PKTint32SHORTEST_FLOW_PKTint16MIN_IP_PKT_LENint16MAX_IP_PKT_LENint32SRC_TO_DST_SECOND_BYTESfloat64DST_TO_SRC_SECOND_BYTESfloat64RETRANSMITTED_IN_BYTESint32RETRANSMITTED_IN_PKTSint16RETRANSMITTED_OUT_BYTESint32RETRANSMITTED_OUT_PKTSint16SRC_TO_DST_AVG_THROUGHPUTint64DST_TO_SRC_AVG_THROUGHPUTint64NUM_PKTS_UP_TO_128_BYTESint32NUM_PKTS_128_TO_256_BYTESint32NUM_PKTS_256_TO_512_BYTESint32NUM_PKTS_512_TO_1024_BYTESint16NUM_PKTS_1024_TO_1514_BYTESint32TCP_WIN_MAX_INint32TCP_WIN_MAX_OUTint32ICMP_TYPEint32ICMP_IPV4_TYPEint16DNS_QUERY_IDint32DNS_QUERY_TYPEint16DNS_TTL_ANSWERint32FTP_COMMAND_RET_CODEint16Labelint8Attackobject

Read notebook PDF .ipynb ↑ Index

nb13 NetFlow-Standardized Intrusion Detection: NF-UNSW-NB15-V2

Network intrusion (NetFlow) 1,986,745 flows x 41 features; positive rate 0.0378.

Corpus. The UNSW-NB15 capture re-extracted by Sarhan et al. (2022) into the standard 43-field NetFlow v2 schema, mirrored on Kaggle as `dhoogla/nfunswnb15v2`. Kaggle: dhoogla/nfunswnb15v2

What it teaches

How to tell a one-column shortcut from a corpus-level generation artifact. Here the ablation removes the strongest feature and the held-out AUC does not move at six decimal places. That is the harder case: nothing collapsed, and nothing was vindicated either.

How it works

A stratified 75/25 split, with training capped at 120,000 stratified rows and 496,687 rows held out. Four learners are ranked on ROC-AUC with a majority-class baseline row inside the table. The audit computes direction-agnostic per-column ROC-AUC on a 60,000-row subsample, the whole-corpus exact-duplicate rate, and train/test exact-row overlap on the first 50,000 held-out rows. The trust grade is the worse of those two rubrics. The model is then refit after de-duplication and after dropping the single strongest column.

Why it matters

The NetFlow v2 schema was proposed precisely so intrusion corpora could be compared across datasets. A leak that survives re-extraction into that schema means the standardization does not repair the benchmark. A sensor deployed on real traffic does not see TTL values partitioned by attack tooling.

Relevance and limits

This stands in for flow-level intrusion detection on standardized telemetry, the form most SOC pipelines actually ingest. It cannot stand in for cross-corpus generalization: the split is random, in-distribution, and single-corpus. Notebook 36 runs the transfer matrix this one deliberately does not.

What you are looking for. This is the NetFlow-v2 re-extraction of the same source capture as nb09, and both leak through TTL. nb09's audit names `ct_state_ttl` at single-feature AUC 0.9918; this one names `MIN_TTL` at 0.9932, self-grading D. Two independent feature-extraction pipelines over one capture reproduce the same artifact, which locates it in the packets rather than in either schema. Note the corpora are not the same rows: 2,540,047 flows at 0.1265 positive there, 1,986,745 at 0.0378 here.

Check first: Whether the flat ablation, 0.999502 to 0.999502, is being read as vindication when it is evidence of many redundant leaks rather than none.

Schema — 43 columns, label “Label”

Column names and types only. No rows of this corpus are hosted here.

Label distribution (proportions from the first 2,000 rows, not the full corpus): 0 99.9% · 1 0.1%

L4_SRC_PORTint32L4_DST_PORTint32PROTOCOLint16L7_PROTOfloat32IN_BYTESint32IN_PKTSint16OUT_BYTESint32OUT_PKTSint16TCP_FLAGSint8CLIENT_TCP_FLAGSint8SERVER_TCP_FLAGSint8FLOW_DURATION_MILLISECONDSint32DURATION_INint16DURATION_OUTint16MIN_TTLint16MAX_TTLint16LONGEST_FLOW_PKTint16SHORTEST_FLOW_PKTint16MIN_IP_PKT_LENint16MAX_IP_PKT_LENint16SRC_TO_DST_SECOND_BYTESfloat32DST_TO_SRC_SECOND_BYTESfloat32RETRANSMITTED_IN_BYTESint32RETRANSMITTED_IN_PKTSint16RETRANSMITTED_OUT_BYTESint32RETRANSMITTED_OUT_PKTSint16SRC_TO_DST_AVG_THROUGHPUTint64DST_TO_SRC_AVG_THROUGHPUTint64NUM_PKTS_UP_TO_128_BYTESint16NUM_PKTS_128_TO_256_BYTESint16NUM_PKTS_256_TO_512_BYTESint16NUM_PKTS_512_TO_1024_BYTESint16NUM_PKTS_1024_TO_1514_BYTESint16TCP_WIN_MAX_INint32TCP_WIN_MAX_OUTint32ICMP_TYPEint32ICMP_IPV4_TYPEint16DNS_QUERY_IDint32DNS_QUERY_TYPEint32DNS_TTL_ANSWERint64FTP_COMMAND_RET_CODEfloat32Labelint8Attackobject

Read notebook PDF .ipynb ↑ Index

nb14 IoT Attack Detection on CICIoT2023

IoT attack detection 1,048,575 flows x 41 features; positive rate 0.9767 - attack is the majority class here, and benign is the rare one.

Corpus. CICIoT2023 (Neto et al., 2023): flow records from a 105-device IoT testbed under 33 attack types. The Kaggle mirror `subhajournal/iotintrusion` supplies a 1,048,575-row subset of the ~47M published corpus. Kaggle: subhajournal/iotintrusion

Provenance chain. Fetched from the Kaggle account subhajournal. The corpus is credited to Neto, E.C.P. et al. Criterion 1 asks you to establish whether those are the same party, and what follows if they are not.

What it teaches

That a letter grade produced by thresholding a sampled statistic is itself a random variable. It also inverts the usual imbalance reflex. The majority-class baseline here is 'always predict attack', and the false-positive rate rests on only 6,119 benign held-out rows.

How it works

A stratified 75/25 split, with training capped at 120,000 stratified rows and 262,144 rows held out. Four learners are ranked on ROC-AUC against a majority-class accuracy baseline of 0.9767. The audit computes per-column direction-agnostic ROC-AUC on a 60,000-row subsample drawn at a fixed seed, plus the exact-duplicate rate and train/test exact-row overlap. Ablation refits after de-duplication and after dropping the single strongest column. Per-group recall is reported across the 33 attack families.

Why it matters

Rubric-driven data-trust grading is now common in applied ML-security reporting, and a grade is only useful if it is stable. If the letter moves with which rows happened to be sampled, it is a coin flip dressed as a measurement. The printed 0.9906 sits six ten-thousandths above the cutoff that decides it.

Relevance and limits

This stands in for large-scale IoT telemetry triage, where flood classes dominate and rare reconnaissance classes carry the operational risk. It cannot stand in for a real deployment, where benign traffic is the overwhelming majority. A false-positive budget set on 6,119 benign flows does not transfer to millions.

What you are looking for. The self-assigned grade genuinely flips. The single-feature audit lands close enough to the 0.99 rubric cutoff that re-drawing the 60,000-row sample can move the letter. Nothing about the corpus changes, only which rows were drawn. Separately, dropping the strongest feature raises the AUC slightly, 0.999394 to 0.999441, which the notebook reads as multi-feature separability rather than as a clean result.

Check first: Re-run the section 10 single-feature audit under a different sample seed and see whether the printed letter moves.

Schema — 47 columns, label “label”

Column names and types only. No rows of this corpus are hosted here.

Label distribution (proportions from the first 2,000 rows, not the full corpus): DDoS-ICMP_Flood 15.0% · DDoS-UDP_Flood 11.5% · DDoS-TCP_Flood 10.3% · DDoS-PSHACK_Flood 9.8% · DDoS-RSTFINFlood 8.8% · DDoS-SynonymousIP_Flood 8.7% · DoS-UDP_Flood 7.4% · DDoS-SYN_Flood 6.9%

flow_durationfloat64Header_Lengthfloat64Protocol Typefloat64Durationfloat64Ratefloat64Sratefloat64Drateint64fin_flag_numberint64syn_flag_numberint64rst_flag_numberint64psh_flag_numberint64ack_flag_numberint64ece_flag_numberint64cwr_flag_numberint64ack_countfloat64syn_countfloat64fin_countfloat64urg_countfloat64rst_countfloat64HTTPint64HTTPSint64DNSint64Telnetint64SMTPint64SSHint64IRCint64TCPint64UDPint64DHCPint64ARPint64ICMPint64IPvint64LLCint64Tot sumfloat64Minfloat64Maxfloat64AVGfloat64Stdfloat64Tot sizefloat64IATfloat64Numberfloat64Magnituefloat64Radiusfloat64Covariancefloat64Variancefloat64Weightfloat64labelobject

Read notebook PDF .ipynb ↑ Index

nb15 Mobile-Money Fraud Detection: the PaySim Simulator

Mobile-money fraud 1,500,000 transactions x 6 features; positive rate 0.0011.

Corpus. PaySim (Lopez-Rojas et al., 2016), an agent-based simulation of mobile-money transfers seeded from a real African provider's aggregates; the first 1,500,000 of its 6,362,620 rows. Kaggle: ealaxi/paysim1

Provenance chain. Fetched from the Kaggle account ealaxi. The corpus is credited to Lopez-Rojas, E.A., Elmir, A. & Axelsson, S. Criterion 1 asks you to establish whether those are the same party, and what follows if they are not.

What it teaches

The difference between 'the audit found nothing' and 'there is nothing to find'. A per-column rank statistic tests one column at a time; a duplicate check tests whole rows. Neither can see a shortcut carried by the relation between two columns.

How it works

A stratified 75/25 split, with training capped at 120,000 stratified rows and 375,000 rows held out. Account identifiers and the `isFlaggedFraud` rule flag are dropped, leaving six features. Four learners are ranked on ROC-AUC against a 0.9989 majority-class accuracy baseline. The audit computes per-column direction-agnostic ROC-AUC on a 60,000-row subsample, the exact-duplicate rate, and train/test exact-row overlap. It then refits after de-duplication and after dropping the single strongest column.

Why it matters

PaySim's documented fraud mechanic is that the attacking agent empties the victim account. So `amount` and every balance column are written by the same script that sets the label. Any score therefore measures how well a learner recovers a scripted rule. Reading it as evidence about real mobile-money fraud is a category error.

Relevance and limits

This stands in for extreme-imbalance rare-event detection, where recall on the minority class and not aggregate AUC is the decision-relevant quantity. It cannot stand in for real mobile-money fraud, and it cannot support any claim about adversarial adaptation, since the adversary here is a deterministic generator.

What you are looking for. Every mechanical check passes: strongest single feature 0.7846, exact-duplicate rate 0.000, train/test overlap 0.000, self-graded A. The headline 0.994731 is nevertheless unexplained, and the notebook says so. Dropping `oldbalanceOrg` costs 0.994731 to 0.955019, a partial drop that settles nothing because the account-emptying rule is spread over three retained columns. An independent refit that removes the two strongest by importance is more suggestive: both can go and the score still stands well clear of chance. This is the clearest case in the set of an audit that cannot reach the artifact it is looking for.

Check first: On fraudulent rows, check whether `amount` equals `oldbalanceOrg` and `newbalanceOrig` goes to zero - the test that neither diagnostic runs.

Schema — 11 columns, label “type”

Column names and types only. No rows of this corpus are hosted here.

Label distribution (proportions from the first 2,000 rows, not the full corpus): PAYMENT 49.9% · CASH_OUT 17.4% · CASH_IN 15.6% · TRANSFER 11.3% · DEBIT 5.8%

stepint64typeobjectamountfloat64nameOrigobjectoldbalanceOrgfloat64newbalanceOrigfloat64nameDestobjectoldbalanceDestfloat64newbalanceDestfloat64isFraudint64isFlaggedFraudint64

Read notebook PDF .ipynb ↑ Index

nb16 Credit-Card Fraud Detection: the Sparkov Simulated Dataset

Credit-card fraud 1,852,394 transactions x 12 features; positive rate 0.0052.

Corpus. Roughly 1.85M simulated card transactions from the Sparkov generator, distributed on Kaggle as `kartik2112/fraud-detection`, with train and test files concatenated. Kaggle: kartik2112/fraud-detection

Provenance chain. Fetched from the Kaggle account kartik2112. The corpus is credited to Harris, B. *Sparkov Data Generation*. Criterion 1 asks you to establish whether those are the same party, and what follows if they are not.

What it teaches

That a threshold-based trust grade and a drop-the-feature ablation can flatly contradict each other, and which of the two is a measurement. It also teaches reading per-group recall as the operational result. Six of the fourteen merchant categories print a recall of 0.000.

How it works

A stratified 75/25 split, with training capped at 120,000 stratified rows and 463,099 rows held out. Direct identifiers, timestamps and the merchant category are dropped; twelve columns reach the learners, with the category retained only as the per-group recall label. Four learners are ranked on ROC-AUC against a 0.9948 majority-class accuracy baseline. The audit computes per-column ROC-AUC on a 60,000-row subsample, the exact-duplicate rate, and train/test exact-row overlap, then refits after de-duplication and after dropping the single strongest column.

Why it matters

A detector that is one amount threshold has no defence against an adversary who splits a transaction. The aggregate 0.9486 conceals exactly that: the model ranks well and decides badly, missing entire merchant categories outright. Ranking quality and decision quality are separate properties and this notebook separates them for you.

Relevance and limits

This stands in for amount-threshold triage, which is a real and widely deployed first filter in card fraud. It cannot stand in for production card fraud detection, which is entity-structured and time-ordered; the split here is random, so cardholder history crosses it freely.

What you are looking for. The model is one column. The notebook's own ablation moves the held-out AUC 0.948632 to 0.574302 when `amt` is dropped. An independent refit removing the top two by importance shows the collapse is already complete after the first, so the second column is worth almost nothing. The notebook still self-grades A, because 0.8494 sits under the 0.85 rubric cutoff. Whether that letter is stable is not something the notebook can tell you. A second, separate defect is printed in the same tables. The selected winner's accuracy 0.994593 fails the majority baseline of 0.9948, and only the unselected RandomForest at 0.995046 clears it.

Check first: Put the section 11 ablation next to the section 10 letter grade; they disagree, and only one of them is a measurement.

Schema — 23 columns, label “is_fraud”

Column names and types only. No rows of this corpus are hosted here.

Label distribution (proportions from the first 2,000 rows, not the full corpus): 0 100.0%

Unnamed: 0int64trans_date_trans_timeobjectcc_numint64merchantobjectcategoryobjectamtfloat64firstobjectlastobjectgenderobjectstreetobjectcityobjectstateobjectzipint64latfloat64longfloat64city_popint64jobobjectdobobjecttrans_numobjectunix_timeint64merch_latfloat64merch_longfloat64is_fraudint64

Read notebook PDF .ipynb ↑ Index

nb17 Large-Scale Financial Transaction Fraud Detection

Financial transaction fraud 1,500,000 transactions x 9 features; positive rate 0.0226 in the slice, against ~3.59% across the full file.

Corpus. A synthetic, community-generated 5,000,000-transaction file on Kaggle (`aryan208/financial-transactions-dataset-for-fraud-detection`) carrying engineered amount, velocity, geo-anomaly and spending-deviation scores; the first 1,500,000 rows are read. Kaggle: aryan208/financial-transactions-dataset-for-fraud-detection

Provenance chain. Fetched from the Kaggle account aryan208. The corpus is credited to Dal Pozzolo, A. et al. Criterion 1 asks you to establish whether those are the same party, and what follows if they are not.

What it teaches

That an audit is only as strong as the failure modes it enumerates. A per-column rank statistic is structurally blind to signal a tree reaches with one split at a single value. An ablation steered by that same statistic inherits the blind spot and keeps deleting the wrong column.

How it works

A stratified 75/25 split, with training capped at 120,000 stratified rows and 375,000 rows held out. Identifiers, IP, device hash, timestamp and `fraud_type` are dropped; the loader ends with `fillna(0.0)`. Four learners are ranked on ROC-AUC against a 0.9774 majority-class accuracy baseline. The audit computes per-column direction-agnostic ROC-AUC on a 60,000-row subsample, the exact-duplicate rate, and train/test exact-row overlap. The ablation then refits after de-duplication and after dropping whichever column won the rank statistic - not whichever column the model actually used.

Why it matters

A passing audit is increasingly offered as evidence that a security model is sound. Here every mechanical check passes and the result does not hold. 'The audit found nothing' and 'there is nothing to find' are different claims, and a reviewer who cannot separate them will sign off on artifacts.

Relevance and limits

This is a methods sandbox for audit design rather than a benchmark; it has no peer-reviewed origin and its features are generated. It stands in for the very common case of a model fit on engineered anomaly scores whose provenance is unstated. It cannot support any claim about real fraud rates or real fraud behaviour.

What you are looking for. The audit reports best single feature 0.5171, zero duplicates and zero contamination, so it returns grade A, and every arithmetic step is correct. The ablation then drops `device_used`, which the winning model ranks eighth of nine, and the AUC moves 0.748990 to 0.748274. An independent refit that removes the model's actual top feature, `time_since_last_transaction`, is the test that would settle it. The notebook names that hypothesis and states plainly that it never ran it. At threshold 0.5 the model flags one benign transaction of 366,543 and scores 0.0 recall on all four transaction types.

Check first: Find `time_since_last_transaction` in the section 9 importance chart, then find the same column near the chance line in the section 10 single-feature chart.

Schema — 18 columns, label “is_fraud”

Column names and types only. No rows of this corpus are hosted here.

Label distribution (proportions from the first 2,000 rows, not the full corpus): False 100.0%

transaction_idobjecttimestampobjectsender_accountobjectreceiver_accountobjectamountfloat64transaction_typeobjectmerchant_categoryobjectlocationobjectdevice_usedobjectis_fraudboolfraud_typefloat64time_since_last_transactionfloat64spending_deviation_scorefloat64velocity_scoreint64geo_anomaly_scorefloat64payment_channelobjectip_addressobjectdevice_hashobject

Read notebook PDF .ipynb ↑ Index

nb18 Malicious URL Detection (Phishing / Defacement / Malware)

Malicious URL detection 632,844 URLs x 84 features; positive rate 0.3328.

Corpus. An aggregated 2026 community dump of 632,844 labelled URLs on Kaggle, its ISCX-URL-style lineage inherited rather than certified per row. One canonical CSV is read; the archive's older variants are deliberately not concatenated. Kaggle: moutasmtamimi/malicious-url-detection-dataset-enhanced-2026

Provenance chain. Fetched from the Kaggle account moutasmtamimi. The corpus is credited to Sahoo, D., Liu, C. & Hoi, S.C.H. Criterion 1 asks you to establish whether those are the same party, and what follows if they are not.

What it teaches

How to price train/test contamination that a notebook measures on itself. A 19% exact-duplicate rate sounds fatal until you see what de-duplication actually costs. It also teaches when 'nothing found on this axis' is the correct finding rather than a failure of effort.

How it works

A stratified 75/25 split, with training capped at 120,000 stratified rows and 158,211 rows held out. Redundant label columns and raw string and timestamp columns are dropped, leaving 84 lexical, host and web-security features. Four learners are ranked on ROC-AUC against a 0.6672 majority-class accuracy baseline. The audit computes per-column direction-agnostic ROC-AUC on a 60,000-row subsample, the whole-corpus exact-duplicate rate, and train/test exact-row overlap on the first 50,000 held-out rows. It then refits after de-duplication and after dropping the single strongest column.

Why it matters

URL classifiers are front-line phishing defence and are almost always evaluated in-distribution on aggregated dumps like this one. The generalisation question that matters is cross-feed transfer, and no split of a single file can answer it. The corpus records no per-URL upstream provenance, so 'that is how it was generated' is not an available explanation for any result.

Relevance and limits

This stands in for lexical and host-based URL triage at the perimeter, where 84 correlated string statistics is a realistic feature budget. It cannot stand in for deployment against a live feed, since the features include live-site probes of the same page. Phishing recall of 0.714, not the 0.983724 aggregate, is the operationally relevant number.

What you are looking for. This one is genuinely sound on the shortcut axis, and saying so is the correct answer. Dropping the strongest feature moves 0.983724 to 0.983691, and an independent refit that removes the two most important columns costs little more. The real weakness is 14.9% train/test exact-row overlap, which the notebook measures itself, grades B on, and states in its conclusion. Judge whether a self-reported, bounded contamination of that size undermines the headline, given what de-duplication costs.

Check first: Check whether the two contamination windows agree before deciding sampling is the objection - then check what de-duplication actually costs: 0.983724 to 0.979117.

Schema — 99 columns, label “label”

Column names and types only. No rows of this corpus are hosted here.

Label distribution (proportions from the first 2,000 rows, not the full corpus): 0 73.2% · 1 18.8% · 2 5.5% · 3 2.5%

urlobjecturl_normalizedobjecturl_hashobjectclass_labelobjectlabelint64typeobjectsource_datasetobjectdataset_versionobjectts_first_seenobjectts_aggregatedobjectreachability_classobjectis_syntax_validboolnorm_actionsobjecthostobjectpathobjectqueryobjectdomain_lenint64path_lenint64query_lenint64path_depthint64subdomain_countint64query_param_countint64has_portint64is_ip_hostint64has_punycodeint64tldobjectis_httpsint64is_shortenerint64digit_count_v2int64letter_count_v2int64special_count_v2int64digit_ratiofloat64letter_ratiofloat64special_ratiofloat64token_countint64max_token_lenint64avg_token_lenfloat64url_entropyfloat64host_entropyfloat64url_len_v2int64url_lenint64@int64?int64-int64=int64.int64#int64%int64+int64$int64!int64*int64,int64//int64digitsint64lettersint64abnormal_urlint64httpsint64Shortining_Serviceint64having_ip_addressint64

… and 39 further columns.

Read notebook PDF .ipynb ↑ Index

nb19 IoT Malware Flow Detection on IoT-23 (Zeek conn.log)

IoT malware traffic 3,685,398 flows x 12 flow-behaviour features, malicious rate 0.593. Every malicious flow is kept and benign is capped at 1,500,000, so that rate is a bounded-sample rate, not a base rate.

Corpus. One Zeek conn.log capture, CTU-IoT-Malware-Capture-35-1, from the Stratosphere Lab / Avast IoT-23 release, chosen because benign and malicious flows appear in the same recording. Kaggle: agungpambudi/network-malware-detection-connection-analysis

Provenance chain. Fetched from the Kaggle account agungpambudi. The corpus is credited to García, S., Parmisano, A. & Erquiaga, M.J. Criterion 1 asks you to establish whether those are the same party, and what follows if they are not.

What it teaches

How an aggregate score can be the property of one loud class. The analytic move is to treat per-family recall as the primary result and the headline AUC as a summary of the dominant family. It also teaches when a rare-class recall figure is itself too thin to report as a finding.

How it works

Four learners fit on a 120,000-row stratified subsample of a 75/25 split, scored on 921,350 held-out flows. IP, port, uid and timestamp columns are dropped first, so no model can memorise which host a flow came from. The audit computes each feature's solo ROC-AUC, the exact-duplicate rate, and the overlap between the first 50,000 test rows and training. An ablation refits on the de-duplicated corpus and again without the top feature; per-family recall skips groups with under five held-out positives.

Why it matters

IoT flow monitors are sold on aggregate accuracy. Here the winning model reports DDoS recall 1.0 and C&C recall 0.0. It catches the flood a rate limiter already handles, and misses the C&C channel that signals compromise. A doctoral reader should be able to say which number is deployment-relevant, and why 0.999900 accuracy is neither.

Relevance and limits

This stands in for the Mirai-class question, whether flow behaviour alone can flag malicious IoT connections without deep-packet inspection. It cannot stand in for cross-capture generalisation. Only one capture is used, and IoT-23's remaining captures are near-single-class, so concatenating them would make which-pcap the real signal.

What you are looking for. The audit names two problems: `history` alone reaches ROC-AUC 0.9996, and 42.2% of held-out rows duplicate a training row. The sharper issue is the deliverable itself. The malicious class is 2,185,302 DDoS flows against 93 C&C and 3 Attack. So the headline C&C recall of 0.0 rests on roughly 23 held-out flows, and it is worth checking whether `Attack` clears the five-positive floor at all.

Check first: Count the held-out C&C flows behind the 0.0 recall before treating that recall as a measurement.

Schema — 1 columns

Column names and types only. No rows of this corpus are hosted here.

ts|uid|id.orig_h|id.orig_p|id.resp_h|id.resp_p|proto|service|duration|orig_bytes|resp_bytes|conn_state|local_orig|local_resp|missed_bytes|history|orig_pkts|orig_ip_bytes|resp_pkts|resp_ip_bytes|tunnel_parents|label|detailed-label

object

Read notebook PDF .ipynb ↑ Index

nb20 NetFlow-Standardized Intrusion Detection: NF-CSE-CIC-IDS2018-V2

Network intrusion (NetFlow) 17,129,715 flows x 40 features, positive rate 0.1184.

Corpus. CSE-CIC-IDS2018 re-featured by Sarhan et al. onto the standard 43-field NetFlow v2 schema, taken from the cleaned Kaggle parquet release. Kaggle: dhoogla/nfcsecicids2018v2

What it teaches

How to read a validity audit that comes back negative. Every diagnostic here is clean, so the skill is separating 'no artifact found' from 'result is deployable'. What remains is a per-family gap, not a validity failure, and the two are graded differently.

How it works

Four learners fit on a 120,000-row stratified subsample of a 75/25 split, scored on 4,282,429 held-out flows against a 0.8816 majority baseline. The `Attack` family column and any object column with more than 50% unique values are dropped before fitting. The audit computes solo feature ROC-AUC, exact-duplicate rate, and train/test exact-row overlap on the first 50,000 test rows. The ablation refits after de-duplication and after removing the top solo feature; recall is broken out over 14 attack groups.

Why it matters

Infilteration recall is 0.276 against an aggregate accuracy of 0.994956. The class that most resembles a real intrusion, a foothold rather than a flood, is the one the model misses. That gap survives a clean audit, which is precisely why aggregate scores cannot be the unit of evaluation.

Relevance and limits

NF-v2 exists so intrusion models can be trained in one environment and tested in another. This notebook measures in-distribution separability on a single corpus and says so; it makes no transfer claim. Notebook 36 runs the cross-corpus transfer matrix that would test the schema's actual purpose.

What you are looking for. Nothing is hidden here. Duplicate rate 0.000, train/test contamination 0.000. Dropping the strongest single feature moves the headline from 0.988914 to 0.988970, so separability is multi-feature. Reporting that the audit finds nothing is the correct answer; the honest criticism is scope, not validity.

Check first: Check whether the Infilteration recall of 0.276 travels with the aggregate accuracy anywhere the headline is quoted.

Schema — 43 columns, label “Label”

Column names and types only. No rows of this corpus are hosted here.

Label distribution (proportions from the first 2,000 rows, not the full corpus): 0 87.5% · 1 12.6%

L4_SRC_PORTint32L4_DST_PORTint32PROTOCOLint8L7_PROTOfloat32IN_BYTESint32IN_PKTSint32OUT_BYTESint32OUT_PKTSint32TCP_FLAGSint16CLIENT_TCP_FLAGSint16SERVER_TCP_FLAGSint16FLOW_DURATION_MILLISECONDSint32DURATION_INint32DURATION_OUTint32MIN_TTLint16MAX_TTLint16LONGEST_FLOW_PKTint32SHORTEST_FLOW_PKTint16MIN_IP_PKT_LENint16MAX_IP_PKT_LENint32SRC_TO_DST_SECOND_BYTESfloat64DST_TO_SRC_SECOND_BYTESfloat64RETRANSMITTED_IN_BYTESint32RETRANSMITTED_IN_PKTSint16RETRANSMITTED_OUT_BYTESint32RETRANSMITTED_OUT_PKTSint16SRC_TO_DST_AVG_THROUGHPUTint64DST_TO_SRC_AVG_THROUGHPUTint64NUM_PKTS_UP_TO_128_BYTESint32NUM_PKTS_128_TO_256_BYTESint16NUM_PKTS_256_TO_512_BYTESint16NUM_PKTS_512_TO_1024_BYTESint16NUM_PKTS_1024_TO_1514_BYTESint32TCP_WIN_MAX_INint32TCP_WIN_MAX_OUTint32ICMP_TYPEint32ICMP_IPV4_TYPEint16DNS_QUERY_IDint32DNS_QUERY_TYPEint16DNS_TTL_ANSWERint32FTP_COMMAND_RET_CODEint8Labelint8Attackobject

Read notebook PDF .ipynb ↑ Index

nb21 Host-Based Intrusion Detection on BETH Kernel Telemetry

Host / kernel telemetry 3,807,196 process events x 5 behaviour features, suspicious rate 0.1197 (evil rate 0.0442).

Corpus. Real syscall-level process events captured on cloud honeypots (BETH, Highnam et al. 2021), nine process CSVs concatenated, hand-labelled `sus` and `evil` by the dataset authors. Kaggle: katehighnam/beth-dataset

Provenance chain. Fetched from the Kaggle account katehighnam. The corpus is credited to Highnam, K., Arulkumaran, K., Hanif, Z. & Jennings, N.R. Criterion 1 asks you to establish whether those are the same party, and what follows if they are not.

What it teaches

What a random split measures when the feature space is small and the rows repeat. Five features cannot generate 3.8M distinct vectors, so a held-out score becomes a lookup rather than a prediction. The skill is reading the duplicate and contamination statistics before the AUC, and knowing which one bounds the other.

How it works

The target is `sus`; `family` splits positives into `evil` and `suspicious-only`. processId, parentProcessId, hostName and userId are dropped, the last because `userId >= 1000` sits close to the human labeller's own cue for `sus`. Four learners are fit on a 120,000-row stratified subsample and scored on 951,799 held-out events. The audit reports solo feature AUC, exact-duplicate rate and train/test overlap; the ablation refits on the de-duplicated corpus and again without the top solo feature.

Why it matters

Host IDS is judged on catching confirmed intrusions, and the notebook reports evil recall 0.967 against suspicious-only recall 0.398. Whether the first number means anything depends entirely on whether the held-out set is genuinely held out. BETH ships an official cross-host split; this notebook uses a random one and flags that as the honest next step.

Relevance and limits

Stands in for eBPF and syscall-based endpoint detection trained on honeypot ground truth, the one host-based problem in a series otherwise built on network flows. It cannot stand in for cross-host generalisation, and `sus` is a hand-assigned judgement of unusualness, not an incident record.

What you are looking for. The exact-duplicate rate is 0.998 and train/test contamination is 0.995. Roughly 7,600 distinct feature vectors carry 3.8M rows, so the held-out set sits almost entirely inside training and the headline measures memorisation. The notebook's defence is that de-duplication raises the AUC to 0.990868. Weigh it: collapsing duplicates resolves conflicting labels on identical vectors, making the task easier rather than cleaner. Notebook 22 concedes exactly this about its own de-duplication; this one does not.

Check first: The ablation drops `returnValue`, the top solo-AUC feature at 0.6976, then concludes no feature is a shortcut; refit without `processName`, the top-importance feature, before accepting that.

Schema — 16 columns, label “sus”

Column names and types only. No rows of this corpus are hosted here.

Label distribution (proportions from the first 2,000 rows, not the full corpus): 0 86.4% · 1 13.7%

timestampfloat64processIdint64threadIdint64parentProcessIdint64userIdint64mountNamespaceint64processNameobjecthostNameobjecteventIdint64eventNameobjectstackAddressesobjectargsNumint64returnValueint64argsobjectsusint64evilint64

Read notebook PDF .ipynb ↑ Index

nb22 Credential-Attack Detection on the RBA Login Dataset

Credential-attack detection 6,477,653 logins x 8 features, attack-IP rate 0.4781. Every attack row is kept and benign is subsampled, so that rate is a bounded-sample rate.

Corpus. 31.3M login attempts to a large online service, released by Wiefling et al. in synthesized form, each labelled `Is Attack IP` from a known-attacker IP list. Kaggle: dasgroup/rba-dataset

Provenance chain. Fetched from the Kaggle account dasgroup. The corpus is credited to Wiefling, S., Jørgensen, P.R., Thunem, S. & Lo Iacono, L. Criterion 1 asks you to establish whether those are the same party, and what follows if they are not.

What it teaches

How to reason about a label that is a property of an entity rather than of the event. The IP address, user ID and timestamp are dropped as direct leaks, but geolocation and ASN are also IP-derived. The skill is apportioning a score between behavioural risk signal and re-encoding of the blocklist that defined the label.

How it works

The corpus is read in 3M-row chunks, keeping every attack row and 12% of benign. `family` is set from `Country`, so per-group recall measures geographic bias. Four learners are fit on a 120,000-row stratified subsample and scored on 1,619,414 held-out logins against a 0.5219 majority baseline. The audit computes solo feature AUC, duplicate rate and train/test overlap; the ablation refits on the de-duplicated corpus and again without `ASN`. `Is Account Takeover` is loaded, listed in DROP, and never modelled.

Why it matters

The operational false-positive rate at threshold 0.5 is 0.2111, or 178,440 of 845,169 benign logins flagged. A risk-based-authentication system challenging one in five legitimate users would not ship, and it still records 0.0 recall for attack logins from six countries. Neither fact is visible in the 0.909410 headline.

Relevance and limits

Stands in for production RBA and impossible-travel login scoring, where the vendor claim is behavioural rather than reputational. It cannot stand in for cross-campaign detection: a model that learned which ASNs attacked in this capture has nothing to say about a fresh campaign from clean infrastructure.

What you are looking for. Train/test contamination is 0.737 and the duplicate rate is 0.897, so most held-out logins exactly repeat a training row. Separately, `family` holds ISO 3166-1 country codes from the `Country` column, not attack families. The per-group panel therefore exposes geographic bias and can never surface a weak attack technique. Both are disclosed; weigh them against the label's IP circularity rather than treat disclosure as a fix.

Check first: Establish what `family` actually contains before reading the six 0.0 recalls as evidence about attack types.

Schema — 16 columns, label “Is Attack IP”

Column names and types only. No rows of this corpus are hosted here.

Label distribution (proportions from the first 2,000 rows, not the full corpus): False 91.6% · True 8.3%

indexint64Login TimestampobjectUser IDint64Round-Trip Time [ms]float64IP AddressobjectCountryobjectRegionobjectCityobjectASNint64User Agent StringobjectBrowser Name and VersionobjectOS Name and VersionobjectDevice TypeobjectLogin SuccessfulboolIs Attack IPboolIs Account Takeoverbool

Read notebook PDF .ipynb ↑ Index

nb23 Cross-Dataset Intrusion Detection on NF-UQ-NIDS-v2 (Merged NetFlow Superset)

Cross-corpus network intrusion 4,000,000-flow stratified sample of 62.7M x 41 features, attack rate 0.669.

Corpus. The merged NF-UQ-NIDS-v2 superset, in which UNSW-NB15, Bot-IoT, ToN-IoT and CSE-CIC-IDS2018 are all re-expressed on one NetFlow v2 schema, taken from a cleaned 62.7M-row Kaggle asset. Kaggle: dhoogla/nfuqnidsv2

What it teaches

How to scope a claim to the split that produced it. A corpus built by merging four capture environments invites a generalisation claim that a pooled random split cannot support. The skill is naming the experiment that would answer the question and refusing to conflate it with the one that was run.

How it works

A stratified 4M sample is drawn so de-duplication and ablation stay tractable, and the `Dataset` source-identifier column is dropped as a capture-identity shortcut. Four learners are fit on a 120,000-row stratified subsample and scored on 1,000,000 held-out flows against a 0.6690 majority baseline. The audit computes solo feature AUC, exact-duplicate rate and train/test overlap on the first 50,000 test rows. The ablation refits on the de-duplicated corpus and again without `DURATION_IN`, and recall is broken out by attack family.

Why it matters

Infilteration recall is 0.168 where aggregate accuracy is 0.987315. Merging four environments does not repair the hard class; it dilutes it. A pooled benchmark can make a detector look more general than any of its parts while carrying the same blind spot each part had.

Relevance and limits

Stands in for the practical question behind schema standardisation: can one model serve heterogeneous environments? It cannot answer it. That requires train-on-one, test-on-another, which notebook 36 runs across four NetFlow-standardized corpora and this one explicitly defers.

What you are looking for. Nothing is concealed. Duplicate rate 0.000, contamination 0.000, and dropping the strongest single feature moves the AUC from 0.998721 to 0.998701. The 4M-of-62.7M sampling and the absence of a cross-environment split are both stated in the notebook, so reporting that the audit finds nothing is the correct answer here.

Check first: The loader prints 21 families while the audit counts 20 groups; resolve which value is the benign class before reading the per-family table.

Schema — 44 columns, label “Label”

Column names and types only. No rows of this corpus are hosted here.

Label distribution (proportions from the first 2,000 rows, not the full corpus): 0 33.8% · 1 66.2%

L4_SRC_PORTint16L4_DST_PORTint16PROTOCOLint16L7_PROTOfloat32IN_BYTESint32IN_PKTSint32OUT_BYTESint32OUT_PKTSint32TCP_FLAGSint16CLIENT_TCP_FLAGSint16SERVER_TCP_FLAGSint16FLOW_DURATION_MILLISECONDSint32DURATION_INint32DURATION_OUTint32MIN_TTLint16MAX_TTLint16LONGEST_FLOW_PKTint32SHORTEST_FLOW_PKTint16MIN_IP_PKT_LENint16MAX_IP_PKT_LENint32SRC_TO_DST_SECOND_BYTESfloat32DST_TO_SRC_SECOND_BYTESfloat32RETRANSMITTED_IN_BYTESint32RETRANSMITTED_IN_PKTSint16RETRANSMITTED_OUT_BYTESint32RETRANSMITTED_OUT_PKTSint16SRC_TO_DST_AVG_THROUGHPUTint64DST_TO_SRC_AVG_THROUGHPUTint64NUM_PKTS_UP_TO_128_BYTESint32NUM_PKTS_128_TO_256_BYTESint32NUM_PKTS_256_TO_512_BYTESint32NUM_PKTS_512_TO_1024_BYTESint32NUM_PKTS_1024_TO_1514_BYTESint32TCP_WIN_MAX_INint32TCP_WIN_MAX_OUTint32ICMP_TYPEint32ICMP_IPV4_TYPEint16DNS_QUERY_IDint32DNS_QUERY_TYPEint32DNS_TTL_ANSWERint32FTP_COMMAND_RET_CODEfloat32Labelint8AttackobjectDatasetobject

Read notebook PDF .ipynb ↑ Index

nb24 Static PE Malware Detection on EMBER-2018

Static PE malware 799,876 labelled PE files x 2341 features, malware rate 0.5000. EMBER's published feature count is 2,381; the loader prints 2341.

Corpus. EMBER-2018 v2 static feature vectors for Windows PE files, covering byte histograms, imports, header fields and string statistics, from the Kaggle parquet mirror. Kaggle: dhoogla/ember-2018-v2-features

Provenance chain. Fetched from the Kaggle account dhoogla. The corpus is credited to Anderson, H.S. & Roth, P. Criterion 1 asks you to establish whether those are the same party, and what follows if they are not.

What it teaches

How to grade a notebook that passes its own audit. Nothing inflates the score, so the failure to find is in a deliverable rather than in a number. The skill is spotting a section that reports a null quantity in the format of a result.

How it works

Rows with `Label == -1`, EMBER's unlabelled split, are filtered before anything is counted, and `family` is set to `df['y'].map({0:'benign',1:'malware'})`. Four learners are fit on a 120,000-row stratified subsample of a 75/25 split and scored on 199,969 held-out files against a 0.5000 majority baseline. The audit computes each feature's solo ROC-AUC, the exact-duplicate rate and train/test exact-row overlap. The ablation refits on the de-duplicated corpus and again without `F499`, the top solo feature at 0.7112.

Why it matters

Static PE classification is the highest-volume production deployment of ML in security. EMBER's own design point is a temporal split, train on earlier months and test on later; this notebook uses a random one. A 0.992918 AUC over a single collection year therefore says nothing about tomorrow's samples, and 0.0388 of benign files are flagged at threshold 0.5.

Relevance and limits

Stands in for the gradient-boosted static-feature detectors shipped inside endpoint products. It cannot stand in for packed or obfuscated malware, which only declares itself at runtime. The anonymised `F###` columns also block attributing any decision to imports, header anomalies or entropy.

What you are looking for. `family` is derived from the label and is binary, so section 9's per-family recall is one bar. The 0.958 is malware recall overall, not a weak-family finding. The `Label == -1` filter is disclosed in prose but its cost is never printed, so no output says how much of the asset was discarded. The validity audit itself is clean: duplicates 0.000, contamination 0.000, top solo feature 0.7112.

Check first: Trace where `family` is constructed before reading the per-family recall panel as evidence of anything.

Schema — 2382 columns, label “Label”

Column names and types only. No rows of this corpus are hosted here.

Label distribution (proportions from the first 2,000 rows, not the full corpus): 0.0 100.0%

F1float32F2float32F3float32F4float32F5float32F6float32F7float32F8float32F9float32F10float32F11float32F12float32F13float32F14float32F15float32F16float32F17float32F18float32F19float32F20float32F21float32F22float32F23float32F24float32F25float32F26float32F27float32F28float32F29float32F30float32F31float32F32float32F33float32F34float32F35float32F36float32F37float32F38float32F39float32F40float32F41float32F42float32F43float32F44float32F45float32F46float32F47float32F48float32F49float32F50float32F51float32F52float32F53float32F54float32F55float32F56float32F57float32F58float32F59float32F60float32

… and 2322 further columns.

Read notebook PDF .ipynb ↑ Index

nb25 Encrypted DNS-over-HTTPS Tunnel Detection (CIRA-CIC-DoHBrw-2020)

Encrypted traffic analysis 499,106 flows x 28 features; positive rate 0.5000, a balance BCCC constructed rather than observed.

Corpus. DNS-over-HTTPS captures from the CIRA/CIC 2020 testbed, reduced to per-flow timing and volume statistics, with tunnels built by dns2tcp, DNSCat2 and Iodine. Kaggle: bcccdatasets/bccc-cira-cic-dohbrw-2020

Provenance chain. Fetched from the Kaggle account bcccdatasets. The corpus is credited to MontazeriShatoori, M., Davidson, L., Kaur, G. & Habibi Lashkari, A. Criterion 1 asks you to establish whether those are the same party, and what follows if they are not.

What it teaches

Detection when the payload is unavailable. Every feature is a side channel - packet-size and inter-arrival statistics of an encrypted stream, not a protocol field an analyst can read. The skill is judging whether such a side channel identifies the technique or only the three tools that produced it.

How it works

Four learners share one stratified 75/25 split, with training capped at 120,000 rows and 124,777 held out. The label rule is one line, y = (Label != 'benign'); SourceIP, DestinationIP, TimeStamp and FlowID are dropped by name. The audit scores each of the 28 columns alone by ROC-AUC on a 60,000-row sample. It also measures the exact-duplicate row rate, and train/test row contamination on the first 50,000 held-out rows. The winner is refit twice: once de-duplicated, once with the strongest single column removed.

Why it matters

DoH removes the plaintext record most enterprise DNS monitoring depends on, so timing is what is left. A detector that works because dns2tcp paces packets a certain way dies when an adversary pads or paces deliberately. Knowing which of the two you have is the difference between a control and a demonstration.

Relevance and limits

This sits in the encrypted-traffic side-channel line - Anderson and McGrew on TLS, Bushart and Rossow on padded DNS. It stands in for tunnel detection without decryption. It cannot stand in for a general DoH detector. Three tunneling tools and a handful of resolvers is a closed world, and 0.5000 is not a field base rate.

What you are looking for. The contamination and shortcut probes come back empty. Exact-row overlap is 0.014, and dropping the strongest column leaves the held-out AUC at 0.999999, so reporting nothing found on that axis is the correct answer. The soft axis is the per-group recall check. This corpus carries exactly one positive family, so the panel prints a single bar at 1.0 and cannot speak to rare-class failure.

Check first: Count the bars in the per-family recall panel before accepting it as evidence of anything.

Schema — 29 columns, label “Label”

Column names and types only. No rows of this corpus are hosted here.

Label distribution (proportions from the first 2,000 rows, not the full corpus): Malicious 51.3% · Benign 48.6%

FlowBytesSentint64FlowSentRatefloat64FlowBytesReceivedint64FlowReceivedRatefloat64PacketLengthVariancefloat64PacketLengthStandardDeviationfloat64PacketLengthMeanfloat64PacketLengthMedianfloat64PacketLengthModeint64PacketLengthSkewFromMedianfloat64PacketLengthSkewFromModefloat64PacketLengthCoefficientofVariationfloat64PacketTimeVariancefloat64PacketTimeStandardDeviationfloat64PacketTimeMeanfloat64PacketTimeMedianfloat64PacketTimeModefloat64PacketTimeSkewFromMedianfloat64PacketTimeSkewFromModefloat64PacketTimeCoefficientofVariationfloat64ResponseTimeTimeVariancefloat64ResponseTimeTimeStandardDeviationfloat64ResponseTimeTimeMeanfloat64ResponseTimeTimeMedianfloat64ResponseTimeTimeModefloat64ResponseTimeTimeSkewFromMedianfloat64ResponseTimeTimeSkewFromModefloat64ResponseTimeTimeCoefficientofVariationfloat64Labelobject

Read notebook PDF .ipynb ↑ Index

nb26 DDoS Detection with Per-Type Recall on CIC-DDoS2019

Volumetric DDoS detection 431,371 flows x 65 features; positive rate 0.7732, so benign traffic is the minority class.

Corpus. CICFlowMeter flow records from the CIC 2019 DDoS testbed, taken from dhoogla's cleaned parquet release, spanning reflective/amplification and volumetric floods. Kaggle: dhoogla/cicddos2019

Provenance chain. Fetched from the Kaggle account dhoogla. The corpus is credited to Sharafaldin, I., Lashkari, A.H., Hakak, S. & Ghorbani, A.A. Criterion 1 asks you to establish whether those are the same party, and what follows if they are not.

What it teaches

The difference between a prediction target and a stratifier. The model fitted here is binary; the DDoS taxonomy never enters the fit and only slices recall afterwards. On an attack-heavy corpus accuracy is base-rate driven, so the informative metrics are per-type recall and the false-positive rate on benign flows.

How it works

Four learners share one stratified 75/25 split, training capped at 120,000 rows, 107,843 flows held out. The label rule is y = (Label != 'benign'); the raw Label is kept separately as family, giving 17 attack types plus Benign. Object columns with more than 50% unique values are dropped as identifier-like, and the rest are integer-encoded before the split. The audit scores every column alone, measures duplicate and train/test contamination rates, then refits the winner de-duplicated and with the top column dropped.

Why it matters

DDoS mitigation is a collateral-damage decision. The printed false-positive rate of 0.0011 on 24,458 benign flows tells you whether legitimate users get blocked; the aggregate AUC does not. That aggregate also conceals WebDDoS at recall 0.895 while every other type sits at 0.996 or above.

Relevance and limits

This is the flow-feature line of DDoS detection descending from Sharafaldin et al. It stands in for spotting volumetric and reflective floods from CICFlowMeter statistics. It cannot stand in for attack-type attribution, since no multi-class model is fitted anywhere, nor for low-rate or adaptive DDoS that never presents as a volume anomaly.

What you are looking for. The leakage probes return little: contamination 0.021, and the ablation refuses to move, whether one column goes or two. That flatness is the thing to judge, not to celebrate. It is equally consistent with a strong detector and with a testbed whose benign and attack distributions barely overlap. The notebook's own ablation section warns of exactly that reading.

Check first: Decide what an ablation that refuses to move actually proves, given how this corpus was generated.

Schema — 78 columns, label “Label”

Column names and types only. No rows of this corpus are hosted here.

Label distribution (proportions from the first 2,000 rows, not the full corpus): DrDoS_NTP 100.0%

Protocolint8Flow Durationint32Total Fwd Packetsint16Total Backward Packetsint16Fwd Packets Length Totalfloat32Bwd Packets Length Totalfloat32Fwd Packet Length Maxfloat32Fwd Packet Length Minfloat32Fwd Packet Length Meanfloat32Fwd Packet Length Stdfloat32Bwd Packet Length Maxfloat32Bwd Packet Length Minfloat32Bwd Packet Length Meanfloat32Bwd Packet Length Stdfloat32Flow Bytes/sfloat64Flow Packets/sfloat64Flow IAT Meanfloat32Flow IAT Stdfloat32Flow IAT Maxfloat32Flow IAT Minfloat32Fwd IAT Totalfloat32Fwd IAT Meanfloat32Fwd IAT Stdfloat32Fwd IAT Maxfloat32Fwd IAT Minfloat32Bwd IAT Totalfloat32Bwd IAT Meanfloat32Bwd IAT Stdfloat32Bwd IAT Maxfloat32Bwd IAT Minfloat32Fwd PSH Flagsint8Bwd PSH Flagsint8Fwd URG Flagsint8Bwd URG Flagsint8Fwd Header Lengthint64Bwd Header Lengthint32Fwd Packets/sfloat32Bwd Packets/sfloat32Packet Length Minfloat32Packet Length Maxfloat32Packet Length Meanfloat32Packet Length Stdfloat32Packet Length Variancefloat32FIN Flag Countint8SYN Flag Countint8RST Flag Countint8PSH Flag Countint8ACK Flag Countint8URG Flag Countint8CWE Flag Countint8ECE Flag Countint8Down/Up Ratiofloat32Avg Packet Sizefloat32Avg Fwd Segment Sizefloat32Avg Bwd Segment Sizefloat32Fwd Avg Bytes/Bulkint8Fwd Avg Packets/Bulkint8Fwd Avg Bulk Rateint8Bwd Avg Bytes/Bulkint8Bwd Avg Packets/Bulkint8

… and 18 further columns.

Read notebook PDF .ipynb ↑ Index

nb27 IoT Botnet Detection on N-BaIoT (Mirai / BASHLITE Device Behaviour)

IoT botnet detection 7,062,606 behaviour snapshots x 115 features; positive rate 0.9213, so benign is the minority class.

Corpus. Damped-window traffic statistics recorded from nine commercial IoT devices before and after laboratory infection with Mirai and BASHLITE. Kaggle: mkashifn/nbaiot-dataset

Provenance chain. Fetched from the Kaggle account mkashifn. The corpus is credited to . Criterion 1 asks you to establish whether those are the same party, and what follows if they are not.

What it teaches

The features are aggregate traffic statistics over decaying time windows, not protocol fields an attacker sets directly. That changes what evasion has to look like and what a feature importance means. The second skill is reading a pooled split as the weaker claim it is: nine devices in one pot cannot tell you whether a model reaches a tenth.

How it works

The label lives in the filename rather than a column - '<device>.benign.csv' versus '<device>.<mirai|gafgyt>.<subtype>.csv' - so the loader parses it and excludes the archive's three metadata CSVs. Four learners share one stratified 75/25 split, training capped at 120,000 rows, 1,765,652 snapshots held out. Device identity is never a feature, and family carries the botnet subtype. The audit scores each of the 115 columns alone, measures duplicate and contamination rates, then refits the winner de-duplicated and minus the top column.

Why it matters

Behavioural baselines are the standard answer for devices too small to carry an agent. Whether one survives a device it has never seen is the only question that matters operationally, and pooling makes that question unaskable. The notebook names cross-device transfer and does not run it.

Relevance and limits

This descends from Meidan et al.'s per-device deep autoencoders, an unsupervised and device-specific design; pooled supervised classification is a different, easier problem. It stands in for behavioural botnet detection on constrained hardware. It cannot stand in for cross-device or cross-firmware generalization, nor for a botnet variant absent from the 2018 capture.

What you are looking for. Every artifact test comes back clean, and that is the thing to interrogate. 64.8% of rows are exact duplicates, yet de-duplicating leaves the held-out AUC at 1.000000. Contamination is 0.066, and the strongest single column, at 0.9665, costs nothing to remove. The six worst family recalls all print 1.0 across ten scored families. Decide whether that describes a detector or a corpus whose classes hardly overlap.

Check first: Whether any split in this notebook trains on some devices and tests on others.

Schema — 115 columns

Column names and types only. No rows of this corpus are hosted here.

MI_dir_L5_weightfloat64MI_dir_L5_meanfloat64MI_dir_L5_variancefloat64MI_dir_L3_weightfloat64MI_dir_L3_meanfloat64MI_dir_L3_variancefloat64MI_dir_L1_weightfloat64MI_dir_L1_meanfloat64MI_dir_L1_variancefloat64MI_dir_L0.1_weightfloat64MI_dir_L0.1_meanfloat64MI_dir_L0.1_variancefloat64MI_dir_L0.01_weightfloat64MI_dir_L0.01_meanfloat64MI_dir_L0.01_variancefloat64H_L5_weightfloat64H_L5_meanfloat64H_L5_variancefloat64H_L3_weightfloat64H_L3_meanfloat64H_L3_variancefloat64H_L1_weightfloat64H_L1_meanfloat64H_L1_variancefloat64H_L0.1_weightfloat64H_L0.1_meanfloat64H_L0.1_variancefloat64H_L0.01_weightfloat64H_L0.01_meanfloat64H_L0.01_variancefloat64HH_L5_weightfloat64HH_L5_meanfloat64HH_L5_stdfloat64HH_L5_magnitudefloat64HH_L5_radiusfloat64HH_L5_covariancefloat64HH_L5_pccfloat64HH_L3_weightfloat64HH_L3_meanfloat64HH_L3_stdfloat64HH_L3_magnitudefloat64HH_L3_radiusfloat64HH_L3_covariancefloat64HH_L3_pccfloat64HH_L1_weightfloat64HH_L1_meanfloat64HH_L1_stdfloat64HH_L1_magnitudefloat64HH_L1_radiusfloat64HH_L1_covariancefloat64HH_L1_pccfloat64HH_L0.1_weightfloat64HH_L0.1_meanfloat64HH_L0.1_stdfloat64HH_L0.1_magnitudefloat64HH_L0.1_radiusfloat64HH_L0.1_covariancefloat64HH_L0.1_pccfloat64HH_L0.01_weightfloat64HH_L0.01_meanfloat64

… and 55 further columns.

Read notebook PDF .ipynb ↑ Index

nb28 Real E-Commerce Card-Fraud Detection on IEEE-CIS (Vesta)

E-commerce card fraud 590,540 transactions x 391 features; positive rate 0.0350, a production fraud base rate.

Corpus. Real labelled card transactions released by Vesta Corporation for the 2019 IEEE-CIS competition - production data, not a simulation. Kaggle: lnasiri007/ieeecis-fraud-detection

Provenance chain. Fetched from the Kaggle account lnasiri007. The corpus is credited to Dal Pozzolo, A., Boracchi, G., Caelen, O., Alippi, C. & Bontempi, G. Criterion 1 asks you to establish whether those are the same party, and what follows if they are not.

What it teaches

What a believable score looks like. The best single feature reaches 0.6882 alone and removing top columns barely moves the headline, which is the profile of distributed signal rather than a shortcut. The counterpart skill is refusing to stop at the aggregate: per-product recall here runs from 0.677 down to 0.177.

How it works

The label rule is y = df['isFraud']; family is ProductCD (W/C/H/R/S), a product code and not an attack taxonomy. TransactionID and the raw time offset TransactionDT are dropped as identifiers, and remaining object columns are integer-encoded before the split, which is transductive. Four learners share one stratified 75/25 split, training capped at 120,000 rows, 147,635 transactions held out. The audit scores each column alone, measures duplicate and contamination rates, then refits the winner de-duplicated and with C4 dropped, moving 0.919380 to 0.917591.

Why it matters

This is the only fraud corpus in the series carrying a real base rate and real analyst economics. A recall of 0.177 on one product code, sitting under a 0.919380 aggregate, is what a fraud team actually feels. Separately, a random split ignores the concept drift Dal Pozzolo et al. show dominates card fraud.

Relevance and limits

It stands in for supervised transaction scoring at realistic imbalance, with anonymized V-features you can audit but not name. It cannot stand in for a deployed system: no time-ordered evaluation and no cost-sensitive threshold. The negative class is 'not reported as fraud', which is not the same as verified legitimate.

What you are looking for. The contamination and shortcut probes find nothing. Overlap is 0.002, and dropping the top column moves the AUC only from 0.919380 to 0.917591, so reporting nothing found on leakage is the correct answer. What is soft is scope. The aggregate conceals a per-product recall of 0.177, and the time-ordered evaluation the literature calls the honest test is named and never run.

Check first: The distance between the 0.919380 headline and the worst per-product recall printed beneath it.

Schema — 394 columns, label “isFraud”

Column names and types only. No rows of this corpus are hosted here.

Label distribution (proportions from the first 2,000 rows, not the full corpus): 0 98.0% · 1 2.1%

TransactionIDint64isFraudint64TransactionDTint64TransactionAmtfloat64ProductCDobjectcard1int64card2float64card3float64card4objectcard5float64card6objectaddr1float64addr2float64dist1float64dist2float64P_emaildomainobjectR_emaildomainobjectC1float64C2float64C3float64C4float64C5float64C6float64C7float64C8float64C9float64C10float64C11float64C12float64C13float64C14float64D1float64D2float64D3float64D4float64D5float64D6float64D7float64D8float64D9float64D10float64D11float64D12float64D13float64D14float64D15float64M1objectM2objectM3objectM4objectM5objectM6objectM7objectM8objectM9objectV1float64V2float64V3float64V4float64V5float64

… and 334 further columns.

Read notebook PDF .ipynb ↑ Index

nb29 System-Log Anomaly Detection on HDFS (Loghub)

System-log anomaly detection 575,061 block sessions x 28 varying event-count features; positive rate 0.0293.

Corpus. Hadoop console logs from the Xu et al. HDFS study, shipped by Loghub already parsed into event templates and grouped by block id into per-session count vectors. Kaggle: tamaniwilliams/hdfs-v1-loghub-dataset-archive

Provenance chain. Fetched from the Kaggle account tamaniwilliams. The corpus is credited to Xu, W., Huang, L., Fox, A., Patterson, D. & Jordan, M.I. Criterion 1 asks you to establish whether those are the same party, and what follows if they are not.

What it teaches

Train/test contamination as the dominant failure mode, and how to measure it instead of assuming it away. Count vectors over a 29-template alphabet collide heavily, so 'held out' can quietly mean 'already seen'. A third lesson is reading class names honestly: these are Success and Fail outcomes, not benign and attack.

How it works

The label rule is y = (Label != 'success'), making Fail the positive class; family is the label-derived Type code, dropped from the features along with BlockId. Four learners share one stratified 75/25 split, training capped at 120,000 rows, 143,766 sessions held out. The audit scores each of the 28 event counts alone, then measures the exact-duplicate rate and train/test row contamination on the first 50,000 held-out rows. The winner is refit de-duplicated and with E9 removed.

Why it matters

A decade of log-anomaly papers quote near-perfect scores on this benchmark. This notebook shows how much of that is the same count vector appearing on both sides of the split. An AIOps model that memorises a few hundred distinct vectors is a lookup table, not a detector.

Relevance and limits

It stands in for the count-based lineage running from Xu et al.'s PCA detector through modern supervised baselines. It cannot stand in for order-sensitive detection: the count matrix discards event sequence, which is exactly what DeepLog exploits. No parser runs here either - the matrix ships pre-built.

What you are looking for. 99.89% of held-out rows are exact duplicates of training rows. The de-contaminated refit prints 0.983574; the corpus-wide duplicate rate prints 0.999. Separately, a number of anomaly types fall below the five-positive recall floor and are skipped - the mechanism is disclosed in the code, the count is never printed.

Check first: How many distinct rows the de-contaminated 0.983574 is actually fitted on.

Schema — 6 columns, label “Label”

Column names and types only. No rows of this corpus are hosted here.

Label distribution (proportions from the first 2,000 rows, not the full corpus): Success 97.7% · Fail 2.4%

BlockIdobjectLabelobjectTypefloat64FeaturesobjectTimeIntervalobjectLatencyint64

Read notebook PDF .ipynb ↑ Index

nb30 Industrial-IoT Intrusion Detection on WUSTL-IIoT-2021

Industrial IoT / OT 1,194,464 flows x 39 features; positive rate 0.0728, and the attack class is roughly 90% DoS.

Corpus. Network flow telemetry from a real water-storage-tank SCADA testbed at Washington University, labelled benign versus DoS, Reconnaissance, command-injection and Backdoor. Kaggle: annaamalaiu/wustl-iiot-2021-dataset

Provenance chain. Fetched from the Kaggle account annaamalaiu. The corpus is credited to Zolanvari, M., Teixeira, M.A., Gupta, L., Khan, K.M. & Jain, R. Criterion 1 asks you to establish whether those are the same party, and what follows if they are not.

What it teaches

That a reported data-trust grade is an estimate with a sampling distribution, not a property of the corpus. The audit's single-feature grade uses hard cutoffs at 0.85, 0.95, 0.99 and 0.999. A value near a boundary is a coin flip dressed as a letter. The OT counterpart lesson: the rare families are the safety-relevant ones.

How it works

The label rule is y = df['Target']; family is the Traffic column (normal / DoS / Reconn / CommInj / Backdoor). StartTime, LastTime, SrcAddr, DstAddr, Sport, Dport, sIpId and dIpId are dropped by name, so endpoint identity is excluded by rule rather than by hope. Four learners share one stratified 75/25 split, training capped at 120,000 rows, 298,616 flows held out. The audit scores each of the 39 columns alone, measures duplicate and contamination rates, then refits the winner de-duplicated and minus DIntPkt.

Why it matters

In an OT plant a false positive can trip a physical process and a missed command injection can damage equipment. Backdoor recall is 0.960 and CommInj 0.971 under a headline of 1.000000, and those two families run to a few hundred flows each. That is where the risk sits and where the sample is thinnest.

Relevance and limits

It stands in for flow-based intrusion detection in an operational-technology network, a setting thinly represented next to enterprise corpora. It cannot stand in for process-physics anomaly detection - there is no sensor or actuator telemetry here, only network flows. Nor for a plant running a different protocol mix.

What you are looking for. The notebook's own rule steps the data-trust grade down at a single-feature AUC of 0.99, and the value it prints is 0.9879. How close that sits to the cutoff is the question the notebook cannot answer about itself. Separately, the headline is 1.000000, and the ablation does not move it.

Check first: How far the single-feature AUC sits from the 0.99 cutoff that decides the grade.

Schema — 49 columns

Column names and types only. No rows of this corpus are hosted here.

StartTimeobjectLastTimeobjectSrcAddrobjectDstAddrobjectMeanint64Sportint64Dportint64SrcPktsint64DstPktsint64TotPktsint64DstBytesint64SrcBytesint64TotBytesint64SrcLoadfloat64DstLoadfloat64Loadfloat64SrcRatefloat64DstRatefloat64Ratefloat64SrcLossint64DstLossint64Lossint64pLossfloat64SrcJitterfloat64DstJitterfloat64SIntPktfloat64DIntPktfloat64Protoint64Durfloat64TcpRttfloat64IdleTimeint64Sumfloat64Minfloat64Maxfloat64sDSbint64sTtlint64dTtlint64sIpIdint64dIpIdint64SAppBytesint64DAppBytesint64TotAppByteint64SynAckfloat64RunTimefloat64sTosint64SrcJitActfloat64DstJitActfloat64TrafficobjectTargetint64

Read notebook PDF .ipynb ↑ Index

nb31 Cyber-Physical Intrusion Detection on the HAI ICS Testbed

ICS / cyber-physical OT 1,323,608 rows x 60 features; 0.68% positive.

Corpus. One-second process telemetry from the HAI hardware-in-the-loop testbed, which couples a GE gas turbine, an Emerson boiler and a FESTO water-treatment process; the loader reads the hai-21.03 release, not the narrower HAI 1.0. Kaggle: icsdataset/hai-security-dataset

Provenance chain. Fetched from the Kaggle account icsdataset. The corpus is credited to Shin, H.-K., Lee, W., Yun, J.-H. & Kim, H. Criterion 1 asks you to establish whether those are the same party, and what follows if they are not.

What it teaches

How to read a detection score when the majority-class accuracy is already 0.9932. Accuracy is dead on arrival here; per-stage recall and the false-positive rate are the only metrics that carry information. It also teaches that a multi-stage label must be assigned by counting flags, not by testing them in priority order.

How it works

Four learners - scaled logistic regression, RandomForest, XGBoost, LightGBM - are fitted on a 120,000-row stratified subsample of the 75% training half and scored on 330,902 held-out rows. The per-stage `family` column is built by summing the three `attack_P*` flags: more than one set becomes `stage_multi`, exactly one becomes that stage, none becomes `stage_unflagged`; those flags are then dropped from the features because they are label-derived. The audit computes three things: the best single-feature ROC-AUC over a 60,000-row sample, the whole-corpus exact-duplicate rate, and train/test exact-row contamination on the first 50,000 held-out rows. The grade is the worse of the first and third. The ablation refits the winning model twice: once on the de-duplicated corpus, once with the strongest single feature removed.

Why it matters

A false alarm on a running plant is not a ticket, it is a trip. The notebook prints 34 false positives out of 328,665 benign seconds, which is the number an operator would actually negotiate over, and it is a different quantity from the AUC. The structural limitation - each second classified independently, ignoring the process dynamics that residual and sequence detectors exploit - is the gap between this and a deployable cyber-physical detector.

Relevance and limits

This stands in for OT anomaly detection where sensor and actuator telemetry is the only observable, the regime Kravchik and Shabtai and the HAICon baselines work in. It cannot stand in for network-layer ICS detection, and it cannot stand in for temporal detection: a point-in-time classifier is a strictly weaker instrument than the sequence models the literature uses on this data.

What you are looking for. The audit finds nothing. Contamination is 0.007, the strongest single feature reaches 0.7465 alone, and neither de-duplication nor dropping that feature moves the held-out AUC by 0.001. Reporting 'nothing found' on the contamination and shortcut axes is the correct answer here. Two things are still open for judgement and neither is measured by the audit: the loader's comment credits its signal count to Shin et al. (2020), which documents HAI 1.0 rather than the hai-21.03 release actually read, and the shipped CSV header carries one more signal than the release paper states; and the split is a random i.i.d. draw over consecutive one-second samples of a continuous physical process.

Check first: Whether a random split over adjacent one-second samples of a continuous process can be called an independent test set, given the measured contamination is only 0.007.

Schema — 226 columns

Column names and types only. No rows of this corpus are hosted here.

TimestampobjectDM-PP01-Rint64DM-FT01Zfloat64DM-FT02Zfloat64DM-FT03Zfloat641001.2-OUTint641001.7-OUT1int641001.7-OUT2int641001.8-OUTint641002.2-OUTint641002.6-OUTint641002.11-OUT1int641002.11-OUT2int641002.12-OUTint641002.16-OUT1int641002.16-OUT2int641002.19-OUTint641002.29-OUTint641002.34-OUTint641003.7-OUTint641003.12-OUT1int641003.12-OUT2int641003.13-OUTint641003.27-OUTint641004.11-OUT1int641004.11-OUT2int641004.12-OUT1int641004.12-OUT2int641004.13-OUTint641004.15-OUT1int641004.15-OUT2int641004.18-OUT1int641004.18-OUT2int641004.21-OUTint641004.24-OUTint641004.29-OUTint641004.36-OUTint641004.37-OUTint641004.38-OUTint641004.39-OUTint641004.41-OUTint641004.44-OUTint641004.52-OUTint641004.53-OUTint641004.62-OUTint641004.76-OUTint641004.78-OUTint641004.79-OUTint641004.80-OUTint641010.2-OUTint641010.3-OUTint641010.4-OUTint641010.5-OUT1int641010.5-OUT2int641010.5-OUT3int641010.5-OUT4int641010.7-OUTint641010.8-OUTint641010.9-OUTint641010.10-OUTint64

… and 166 further columns.

Read notebook PDF .ipynb ↑ Index

nb32 In-Vehicle CAN-Bus Intrusion Detection (Car-Hacking / HCRL)

In-vehicle CAN bus 3,931,517 rows x 10 features; 59.3% positive.

Corpus. Real in-vehicle CAN-bus traffic from a Hyundai YF Sonata (HCRL), carrying DoS, fuzzing, gear-spoofing and RPM-spoofing injections; each row is one frame, an arbitration ID plus DLC and up to eight payload bytes. Kaggle: pranavjha24/car-hacking-dataset

Provenance chain. Fetched from the Kaggle account pranavjha24. The corpus is credited to Song, H.M., Woo, J. & Kim, H.K. Criterion 1 asks you to establish whether those are the same party, and what follows if they are not.

What it teaches

How to tell duplicate inflation apart from genuine separability, and how to check that the ablation meant to make that distinction can actually make it. The feature space here is ten low-cardinality integers, so exact-row repetition is structural rather than accidental. Reading an ROC-AUC of exactly 1.000000 as a statement about the frame alphabet, not about the detector, is the skill.

How it works

Every attack frame is kept and normal frames are capped at 400,000 per capture file. Four learners then fit on a 120,000-row stratified subsample and are scored on 982,880 held-out rows. The timestamp is dropped, so the model sees only the arbitration ID, the DLC and the eight payload bytes; hex fields are parsed to integers and unparseable cells go to zero. The audit reports the best single-feature AUC (0.7113 on payload byte b3), the whole-corpus duplicate rate (0.853) and train/test exact-row contamination (0.811), taking the worse of the two grades. The ablation refits the winner on `X.drop_duplicates()` and again with b3 removed.

Why it matters

The CAN bus has no authentication, so any node that reaches it can inject. This notebook solves the easy half of that problem to saturation: injected frames carry anomalous payloads. The saturation is precisely why the result cannot be extrapolated. Dropping timing removes the signal that flooding and masquerade detection actually rely on.

Relevance and limits

It stands in for payload-based CAN IDS, the per-frame classifier baseline that CNN and GAN approaches on this corpus are measured against. It cannot stand in for timing or sequence detection. Nor can it speak to masquerade and replay attacks that reuse valid IDs with plausible payloads, the case where payload-only detection has nothing left to look at.

What you are looking for. 85.3% of rows are exact duplicates and 81.1% of held-out rows also appear in training, so the split is largely memorised. The notebook's defence is that de-duplication leaves the AUC at 1.0, which would mean the separability is real. But `X.drop_duplicates()` keeps the first occurrence of each feature vector and takes that row's label; vectors that appear under both labels are silently resolved to whichever came first rather than reconciled. Decide whether an ablation that deletes the ambiguous cases can support the conclusion it is being asked to support.

Check first: How many distinct feature vectors carry both labels, and what `X.drop_duplicates()` does to them before the de-duplicated AUC is computed.

Schema — 12 columns

Column names and types only. No rows of this corpus are hosted here.

1478191030.045114float640316object8int6405object22object68object09object22.1object20object00object75objectRobject

Read notebook PDF .ipynb ↑ Index

nb33 Cyber-Physical Intrusion Detection on the SWaT Water-Treatment Testbed

ICS / water treatment 1,441,719 rows x 46 features; 3.79% positive.

Corpus. The SUTD Secure Water Treatment testbed: a real six-stage water-treatment plant run for eleven days with staged cyber-attacks. The original is request-gated by iTrust under terms that forbid redistribution, so no excerpt is published here. The loader reads a third-party Kaggle mirror whose fidelity to the gated original is unverifiable. Kaggle: vishala28/swat-dataset-secure-water-treatment-system

Provenance chain. Fetched from the Kaggle account vishala28. The corpus is credited to Goh, J., Adepu, S., Junejo, K.N. & Mathur, A. Criterion 1 asks you to establish whether those are the same party, and what follows if they are not.

What it teaches

What a per-group recall breakdown is worth when the group column is derived from the label it is supposed to stratify. Here `family` is assigned as attack where the label is 1 and normal otherwise, so the breakdown is a restatement of overall attack recall and adds nothing. Recognising a vacuous group split before quoting it is the skill this notebook exercises.

How it works

The merged CSV is read, the `Normal/Attack` column becomes the binary label, and the timestamp plus label columns are dropped; object-typed identifier columns and constant columns are then removed, leaving 46 signals. Four learners fit on a 120,000-row stratified subsample and are scored on 360,430 held-out rows, with the majority-class accuracy of 0.9621 printed inside the ranking table. The audit computes the best single-feature AUC (0.9002 on PIT503), the duplicate rate (0.356) and train/test exact-row contamination (0.062), grading on the worse. The ablation refits after de-duplication, which removes 36% of rows, and again with PIT503 dropped.

Why it matters

At a 3.79% attack rate the accuracy column is decoration; the 63 false positives out of 346,775 benign seconds are the operational quantity, because a false trip disrupts water treatment. This is also the second physically distinct ICS testbed in the series, which makes the SWaT-to-HAI transfer question concrete - and that transfer is exactly what neither notebook measures.

Relevance and limits

It sits alongside the Kravchik-Shabtai line of work on SWaT, as the point-in-time baseline those residual and 1D-CNN detectors improve on. It stands in for process-signal anomaly detection on a single multi-stage plant. It cannot stand in for attack characterisation: the merged file carries no per-scenario type, so nothing here distinguishes one manipulation from another.

What you are looking for. Two things, and the first is the data itself. The notebook's outputs come from a Kaggle mirror that iTrust's terms permit nobody to have published, so provenance is unresolved before you score anything else. Section 3 gives you the verified invariants to test a legitimately obtained copy against: 1,387,098 normal rows, 54,621 attack rows, 1,441,719 combined, attack rate 0.0379. The second is that `family` is a restatement of the label, so the per-group recall table has one row and carries no information beyond overall attack recall.

Check first: Request the original from iTrust with your academic address, rebuild merged.csv from the two workbooks, and check whether your row counts and attack rate match the printed ones.

Schema — 53 columns

Column names and types only. No rows of this corpus are hosted here.

TimestampobjectFIT101float64LIT101float64 MV101float64P101int64P102int64 AIT201float64AIT202float64AIT203float64FIT201float64 MV201float64 P201float64 P202float64P203int64 P204float64P205int64P206int64DPIT301float64FIT301float64LIT301float64MV301int64MV302int64 MV303float64MV304int64P301int64P302int64AIT401float64AIT402float64FIT401float64LIT401float64P401int64P402int64P403int64P404int64UV401int64AIT501float64AIT502float64AIT503float64AIT504float64FIT501float64FIT502float64FIT503float64FIT504float64P501int64P502int64PIT501float64PIT502float64PIT503float64FIT601float64P601int64P602int64P603int64Normal/Attackobject

Read notebook PDF .ipynb ↑ Index

nb34 In-Vehicle CAN Intrusion Detection on OTIDS (Second Vehicle)

In-vehicle CAN bus 3,744,041 rows x 11 features; 59.94% positive.

Corpus. Real CAN frames from a second HCRL vehicle, distinct from the Car-Hacking capture: a normal file plus three attack captures covering DoS, fuzzy and impersonation, each row an arbitration ID with eight payload bytes. Kaggle: bikashkundu/can-hcrl-otids

Provenance chain. Fetched from the Kaggle account bikashkundu. The corpus is credited to Lee, H., Jeong, S.-H. & Kim, H.K. Criterion 1 asks you to establish whether those are the same party, and what follows if they are not.

What it teaches

How to read an ablation whose result moves in the wrong direction. De-duplication is supposed to remove inflation, so a contaminated headline should fall; here it rises from 0.954777 to 0.996659. The skill is diagnosing that the ablation changed the task rather than cleaned the estimate, and refusing to bank the higher number.

How it works

All attack frames are kept and normal frames are capped at 1,500,000, so the loaded total is bounded, not the archive size. Timestamp and label are dropped, leaving the CAN ID and payload bytes; four learners fit on a 120,000-row stratified subsample and are scored on 936,011 held-out rows, XGBoost winning at 0.954777 AUC and 0.858807 accuracy. Recall is reported per attack capture - 0.809, 0.896, 0.944 - rather than per named attack, because the file-to-attack mapping is not verifiable. The audit gives a best single-feature AUC of 0.5773 on ID1, a duplicate rate of 0.903 and train/test contamination of 0.830; the ablation refits on the de-duplicated corpus (90% of rows removed) and without ID1 (0.946496).

Why it matters

Impersonation reuses legitimate arbitration IDs, which is exactly where payload-only detection has the least to work with, and it is the attack Car-Hacking lacks. The printed false-positive rate is 0.1586 - 59,472 benign frames flagged out of 375,000 - which is not a deployable operating point on a vehicle bus. Whether frame-level features learned on one vehicle survive on another is the question this corpus makes askable, and the notebook does not answer it.

Relevance and limits

As the cross-vehicle companion to the Car-Hacking notebook, it stands in for the generalisation question in automotive IDS: are payload features vehicle-specific? It cannot stand in for the detector OTIDS was actually built to support - Lee et al. use request/response time intervals and offset ratios, and timing is dropped here entirely.

What you are looking for. 83.0% of held-out rows are duplicates of training rows, a fully memorised split. The de-duplication ablation then makes things worse, not better: the loader concatenates every attack frame above every normal frame, and `X.drop_duplicates()` keeps the first occurrence, so every feature vector that appears under both labels resolves to `attack`. The conflicted rows - the ones no frame-level detector can get right - are deleted rather than reconciled, which makes 0.996659 an upper bound on an easier task. The notebook states this caveat; check that its own ledger obeys it.

Check first: Why the de-duplicated AUC went up rather than down, and which direction the label conflicts were resolved in when it did.

Schema — 11 columns

Column names and types only. No rows of this corpus are hosted here.

TSfloat64ID1int64DLC0int64DLC1int64DLC2int64DLC3int64DLC4int64DLC5int64DLC6int64DLC7int64targetint64

Read notebook PDF .ipynb ↑ Index

nb35 Android Malware Detection & Category Characterization (CCCS-CIC-AndMal-2020)

Mobile / Android malware 357,805 rows x 9,417 features; 54.67% positive.

Corpus. Static-analysis features of Android applications - permissions, intents, components and API-call flags, no execution - from CCCS-CIC-AndMal-2020, contrasting benign apps with 14 malware categories. Kaggle: dhoogla/cccscicandmal2020

Provenance chain. Fetched from the Kaggle account dhoogla. The corpus is credited to Rahali, A., Lashkari, A.H., Kaur, G., Taheri, L., Gagnon, F. & Massicotte, F. Criterion 1 asks you to establish whether those are the same party, and what follows if they are not.

What it teaches

This is a case where the de-duplication ablation actually bites and the notebook is obliged to change its own answer: 0.997291 becomes 0.984201 once repeated rows are removed. The skill is carrying that revision through every downstream claim rather than quoting it once and reverting. The wide-and-short shape - 9,417 columns over 357,805 rows - is also a different failure regime from the ten-column CAN corpora.

How it works

The static-features parquet is loaded, the label column becomes benign versus malware, and the label string is retained as `family` so recall can be reported per malware category. Four learners fit on a 120,000-row stratified subsample and are scored on 89,452 held-out rows; RandomForest wins at 0.997291 AUC and 0.980716 accuracy against a 0.5467 majority baseline. The audit reports the best single-feature AUC (0.8737 on F50), a duplicate rate of 0.519 and train/test contamination of 0.514. The ablation refits on the de-duplicated corpus, which drops 52% of rows and yields 0.984201, and again without F50, which yields 0.997303.

Why it matters

Static triage is how app stores scan at volume. The weakest recalls land on the uncategorised bucket at 0.806, PUA at 0.898, and zero-day and backdoor at 0.937 - the apps a store most needs to stop. That half the apps share an identical 9,417-dimensional feature vector with another app is itself a finding about the feature set, not only about the split. A random split also ignores the temporal drift that TESSERACT shows dominates real Android-malware deployment.

Relevance and limits

It stands in for pre-execution static triage of mobile applications, the Drebin lineage, and it is a distinct domain from PE malware. It cannot stand in for dynamic or behavioural analysis, which is how packing and obfuscation defeat static features. Nor can it stand in for a time-aware evaluation: the split is random, so the number says nothing about tomorrow's samples.

What you are looking for. Train/test contamination is 51.4%, so roughly half the held-out apps have an exact feature-vector twin in training. The notebook handles this correctly: it names 0.984201, the de-duplicated figure, as the honest number rather than the 0.997291 headline. That leaves a narrower question for the reader. The per-category recalls and the false-positive rate were computed on the contaminated split and never recomputed after de-duplication. Decide which of the notebook's reported quantities the correction actually reached.

Check first: Whether the per-category recalls quoted in the conclusion were recomputed on the de-duplicated corpus or carried over unchanged from the 51.4%-contaminated split.

Schema — 9505 columns, label “Label”

Column names and types only. No rows of this corpus are hosted here.

Label distribution (proportions from the first 2,000 rows, not the full corpus): Benign 100.0%

F0objectF1int16F2int16F3int16F4int16F5int8F6int8F7int8F8int8F9int8F10int8F11int8F12int8F13int8F14int8F15int8F16int8F17int8F18int8F19int8F20int8F21int8F22int8F23int8F24int8F25int8F26int8F27int8F28int8F29int8F30int8F31int8F32int8F33int8F34int8F35int8F36int8F37int8F38int8F39int8F40int8F41int8F42int8F43int8F44int8F45int8F46int8F47int8F48int8F49int8F50int8F51int8F52int8F53int8F54int8F55int8F56int8F57int8F58int8F59int8

… and 9445 further columns.

Read notebook PDF .ipynb ↑ Index

nb36 Does It Transfer? Cross-Dataset Intrusion Detection on One Schema

Cross-corpus NIDS transfer 1,600,000 rows x 41 features; 46.94% positive pooled, with per-corpus attack rates of 0.9957, 0.7258, 0.0378 and 0.1184.

Corpus. Four intrusion corpora - BoT-IoT, ToN-IoT, UNSW-NB15 and CSE-CIC-IDS2018 - re-featured by Sarhan et al. into a single NetFlow v2 schema, sampled to 400,000 stratified flows each. Kaggle: dhoogla/nfbotiotv2

What it teaches

The difference between a benchmark score and a detector. Holding the feature schema fixed and varying only the capture isolates distribution shift from schema mismatch, which is what makes the comparison interpretable at all. It also teaches how to read a transfer AUC below 0.5: not a weak model, but a learned rule that is inverted on the new corpus.

How it works

This is the only notebook in the set that runs a cross-corpus matrix. Each corpus is split 75/25 once, before any model is fitted, then one XGBoost - 120 trees, depth 6 - is fitted on 120,000 rows of that corpus alone. Every model is then scored on every corpus: the diagonal on that corpus's own held-out quarter, the off-diagonal on the full 400,000-row target sample, which the model has never seen by construction. Sections 8 to 12 rerun the standard pooled-split rig, audit and ablation as a warm-up, with `family` set to the source corpus so the per-group recall reads per-corpus.

Why it matters

Every other notebook in the series ends by naming the cross-distribution test as owed and not run. This one runs it. The diagonal mean is 0.9966; the off-diagonal mean is 0.5434, with 5 of 12 cells below random. The worst cell, train BoT-IoT and test ToN-IoT, sits at 0.3315. That is the quantitative form of the claim that a near-perfect in-distribution number is a property of a capture rather than of a detector.

Relevance and limits

This is where the NIDS literature's routine 0.99s meet Sommer and Paxson and the Arp et al. pitfall catalogue, and it is the capstone the other 35 notebooks should be read against. It stands in for moving a trained detector onto a different network, attack generator and year. It cannot stand in for temporal drift inside one network, nor for adversarial evasion. The off-diagonal mean averages twelve cells; it is not a forecast for any specific deployment.

What you are looking for. The pooled audit is genuinely clean: a 0.000 duplicate rate, 0.000 train/test contamination, a strongest single feature at 0.8643, and an ablation that moves nothing. Reporting 'nothing found' on the contamination and shortcut axes is the correct answer here. Two things are still open. The matrix reports ROC-AUC and nothing else, so no cell converts into an operating point. And the section 8 headline of 0.999200 comes from pooling all four corpora and splitting at random, the most flattering construction available. It sits inside the one notebook whose thesis is that such numbers mislead. The notebook calls it a warm-up; decide whether it belongs there at all.

Check first: That the transfer matrix reports ROC-AUC only - no threshold, no false-positive rate, no per-corpus recall - and what that forbids you from concluding from it.

Schema — 43 columns, label “Label”

Column names and types only. No rows of this corpus are hosted here.

Label distribution (proportions from the first 2,000 rows, not the full corpus): 0 0.4% · 1 99.6%

Read notebook PDF .ipynb ↑ Index