Security tools, checked against what they claim.
I'm Sami Ali, a student at George Mason University (class of 2028). I write detection rules, code analyzers and vulnerability-ranking tools, then test them against something other than my own opinion.
A public SigmaHQ rule is meant to notice a program being set to run with its owner's privileges. I replayed a real recording of that attack, 36 events with 2 attack steps in it, and the rule caught neither. The rule I wrote catches both.
Among 36 events recorded from Atomic Red Team's T1548.001 tests on a GitHub-hosted Linux runner, 2 are the attack itself (the two chmod executions). SigmaHQ's stock Setuid and Setgid rule flagged 0 of those 2; the replacement rule flags both. The other 34 events are not attack events.
As found: fail recording run 41838714456 failed 2026-11-04 main CI run 43100952362 passed 2026-11-11
The recording run's gate failed because the stock rule missed the attack: that failure is the finding. The second run is the latest passing CI run on main, which replays the committed recording with the replacement rule.
- not expected and not matched
- expected but not matched
- matched
0 of 2 attack events flagged, among 36 recorded events
2 of 2 attack events flagged, among 36 recorded events
The work
The strips are the check: top row as found, bottom row as left.
tripline v0.3.0
Detection engineering
When a detection rule changes, this replays recorded attacks through it and fails the build if the rule stays quiet.
Reading 0 to 2 of 2 attack events
sinkline v0.5.1
Application security
Reads Python web code and flags places where request data reaches a dangerous call, such as a SQL query, without being cleaned first.
Reading +0.21 to +0.52 (not held out)
vulnlab v0.1.1
Web security training lab, Python
Three small web apps with one planted bug each, a fix for each, and tests that show the damage and then show the fix stopping it. It runs only on your own machine.
Reading 3 of 3 fixes verified; sinkline 2 of 3 detected (2 false positives on fixes)
kevgate v0.1.1
Vulnerability management and DevSecOps
Takes vulnerability scanner output and returns a fix-first list, with bugs attackers are already exploiting at the top.
Reading 6 of 6 mutants killed
Smaller, and still checked
kevgate-admit v0.1.0
Kubernetes security, Go
A gatekeeper for Kubernetes that refuses, or warns about, workloads whose container images carry known-exploited vulnerabilities.
Reading 5 of 5 review findings closed
fuzzdiff v0.2.0
Memory safety and fuzzing, Rust
A web-request parser rewritten from C into memory-safe Rust, checked by feeding both versions the same bytes and listing every difference.
Reading 5 of 5 seeded bugs detected
kev-risk-prioritization v0.3.0
Vulnerability management
A short Python script that ranks scan findings so the known-exploited ones come first.
Reading 12 vs 9 KEV in the top 12 (synthetic demo)
Experience
U.S. Department of Homeland Security (CBP), HS-POWER Program, Vulnerability Assessment Team analyst intern, May to August 2026. Matched Nessus/Tenable scan results against the CISA KEV catalog, designed a risk-based prioritization method, and built a Power BI dashboard and executive briefing. Details
Education and certifications
- B.S. Information Technology, Cybersecurity Minor, George Mason University, expected May 2028
- CompTIA Security+ and AWS Certified Cloud Practitioner
- In progress: OSCP and AWS Certified Security - Specialty
The evidence
The numbers above come from these two tables and the CI runs they link to. Both are complete; they are closed so the page stays readable.
Record of checksWhat each of the 7 projects was checked againstThe strips show the same units as the project's own page, reduced to at most 24 ticks: top row as found, bottom row as left.
| Instrument | Checked against | As found, then as left | Reading | Evidence |
|---|---|---|---|---|
| triplinev0.3.0 | SigmaHQ stock rule; 2 attack events among 36 recorded events | 0 to 2 of 2 attack events | run 43100952362 passed 2026-11-11 | |
| sinklinev0.5.1 | OWASP BenchmarkPython v0.1, 10 taint categories; Bandit +0.07 | +0.21 to +0.52 (not held out) | run 43136290658 passed 2026-11-12 | |
| vulnlabv0.1.1 | Each lab's fixed service and its tests; sinkline 0.5.0; Sigma rules replayed on recorded logs | 3 of 3 fixes verified; sinkline 2 of 3 detected (2 false positives on fixes) | run 43134837698 passed 2026-11-12 | |
| kevgatev0.1.1 | Its own six surviving mutants and 16 controls; kind cluster in CI | 6 of 6 mutants killed | run 42960808156 passed 2026-11-11 | |
| kevgate-admitv0.1.0 | Python score on 1000 vectors; synthetic and real Trivy reports; kind in CI | 5 of 5 review findings closed | run 43081479456 passed 2026-11-11 | |
| fuzzdiffv0.2.0 | The C original and httparse, same bytes | 5 of 5 seeded bugs detected | run 43037960324 passed 2026-11-11 | |
| kev-risk-prioritizationv0.3.0 | CVSS-only sort of 20 synthetic findings; pinned real KEV and EPSS snapshot | 12 vs 9 KEV in the top 12 (synthetic demo) | run 43134445948 passed 2026-11-12 |
What failed first16 problems found first, each copied from the repository text it links toWhat was broken or missing before the fix.
| Instrument | What the check found | State | Source |
|---|---|---|---|
| tripline | The stock SigmaHQ Setuid and Setgid rule flagged 0 of the 2 attack events (the two chmod executions) among 36 recorded events. A replacement rule flags both. | Fixed | docs/findings.md, F-001 |
| tripline | Weighted coverage is 98% (61 of 62 points), but only 52% (32 of 62, 8 scenarios) passes on unmodified SigmaHQ rules alone. The other 5 passing scenarios (29 points) need a rule written in the tripline repository. | Read with the split | README, scorecard |
| tripline | Against 251 generated command-line variants of three Linux attacks, the expected rules catch 200. The stock setuid rule catches 0 of 108 setuid variants; the replacement catches 106. | Open | docs/robustness.md |
| tripline | On a 675-event benign Linux corpus, SigmaHQ's New Cron File Created matched 2 events (a chkrootkit install). 20 of 26 rules, all Windows and AWS, were not evaluated. | Open | docs/benign-corpus.md |
| tripline | 1 of the 7 Windows scenarios (T1027) has no accepted recording and counts as zero in both the 98% and the 52%. | Pending | F-003 |
| sinkline | On the independent OWASP BenchmarkPython suite, 0.4.1 scored +0.21 on the 10 taint categories, with 0 of 16 LDAP and 5 of 51 XPath cases reported. 0.5.1 scores +0.52 (the same per-test table as 0.5.0), but the fixes were chosen from those failures, so it is not a held-out score. | Not held out | bench/external/README.md |
| sinkline | Over all 14 categories Bandit leads (+0.36 against +0.27): sinkline has no check for weak hashes, weak random numbers, cookie flags or session contents. | Open | results/v0.5.0 (0.5.1 rows are identical) |
| sinkline | The GitHub Action put its inputs into a shell line, a script-injection flaw in its own code. Fixed in 0.4.1. | Fixed | CHANGELOG 0.4.1, Security |
| kevgate | Six deliberate defects survived 227 tests, including swapping the constant-time key comparison for ==. The code was correct, but no test would have failed. 0.1.1 adds tests that kill all six. | Fixed | CHANGELOG 0.1.1 |
| kevgate-admit | The image parser accepted ghcr.io/org/../evil, and ignoreUnfixed could let a KEV finding with no fix pass denyKev. Both are fixed in v0.1.0. | Fixed | CHANGELOG 0.1.0, Hardening |
| vulnlab | sinkline reports the fixed versions of the SSRF and SQL injection labs too, because it does not recognize a custom allow-list or address check as a sanitizer, and reports nothing on the IDOR lab, which is not a taint flow. | Open | README, labs and results |
| vulnlab | A review of 0.1.0 found that the fixed SSRF service answered 500 to malformed URLs, that both IDOR variants issued a login token for an unknown user with a null password, and that no test failed if the fixed SSRF service's address check was removed. All three are fixed in 0.1.1 and each has a test. | Fixed | CHANGELOG 0.1.1 |
| fuzzdiff | A mutation sweep found two one-line changes (a chunk-overhead ratio and a status-line guard) that none of the 0.1.0 tests noticed. 0.2.0 names them as seeded bugs and adds cases that catch them. | Fixed | README, seeded bugs |
| fuzzdiff | The harness treated any folded-header error in strict mode as the documented deviation, which would have hidden a real bug. The exception is now narrowed. | Fixed | pull request #2 |
| kev-risk-prioritization | Earlier text called the score "in the spirit of" CISA BOD 26-04, and a measurements table reported properties of invented values. Both were corrected: it is a weighted score of its own design, and the table was removed. | Fixed | CHANGELOG 0.2.0, Corrections |
| kev-risk-prioritization | In 0.2.0 a finding without CVSS was scored on a different scale from one with it (the same KEV finding scored 73.5 without CVSS and 59.5 with CVSS 0.0), so a mixed list was ranked with incomparable scores. 0.3.0 refuses such a list unless --allow-mixed, which scores every row without CVSS. | Fixed | CHANGELOG 0.3.0 |