As foundThe 0.1.0 tests would not have noticed six defects, and five gaps remained
Coverage numbers do not show whether a test would notice a defect being introduced. The review round after the first release (v0.1.0) asked of each control what would fail if it broke, and found six places where nothing would, plus five gaps in how the controls were observed or built.
The first finding is about the tests, not the code. v0.1.0 already compared keys with hmac.compare_digest and no early exit; the 227 tests would still have passed if someone had replaced it with ==. Each of the six findings below was checked again on 2026-11-11 against the v0.1.0 tag.
Table, 6 rows: Reading ledger: what the review found in v0.1.0
| Check | Expected | Observed at v0.1.0 | Verdict | Evidence |
|---|---|---|---|---|
| Six mutants against the suite | A failing test for each | All 227 tests passed with each mutant applied, including mutants of the key comparison (an early exit, and == in place of compare_digest; the shipped code used compare_digest) and a body-size limit that counted only the last chunk | As found: fail | rerun script (below) |
| Alert after a restart | The first authentication failure after a restart is counted | The labelled kevgate_auth_failures_total series only came into existence on the first failure. The repository's comment added in 0.1.1 says Prometheus' increase() then has no earlier sample, so the first event would never alert | As found: fail | metrics.py at v0.1.0 |
| Readiness | Not ready when storage cannot take a write | /readyz ran SELECT 1 and checked that the KEV catalog loaded. It did not look at free space, a failing write or database integrity | As found: fail | app.py at v0.1.0 |
| Failed authentication | Repeated bad keys from one source are throttled | A bad or missing key was counted and refused with 401, with no limit per source address. The per-key bucket applied only after a key authenticated | As found: fail | require() at v0.1.0 |
| Dependency lock | A committed lock with hashes, installed with --require-hashes by CI | requirements.lock was not in the repository. The security workflow generated it on each run with deploy/scripts/lock.sh, and ci installed with pip install -e ".[dev]", without hashes | As found: fail | ci.yml at v0.1.0 |
| OIDC trust policy | Weakening the policy fails a check | No test pinned it. The repository's own text says validate, tflint, checkov and Trivy accept a trust policy weakened to StringLike with a wildcard, so none of them would have caught it | As found: fail | DEPLOYMENT.md |
Table, 6 rows: The six mutants, re-run on 2026-11-11 against both tags (one change at a time to the source, full suite, stop at the first failure)
| Mutant (a deliberate defect) | v0.1.0, 227 tests | v0.1.1, 277 tests |
|---|---|---|
| Early-exit key comparison | Survived | Killed |
== in place of hmac.compare_digest | Survived | Killed |
BEGIN instead of BEGIN IMMEDIATE | Survived | Killed |
| Body size counted from the last chunk only | Survived | Killed |
| CSV merge taking the minimum CVSS | Survived | Killed |
| Upsert skipping EPSS | Survived | Killed |
As leftSix mutants killed, five gaps closed, controls cited to tests and CI
Release 0.1.1 adds the tests that kill the six mutants (tests/test_mutation_kills.py, 227 tests become 277) and closes the other five findings. Each change below is in the repository's CHANGELOG and is cited by a test or a CI step in the matrix above.
Table, 6 rows: Reading ledger: what 0.1.1 changed
| Check | Expected | Observed at v0.1.1 | Verdict | Evidence |
|---|---|---|---|---|
| Six mutants against the suite | A failing test for each | Each mutant makes at least one of the 277 tests fail | As left: pass | the table above; run 42960808156 passed 2026-11-11 |
| Alert after a restart | The first failure is counted | All three labelled series exist at 0 from startup (shown below) | As left: pass | test_labelled_counters_exist_at_zero_on_startup |
| Readiness | Not ready when storage cannot take a write | /readyz also checks free space (1 MiB), a write taken inside a transaction that is rolled back, and a cached quick_check that is time-boxed | As left: pass | tests/test_readiness.py |
| Failed authentication | Repeated bad keys from one source are throttled | A per-source failure limiter is checked before any key is compared. X-Forwarded-For is read only for peers in KEVGATE_TRUSTED_PROXIES (shown below) | As left: pass | tests/test_auth_failures.py |
| Dependency lock | A committed lock with hashes, installed with --require-hashes | requirements.lock is committed; ci, the image build and pip-audit use it. Dependabot is configured for pip, Actions and Docker | As left: pass | run 42960808156 passed 2026-11-11 (ci), run 42960808037 passed 2026-11-11 (security) |
| OIDC trust policy | Weakening the policy fails a check | A terraform test pins one Allow, two StringEquals conditions, aud and one exact sub for the repository and branch, with no wildcard. 4 test runs pass | As left: pass | run 42956666367 passed 2026-11-11 (terraform) |
Run on 2026-11-11 from a fresh clone at tag v0.1.1 (commit 973c8c1), Python 3.12.2, on Windows 11, on localhost (port 8765, shown as 8000 below). Production mode with a generated key file, the rate limit set to 6 per minute per key and the failed-authentication limit set to 5 per minute (the default is 20). The data is synthetic (real CVE IDs, invented assets and scores) and the KEV catalog is the 5-entry sample bundled with the repo. Keys are shown as placeholders; they were throwaway keys for this run, and a search of the server log for all three found none.
$ curl -s localhost:8000/readyz
{"status":"ready","checks":{"db":"ok","storage":"ok","kev":"ok"}}
$ curl -s localhost:8000/metrics | grep -E "^kevgate_(auth|rate|kev)" | grep -v created
kevgate_auth_failures_total{reason="missing"} 0.0
kevgate_auth_failures_total{reason="invalid"} 0.0
kevgate_auth_failures_total{reason="forbidden"} 0.0
kevgate_rate_limited_total 0.0
kevgate_auth_throttled_total 0.0
kevgate_kev_entries 5.0Output, 13 lines: curl against a local kevgate: auth, scopes, ingest
$ curl -si localhost:8000/queue # no key
HTTP/1.1 401 Unauthorized
www-authenticate: Bearer realm="kevgate"
{"detail":"missing or malformed bearer token"}
$ curl -si localhost:8000/queue -H "Authorization: Bearer $SCANNER_KEY" # findings:write only
HTTP/1.1 403 Forbidden
{"detail":"API key lacks required scope 'queue:read'"}
$ curl -si -X POST localhost:8000/findings -H "Authorization: Bearer $SCANNER_KEY" \
-H "Content-Type: application/json" -d @seed.json
HTTP/1.1 201 Created
{"accepted":6}The ranked queue, reformatted from the JSON response. The two highest-CVSS non-KEV findings (9.8) rank below a KEV finding with CVSS 8.8, which is the point of the scoring:
$ curl -s "localhost:8000/queue?limit=6" -H "Authorization: Bearer $ANALYST_KEY"
risk kev cve asset acr cvss epss
99.2 true CVE-2024-3400 fw-edge-01 8 10.0 0.96
97.1 true CVE-2021-34527 dc-01 8 8.8 0.97
96.5 true CVE-2021-44228 web-01 7 10.0 0.97
91.8 true CVE-2023-4966 vpn-02 6 9.4 0.94
37.1 false CVE-2023-38545 build-03 3 9.8 0.50
34.4 false CVE-2022-42889 app-07 4 9.8 0.20Output, 25 lines: Rate limit, failed-authentication limit and metrics
$ (8 requests) curl -s -o /dev/null -w "%{http_code}" localhost:8000/stats -H "Authorization: Bearer $BURST_KEY"
request 1 -> 200 ... request 6 -> 200
request 7 -> 429
request 8 -> 429
$ curl -si localhost:8000/stats -H "Authorization: Bearer $BURST_KEY"
HTTP/1.1 429 Too Many Requests
retry-after: 8
{"detail":"rate limit exceeded"}
$ (6 requests) curl -s -o /dev/null -w "%{http_code}" localhost:8000/queue -H "Authorization: Bearer kvg_wrong"
bad key 1 -> 401 ... bad key 5 -> 401
bad key 6 -> 429
$ curl -si localhost:8000/queue -H "Authorization: Bearer $ANALYST_KEY" # a valid key, same source
HTTP/1.1 429 Too Many Requests
retry-after: 11
{"detail":"too many failed authentication attempts"}
$ curl -s localhost:8000/metrics | grep -E "^kevgate_(auth|rate)" | grep -v created
kevgate_auth_failures_total{reason="missing"} 1.0
kevgate_auth_failures_total{reason="invalid"} 5.0
kevgate_auth_failures_total{reason="forbidden"} 1.0
kevgate_rate_limited_total 3.0
kevgate_auth_throttled_total 3.0Trimmed: JSON bodies reflowed into a table; response headers other than the status line, www-authenticate and retry-after removed; the /metrics commands are simplified filters; the eight stats requests and the six bad-key requests were shell loops, written in short form here, and the bad-key run waited a minute after the earlier requests so the failure bucket was full. The valid-key request being refused is the documented trade-off of throttling by source address: a source that has spent its failure budget is refused before any key is compared.
Table, 9 rows: Results
| Measure | Value | Source |
|---|---|---|
| Tests at the tag | 277 collected and 277 passed (pytest 9.2.0); 99.92% coverage with branches (1092 statements, 0 missed; 202 branches, 1 partly covered) | pytest --cov from the fresh clone; run 42960808156 passed 2026-11-11 (ci, on both Python 3.11 and 3.12) |
| Lint and types | ruff check and ruff format --check clean (ruff 0.17.4); mypy --strict src reports no issues in 16 source files (mypy 2.4.1) | the same clone; run 42960808156 passed 2026-11-11 |
| Mutants | 6 survive the 227 tests at v0.1.0; 6 are killed by the 277 tests at v0.1.1 | scripts/rerun_kevgate_mutants.py in this site's repository |
| CI on main | ci, security and deploy-kind green on the tag commit; terraform green on the commit before it (the workflow only runs when infra/ changes, and nothing there changed since) | run 42960808156 passed 2026-11-11, run 42960808037 passed 2026-11-11, run 42960807983 passed 2026-11-11, run 42956666367 passed 2026-11-11 |
| Kind cluster | 12 smoke checks; labelled client allowed and unlabelled pod blocked; no outbound HTTPS from the pod; privileged pod rejected; 3 of the 4 Sigma-targeted audit events found in the API server log | run 42960807983 passed 2026-11-11 (deploy-kind) |
| ZAP baseline | 67 rules pass, 0 warnings, 0 failures, 1 rule ignored (listed with a reason in deploy/zap/rules.tsv) | run 42960807983 passed 2026-11-11; PIPELINE.md |
| Terraform | 20 resources applied to LocalStack, empty second plan, 20 destroyed; 4 terraform test runs pass; checkov 0 failed (13 skipped, each justified inline) | run 42956666367 passed 2026-11-11; DEPLOYMENT.md |
| Release | v0.1.1 (973c8c1, annotated tag, published 2026-11-11). v0.1.0 was the first release. | GitHub release |
| Image | ghcr.io/samiali38183/kevgate:0.1.1 is sha256:526cae18…. Signed with cosign (keyless) by the release job, which verified the signature before finishing. SBOM attestation lists 104 components; the SBOM and provenance statements name that digest as their subject. The package can be read without credentials: an anonymous token and manifest request both succeeded on 2026-11-11 (the same manifest request without a token gets 401, as a registry does). The tag has no v prefix on GHCR. | run 42960813836 passed 2026-11-11 (release; v0.1.0: run 42922880036 passed 2026-11-10) |
What was not done: the signature was verified by the release job, not again here (no cosign was available), and nothing in the cluster enforces it.
Technical detailsHow it works
design, implementation points and diagram
The problem
A raw scanner export sorts by CVSS, which puts a theoretical 9.8 ahead of a 7.5 that attackers are using today. KEV membership is the strongest prioritization signal that is freely available, so the service makes it the dominant factor and uses the rest to order everything else.
The second question is whether a small service like this can be shipped the way a production team would, with the security controls tested rather than described.
What it does
The score is 100 * (0.40*kev + 0.25*acr/8 + 0.20*epss + 0.20*cvss/10) / 1.05, from 0 to 100. The KEV weight equals the EPSS and CVSS weights combined, so a KEV finding never scores below a non-KEV finding on an asset with the same criticality; the one degenerate tie is broken in favor of KEV.
The API takes JSON findings or a Nessus/Tenable CSV export, upserts on (CVE, asset), and serves a ranked /queue, per-CVE views and stats. Scopes are findings:write, queue:read and admin. There are health, readiness and Prometheus endpoints, and a separate CLI command refreshes the KEV file from CISA so the API process never makes outbound calls.
Describe this diagram
Request path, namespace kevgate. Pod Security restricted enforced.
- Client pod. label kevgate-client=true.
- NetworkPolicy kevgate. ingress: labelled pods, TCP 8000; egress: DNS only.
- Service kevgate. ClusterIP, port 8000.
- Pod kevgate. UID 10001, read-only root, no capabilities, no service-account token.
- Volume kevgate-data. /data: SQLite; keys from a read-only Secret.
Pipeline, .github/workflows. pull requests and pushes to main, then tags.
- ci. ruff, mypy, pytest (Python 3.11, 3.12).
- security. Semgrep, Bandit, pip-audit, gitleaks, checkov, Trivy.
- deploy-kind. kind and Calico: smoke, NetworkPolicy, Pod Security, ZAP.
- terraform. infra/ changes: validate, OIDC test, LocalStack.
- release, on a version tag. scan, push to GHCR, sign, SBOM, provenance, verify.
Three implementation points
Key checks that do not leak timing
Keys are 256-bit random values and the server stores only their SHA-256. KeyStore.authenticate hashes the presented token and compares it against every stored digest with hmac.compare_digest, never exiting early, so response time does not reveal whether a key matched or how many bytes were right. A fast hash is deliberate: slow password hashes exist to protect low-entropy passwords, and running one on every request would hand an attacker a CPU lever. Until 0.1.1 no test would have noticed if that loop were changed; the two tests that now do are in the matrix.
Limiters in the right order
A token bucket per API key returns 429 with Retry-After. A second limiter counts failed authentications per source address and is checked before any key is looked at, so a brute-force attempt is throttled instead of getting free guesses. The failure limiter uses peek and charge so that checking does not itself spend a token. Both are in memory, which is a stated limitation: they are per replica and reset on restart. Clients behind one NAT or proxy share a budget, so an attacker there can lock legitimate clients out until the bucket refills.
Controls proven by tests, and a hash-locked build
The kind job does not just render the Helm chart. deploy/scripts/netpol-test.sh starts a labeled pod that must reach the service, an unlabeled pod in the same namespace that must time out, checks that the kevgate pod cannot open an outbound HTTPS connection, and submits a privileged pod that Pod Security restricted must reject. A second script then confirms that three of the four audit events the repository's Sigma rules target are in the API server audit log. Dependencies are pinned in requirements.lock with hashes; CI and the Docker build install it with --require-hashes and pip-audit scans that exact file. A weekly lock-freshness workflow is meant to fail when a fresh compile would change the lock; it had no run on record when this page was checked.