mirror of
https://github.com/Z3Prover/z3
synced 2026-05-28 12:56:28 +00:00
b3fff5b399
This adds an agentic workflow that analyzes `compare_stats.html` over a rolling 30-hour window and publishes a GitHub Discussion summarizing bugs, crashes, and anomalies. It explicitly captures unknown-outlier patterns where a benchmark is `unknown` while peers in the same set are mostly `sat`/`unsat`/`timeout`. - **Workflow added** - Introduces `.github/workflows/compare-stats-anomaly-reporter.md` (plus compiled `.lock.yml`). - Supports `workflow_dispatch` and scheduled execution. - Uses safe discussion output with auto-close of older reports for the same stream. - **Data acquisition + robustness** - Fetches `http://mtzguido.tplinkdns.com:8081/z3/compare_stats.html` with `curl` and `wget` fallback. - Adds integrity checks (non-empty HTML/table presence) and explicit incomplete-report behavior on fetch/parse failures. - **30-hour analysis semantics** - Filters rows by timestamp candidates (`time`, `timestamp`, `date`, `run`, etc.) using UTC. - Falls back to full-table analysis when timestamps are unavailable, and marks the report accordingly. - **Classification logic** - Detects bug/crash signals from status/details (`crash`, `segfault`, `assert`, `abort`, `exception`, `error`, `failed`, `bug`). - Detects: - unknown-outlier anomalies (thresholded minority `unknown` in otherwise decisive SAT-family outcomes), - status divergences (conflicting non-timeout outcomes for same benchmark), - repeated hard-failure anomalies. - **Discussion output shape** - Produces a compact report with executive counts, bug/crash table, anomaly subsections, and raw extraction summary/limitations. ```yaml safe-outputs: create-discussion: title-prefix: "[Compare Stats] " category: "agentic workflows" close-older-discussions: true ``` --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| actions | ||
| agents | ||
| aw | ||
| prompts | ||
| scripts | ||
| skills | ||
| workflows | ||
| CI_MIGRATION.md | ||
| CI_TESTING.md | ||
| copilot-instructions.md | ||
| dependabot.yml | ||