mirror of
https://github.com/Z3Prover/z3
synced 2026-05-28 12:56:28 +00:00
b3fff5b399
This adds an agentic workflow that analyzes `compare_stats.html` over a rolling 30-hour window and publishes a GitHub Discussion summarizing bugs, crashes, and anomalies. It explicitly captures unknown-outlier patterns where a benchmark is `unknown` while peers in the same set are mostly `sat`/`unsat`/`timeout`. - **Workflow added** - Introduces `.github/workflows/compare-stats-anomaly-reporter.md` (plus compiled `.lock.yml`). - Supports `workflow_dispatch` and scheduled execution. - Uses safe discussion output with auto-close of older reports for the same stream. - **Data acquisition + robustness** - Fetches `http://mtzguido.tplinkdns.com:8081/z3/compare_stats.html` with `curl` and `wget` fallback. - Adds integrity checks (non-empty HTML/table presence) and explicit incomplete-report behavior on fetch/parse failures. - **30-hour analysis semantics** - Filters rows by timestamp candidates (`time`, `timestamp`, `date`, `run`, etc.) using UTC. - Falls back to full-table analysis when timestamps are unavailable, and marks the report accordingly. - **Classification logic** - Detects bug/crash signals from status/details (`crash`, `segfault`, `assert`, `abort`, `exception`, `error`, `failed`, `bug`). - Detects: - unknown-outlier anomalies (thresholded minority `unknown` in otherwise decisive SAT-family outcomes), - status divergences (conflicting non-timeout outcomes for same benchmark), - repeated hard-failure anomalies. - **Discussion output shape** - Produces a compact report with executive counts, bug/crash table, anomaly subsections, and raw extraction summary/limitations. ```yaml safe-outputs: create-discussion: title-prefix: "[Compare Stats] " category: "agentic workflows" close-older-discussions: true ``` --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| shared | ||
| a3-python.lock.yml | ||
| a3-python.md | ||
| academic-citation-tracker.lock.yml | ||
| academic-citation-tracker.md | ||
| agentics-maintenance.yml | ||
| android-build.yml | ||
| api-coherence-checker.lock.yml | ||
| api-coherence-checker.md | ||
| build-warning-fixer.lock.yml | ||
| build-warning-fixer.md | ||
| build-z3-cache.yml | ||
| ci.yml | ||
| code-conventions-analyzer.lock.yml | ||
| code-conventions-analyzer.md | ||
| code-simplifier.lock.yml | ||
| code-simplifier.md | ||
| compare-stats-anomaly-reporter.lock.yml | ||
| compare-stats-anomaly-reporter.md | ||
| copilot-setup-steps.yml | ||
| coverage.yml | ||
| cross-build.yml | ||
| csa-analysis.lock.yml | ||
| csa-analysis.md | ||
| docs.yml | ||
| issue-backlog-processor.lock.yml | ||
| issue-backlog-processor.md | ||
| mark-prs-ready-for-review.yml | ||
| memory-safety-report.lock.yml | ||
| memory-safety-report.md | ||
| memory-safety.yml | ||
| msvc-static-build-clang-cl.yml | ||
| msvc-static-build.yml | ||
| nightly-validation.yml | ||
| nightly.yml | ||
| nuget-build.yml | ||
| ocaml.yaml | ||
| ostrich-benchmark.lock.yml | ||
| ostrich-benchmark.md | ||
| pyodide.yml | ||
| qf-s-benchmark.lock.yml | ||
| qf-s-benchmark.md | ||
| release-notes-updater.lock.yml | ||
| release-notes-updater.md | ||
| release.yml | ||
| smtlib-benchmark-finder.lock.yml | ||
| smtlib-benchmark-finder.md | ||
| specbot-crash-analyzer.lock.yml | ||
| specbot-crash-analyzer.md | ||
| tactic-to-simplifier.lock.yml | ||
| tactic-to-simplifier.md | ||
| tptp-benchmark.lock.yml | ||
| tptp-benchmark.md | ||
| wasm-release.yml | ||
| wasm.yml | ||
| Windows.yml | ||
| wip.yml | ||
| workflow-suggestion-agent.lock.yml | ||
| workflow-suggestion-agent.md | ||
| zipt-code-reviewer.lock.yml | ||
| zipt-code-reviewer.md | ||