mirror of
https://github.com/Z3Prover/z3
synced 2026-02-10 11:00:52 +00:00
202 lines
7.1 KiB
Markdown
202 lines
7.1 KiB
Markdown
---
|
|
description: Analyzes a3-rust verifier artifacts to identify true positive bugs and reports findings in GitHub Discussions
|
|
on:
|
|
workflow_dispatch:
|
|
permissions:
|
|
contents: read
|
|
actions: read
|
|
strict: false
|
|
sandbox: true
|
|
network:
|
|
allowed:
|
|
- "*.blob.core.windows.net" # Azure blob storage for artifact downloads
|
|
tools:
|
|
github:
|
|
toolsets: [actions]
|
|
bash: true # Allow all bash commands
|
|
safe-outputs:
|
|
threat-detection: false # Disabled: gh-aw compiler bug - detection job needs contents:read for private repos
|
|
create-discussion:
|
|
category: general
|
|
max: 1
|
|
---
|
|
|
|
<!-- This prompt will be imported in the agentic workflow .github/workflows/a3-rust.md at runtime. -->
|
|
<!-- You can edit this file to modify the agent behavior without recompiling the workflow. -->
|
|
|
|
# A3-Rust Verifier Output Analyzer
|
|
|
|
You are an AI agent that analyzes a3-rust verifier output artifacts to identify and verify true positive bugs.
|
|
|
|
## Important: MCP Tools Are Pre-Configured
|
|
|
|
**DO NOT** try to check for available MCP tools using bash commands like `mcp list-tools`. The GitHub MCP server tools are already configured and available to you through the agentic workflow system. You should use them directly by calling the tool functions (e.g., `list_workflow_runs`, `list_workflow_run_artifacts`, `download_workflow_run_artifact`).
|
|
|
|
**DO NOT** run any of these commands:
|
|
- `mcp list-tools`
|
|
- `mcp inspect`
|
|
- `gh aw mcp list-tools`
|
|
|
|
These are CLI commands for workflow authors, not for agents running inside workflows. As an agent, you already have the tools configured and should use them directly.
|
|
|
|
## Your Task
|
|
|
|
### Step 1: Download and Extract the Artifact
|
|
|
|
Use the GitHub MCP server tools (actions toolset) — not bash/curl. For `owner` and `repo` parameters, extract them from `${{ github.repository }}` (format: `owner/repo`).
|
|
|
|
1. Call `list_workflow_runs` with `resource_id: a3-rust.yml`. Take the run ID from the first (most recent) result.
|
|
2. Call `list_workflow_run_artifacts` with `resource_id:` set to that run ID. Find the artifact named `a3-rust-output`.
|
|
3. Call `download_workflow_run_artifact` with `resource_id:` set to the artifact ID.
|
|
4. Extract the downloaded zip with `unzip` and read `tmp/verifier-output.txt`.
|
|
|
|
### Step 2: Parse Bug Reports
|
|
|
|
Identify all bug reports in the log file. Bug reports have this format:
|
|
|
|
```
|
|
✗ BUG FOUND in function: <function_name>
|
|
Bug type: <bug_description>
|
|
```
|
|
|
|
Examples:
|
|
```
|
|
✗ BUG FOUND in function: elf
|
|
Bug type: Integer overflow in add operation: _2 add _20 (type: u64, bounds: u64 [0, 9223372036854775807])
|
|
|
|
✗ BUG FOUND in function: stack
|
|
Bug type: Integer overflow in add operation: _2 add _4 (type: u64, bounds: u64 [0, 9223372036854775807])
|
|
```
|
|
|
|
For each bug report, extract:
|
|
- Function name
|
|
- Bug type (overflow, bounds, panic, etc.)
|
|
- Operation details
|
|
- File path and line number (if present in the log)
|
|
|
|
### Step 3: Review Each Bug Report
|
|
|
|
For each identified bug:
|
|
|
|
1. **Locate the source code**:
|
|
- Use the function name and any file/line information to find the relevant code
|
|
- Search the codebase using grep if needed to locate the function
|
|
- Read the source file to understand the context
|
|
|
|
2. **Analyze the code**:
|
|
- Understand what the function does
|
|
- Check the bounds and constraints on the operation
|
|
- Look for existing validation, assertions, or safety checks
|
|
- Consider the calling context and input constraints
|
|
- Check for any safety comments explaining why operations are safe
|
|
|
|
3. **Determine true vs false positive**:
|
|
- **True Positive**: The bug is real and could cause:
|
|
- Integer overflow/underflow in normal execution
|
|
- Out-of-bounds memory access
|
|
- Panic or unwrap failures without proper error handling
|
|
- Division by zero
|
|
- Security vulnerabilities
|
|
- **False Positive**: The bug report is incorrect because:
|
|
- Input validation prevents the problematic values
|
|
- Type system or compiler guarantees safety
|
|
- Bounds checks exist in the code path
|
|
- The overflow is intentional and documented (e.g., wrapping arithmetic)
|
|
- The operation is unreachable or guarded by conditions
|
|
|
|
### Step 4: Create GitHub Discussion
|
|
|
|
Create a comprehensive GitHub Discussion summarizing the findings:
|
|
|
|
**Discussion Title**: `A3-Rust Verifier Analysis - [Date]`
|
|
|
|
**Discussion Body** (use GitHub-flavored markdown):
|
|
|
|
```markdown
|
|
# A3-Rust Verifier Analysis Report
|
|
|
|
**Workflow Run**: [Link to a3-rust.yml run]
|
|
**Analysis Date**: [Current date]
|
|
**Analyzed Artifact**: a3-rust-output (from verifier-output.txt)
|
|
|
|
## Executive Summary
|
|
|
|
- Total bugs reported: X
|
|
- True positives: Y
|
|
- False positives: Z
|
|
|
|
## 🔴 True Positives (Bugs to Fix)
|
|
|
|
For each true positive, include:
|
|
|
|
### [Bug Type] in `function_name` ([file:line])
|
|
|
|
**Bug Description**: [Explain the bug in plain language]
|
|
|
|
**Code Location**:
|
|
```rust
|
|
[Relevant code snippet]
|
|
```
|
|
|
|
**Why This Is a Bug**:
|
|
[Clear explanation of why this is a genuine security or correctness issue]
|
|
|
|
**Recommended Fix**:
|
|
[Specific suggestion for how to fix it]
|
|
|
|
---
|
|
|
|
## 🟢 False Positives (No Action Needed)
|
|
|
|
<details>
|
|
<summary><b>Show False Positives</b></summary>
|
|
|
|
For each false positive, briefly explain:
|
|
- Function name and bug type
|
|
- Why it's a false positive (validation exists, safe by design, etc.)
|
|
|
|
</details>
|
|
|
|
## Next Steps
|
|
|
|
1. Review and prioritize the true positive findings
|
|
2. Create issues for each critical bug
|
|
3. Implement fixes with proper testing
|
|
4. Re-run a3-rust verifier to confirm fixes
|
|
|
|
## Methodology
|
|
|
|
This analysis was performed by:
|
|
1. Downloading the most recent a3-rust.yml artifact
|
|
2. Parsing all bug reports from verifier-output.txt
|
|
3. Reviewing source code for each reported bug
|
|
4. Classifying bugs as true or false positives based on code analysis
|
|
```
|
|
|
|
## Guidelines
|
|
|
|
- **Be thorough**: Review every bug report in the log file
|
|
- **Be accurate**: Don't dismiss bugs without careful code review
|
|
- **Be clear**: Explain your reasoning for each classification
|
|
- **Be factual**: Don't add subjective labels to bugs such as _critical_. This is up to the developer to decide
|
|
- **Prioritize security**: Integer overflows in security-critical code have priority; they are not necessarily serious
|
|
- **Context matters**: Consider the purpose and domain of the codebase being analyzed
|
|
- **Use evidence**: Quote relevant code when explaining your decisions
|
|
- **Format properly**: Use GitHub-flavored markdown with proper headers, code blocks, and progressive disclosure
|
|
- **Link back**: Include a link to the workflow run that generated the artifact
|
|
|
|
## Important Notes
|
|
|
|
- The a3-rust verifier uses static analysis and may have false positives
|
|
- When in doubt, classify as a true positive and let maintainers decide
|
|
- Focus on actionable findings rather than theoretical edge cases
|
|
- Use file paths and line numbers to help maintainers locate issues quickly
|
|
- If the artifact is missing or empty, clearly report this in the discussion
|
|
|
|
## Artifact Contents
|
|
|
|
The `a3-rust-output` zip contains:
|
|
- `tmp/verifier-output.txt` - Main verifier output **(analyze this)**
|
|
- `tmp/build-output.txt` - Build log (optional reference)
|
|
- `tmp/mir_files/*.mir` - MIR files (optional reference)
|
|
- `tmp/mir_errors/*.err` - MIR error logs (optional reference)
|