mirror of
https://github.com/Z3Prover/z3
synced 2026-01-18 16:28:56 +00:00
Fix api-coherence workflow to verify and filter resolved issues (#8201)
* Initial plan * Update api-coherence workflow to verify and filter resolved issues Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com>
This commit is contained in:
Copilot
GitHub
parent
2436943794
commit
7d4964a2f0
2 changed files with 87 additions and 27 deletions
66
.github/workflows/api-coherence-checker.lock.yml
generated
vendored
66
.github/workflows/api-coherence-checker.lock.yml
generated
vendored
|
|
@ -83,14 +83,14 @@ jobs:
|
|||
- name: Create gh-aw temp directory
|
||||
run: bash /opt/gh-aw/actions/create_gh_aw_tmp_dir.sh
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@0c366fd6a839edf440554fa01a7085ccba70ac98 # v5.0.1
|
||||
uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd # v5.0.1
|
||||
|
||||
- name: Setup .NET
|
||||
uses: actions/setup-dotnet@2016bd2012dba4e32de620c46fe006a3ac9f0602 # v5.0.1
|
||||
uses: actions/setup-dotnet@67a3573c9a986a3f9c594539f4ab511d57bb3ce9 # v4.3.1
|
||||
with:
|
||||
dotnet-version: '8.0'
|
||||
- name: Setup Java
|
||||
uses: actions/setup-java@f2beeb24e141e01a676f977032f5a29d81c9e27e # v5.1.0
|
||||
uses: actions/setup-java@c1e323688fd81a25caa38c78aa6df2d33d3e20d9 # v4.8.0
|
||||
with:
|
||||
java-version: '21'
|
||||
distribution: temurin
|
||||
|
|
@ -104,7 +104,7 @@ jobs:
|
|||
with:
|
||||
python-version: '3.12'
|
||||
- name: Setup uv
|
||||
uses: astral-sh/setup-uv@61cb8a9741eeb8a550a1b8544337180c0fc8476b # v7.2.0
|
||||
uses: astral-sh/setup-uv@d4b2f3b6ecc6e67c4457f6d3e41ec42d3d0fcb86 # v5.4.2
|
||||
- name: Install Python language service
|
||||
run: pip install --quiet python-lsp-server
|
||||
- name: Install TypeScript language service
|
||||
|
|
@ -469,6 +469,13 @@ jobs:
|
|||
- Current progress through the API surface
|
||||
- Any pending suggestions or issues found
|
||||
|
||||
**Important**: If you have cached pending suggestions or issues:
|
||||
- **Re-verify each cached issue** before including it in the report
|
||||
- Check if the missing API has been implemented since the last run
|
||||
- Use Serena, grep, or glob to verify the current state of the code
|
||||
- **Mark issues as resolved** if the code now includes the previously missing functionality
|
||||
- **Remove resolved issues** from the cache and do NOT include them in the report
|
||||
|
||||
If this is your first run or memory is empty, initialize a tracking structure to systematically cover all APIs over multiple runs.
|
||||
|
||||
### 2. Select APIs to Analyze (Focus on a Few at a Time)
|
||||
|
|
@ -525,6 +532,11 @@ jobs:
|
|||
- **Suggested fix**: Specific recommendation (e.g., "Add `Z3_solver_get_reason_unknown` wrapper to Python API")
|
||||
- **Priority**: High (core functionality), Medium (useful feature), Low (nice-to-have)
|
||||
|
||||
**Critical**: Before finalizing recommendations:
|
||||
- **Verify each recommendation** is still valid by checking the current codebase
|
||||
- **Do not report issues that have been resolved** - verify the code hasn't been updated to fix the gap
|
||||
- Only include issues that are confirmed to still exist in the current codebase
|
||||
|
||||
### 6. Create Discussion with Results
|
||||
|
||||
Create a GitHub Discussion with:
|
||||
|
|
@ -532,19 +544,28 @@ jobs:
|
|||
- **Content Structure**:
|
||||
- Summary of APIs analyzed in this run
|
||||
- Statistics (e.g., "Analyzed 15 functions across 6 languages")
|
||||
- Coherence findings organized by priority
|
||||
- **Resolution status**: Number of previously cached issues now resolved (if any)
|
||||
- Coherence findings organized by priority (only unresolved issues)
|
||||
- Specific recommendations for each gap found
|
||||
- Progress tracker: what % of APIs have been analyzed so far
|
||||
- Next areas to analyze in future runs
|
||||
|
||||
**Important**: Only include issues that are confirmed to be unresolved in the current codebase. Do not report resolved issues as if they are still open or not started.
|
||||
|
||||
### 7. Update Cache Memory
|
||||
|
||||
Store in cache memory:
|
||||
- APIs analyzed in this run (add to cumulative list)
|
||||
- Progress percentage through total API surface
|
||||
- Any high-priority issues that need follow-up
|
||||
- **Only unresolved issues** that need follow-up (after re-verification)
|
||||
- **Remove resolved issues** from the cache
|
||||
- Next APIs to analyze in the next run
|
||||
|
||||
**Critical**: Keep cache fresh by:
|
||||
- Re-verifying all cached issues periodically (at least every few runs)
|
||||
- Removing issues that have been resolved from the cache
|
||||
- Not perpetuating stale information about resolved issues
|
||||
|
||||
## Guidelines
|
||||
|
||||
- **Be systematic**: Work through APIs methodically, don't skip around randomly
|
||||
|
|
@ -552,6 +573,8 @@ jobs:
|
|||
- **Be actionable**: Recommendations should be clear enough for a developer to implement
|
||||
- **Use Serena effectively**: Leverage Serena's language service integration for Java, Python, TypeScript, and C# to get accurate API information
|
||||
- **Cache your progress**: Always update cache memory so future runs build on previous work
|
||||
- **Keep cache fresh**: Re-verify cached issues before reporting them to ensure they haven't been resolved
|
||||
- **Don't report resolved issues**: Always check if a cached issue has been fixed before including it in the report
|
||||
- **Focus on quality over quantity**: 3-5 API families analyzed thoroughly is better than 20 analyzed superficially
|
||||
- **Consider developer experience**: Flag not just missing features but also confusing naming or parameter differences
|
||||
|
||||
|
|
@ -564,8 +587,14 @@ jobs:
|
|||
Analyzed: Solver APIs, BitVector operations, Context creation
|
||||
Total functions checked: 18
|
||||
Languages covered: 6
|
||||
Previously cached issues resolved: 2
|
||||
Inconsistencies found: 7
|
||||
|
||||
## Resolution Updates
|
||||
The following cached issues have been resolved since the last run:
|
||||
- ✅ BitVector Rotation in Java - Implemented in commit abc123
|
||||
- ✅ Solver Statistics API in C# - Fixed in PR #5678
|
||||
|
||||
## Progress
|
||||
- APIs analyzed so far: 45/~200 (22.5%)
|
||||
- This run: Solver APIs, BitVector operations, Context creation
|
||||
|
|
@ -573,14 +602,15 @@ jobs:
|
|||
|
||||
## High Priority Issues
|
||||
|
||||
### 1. Missing BitVector Rotation in Java
|
||||
**What**: Bit rotation functions `Z3_mk_rotate_left` and `Z3_mk_rotate_right` are not exposed in Java
|
||||
**Available in**: C, C++, Python, .NET, TypeScript
|
||||
**Missing in**: Java
|
||||
**Fix**: Add `mkRotateLeft(int i)` and `mkRotateRight(int i)` methods to `BitVecExpr` class
|
||||
**File**: `src/api/java/BitVecExpr.java`
|
||||
### 1. Missing BitVector Sign Extension in TypeScript
|
||||
**What**: Bit sign extension function `Z3_mk_sign_ext` is not exposed in TypeScript
|
||||
**Available in**: C, C++, Python, .NET, Java
|
||||
**Missing in**: TypeScript
|
||||
**Fix**: Add `signExt(int i)` method to `BitVecExpr` class
|
||||
**File**: `src/api/js/src/high-level/`
|
||||
**Verified**: Checked current codebase on [Date] - still missing
|
||||
|
||||
### 2. Inconsistent Solver Statistics API
|
||||
### 2. Inconsistent Solver Timeout API
|
||||
...
|
||||
|
||||
## Medium Priority Issues
|
||||
|
|
@ -922,7 +952,7 @@ jobs:
|
|||
echo "Agent Conclusion: $AGENT_CONCLUSION"
|
||||
- name: Download agent output artifact
|
||||
continue-on-error: true
|
||||
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7.0.0
|
||||
uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6.0.0
|
||||
with:
|
||||
name: agent-output
|
||||
path: /tmp/gh-aw/safeoutputs/
|
||||
|
|
@ -994,13 +1024,13 @@ jobs:
|
|||
destination: /opt/gh-aw/actions
|
||||
- name: Download agent artifacts
|
||||
continue-on-error: true
|
||||
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7.0.0
|
||||
uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6.0.0
|
||||
with:
|
||||
name: agent-artifacts
|
||||
path: /tmp/gh-aw/threat-detection/
|
||||
- name: Download agent output artifact
|
||||
continue-on-error: true
|
||||
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7.0.0
|
||||
uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6.0.0
|
||||
with:
|
||||
name: agent-output
|
||||
path: /tmp/gh-aw/threat-detection/
|
||||
|
|
@ -1154,7 +1184,7 @@ jobs:
|
|||
destination: /opt/gh-aw/actions
|
||||
- name: Download agent output artifact
|
||||
continue-on-error: true
|
||||
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7.0.0
|
||||
uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6.0.0
|
||||
with:
|
||||
name: agent-output
|
||||
path: /tmp/gh-aw/safeoutputs/
|
||||
|
|
@ -1190,7 +1220,7 @@ jobs:
|
|||
with:
|
||||
destination: /opt/gh-aw/actions
|
||||
- name: Download cache-memory artifact (default)
|
||||
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7.0.0
|
||||
uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6.0.0
|
||||
continue-on-error: true
|
||||
with:
|
||||
name: cache-memory
|
||||
|
|
|
|||
48
.github/workflows/api-coherence-checker.md
vendored
48
.github/workflows/api-coherence-checker.md
vendored
|
|
@ -52,6 +52,13 @@ Check your cache memory for:
|
|||
- Current progress through the API surface
|
||||
- Any pending suggestions or issues found
|
||||
|
||||
**Important**: If you have cached pending suggestions or issues:
|
||||
- **Re-verify each cached issue** before including it in the report
|
||||
- Check if the missing API has been implemented since the last run
|
||||
- Use Serena, grep, or glob to verify the current state of the code
|
||||
- **Mark issues as resolved** if the code now includes the previously missing functionality
|
||||
- **Remove resolved issues** from the cache and do NOT include them in the report
|
||||
|
||||
If this is your first run or memory is empty, initialize a tracking structure to systematically cover all APIs over multiple runs.
|
||||
|
||||
### 2. Select APIs to Analyze (Focus on a Few at a Time)
|
||||
|
|
@ -108,6 +115,11 @@ For each inconsistency found, provide:
|
|||
- **Suggested fix**: Specific recommendation (e.g., "Add `Z3_solver_get_reason_unknown` wrapper to Python API")
|
||||
- **Priority**: High (core functionality), Medium (useful feature), Low (nice-to-have)
|
||||
|
||||
**Critical**: Before finalizing recommendations:
|
||||
- **Verify each recommendation** is still valid by checking the current codebase
|
||||
- **Do not report issues that have been resolved** - verify the code hasn't been updated to fix the gap
|
||||
- Only include issues that are confirmed to still exist in the current codebase
|
||||
|
||||
### 6. Create Discussion with Results
|
||||
|
||||
Create a GitHub Discussion with:
|
||||
|
|
@ -115,19 +127,28 @@ Create a GitHub Discussion with:
|
|||
- **Content Structure**:
|
||||
- Summary of APIs analyzed in this run
|
||||
- Statistics (e.g., "Analyzed 15 functions across 6 languages")
|
||||
- Coherence findings organized by priority
|
||||
- **Resolution status**: Number of previously cached issues now resolved (if any)
|
||||
- Coherence findings organized by priority (only unresolved issues)
|
||||
- Specific recommendations for each gap found
|
||||
- Progress tracker: what % of APIs have been analyzed so far
|
||||
- Next areas to analyze in future runs
|
||||
|
||||
**Important**: Only include issues that are confirmed to be unresolved in the current codebase. Do not report resolved issues as if they are still open or not started.
|
||||
|
||||
### 7. Update Cache Memory
|
||||
|
||||
Store in cache memory:
|
||||
- APIs analyzed in this run (add to cumulative list)
|
||||
- Progress percentage through total API surface
|
||||
- Any high-priority issues that need follow-up
|
||||
- **Only unresolved issues** that need follow-up (after re-verification)
|
||||
- **Remove resolved issues** from the cache
|
||||
- Next APIs to analyze in the next run
|
||||
|
||||
**Critical**: Keep cache fresh by:
|
||||
- Re-verifying all cached issues periodically (at least every few runs)
|
||||
- Removing issues that have been resolved from the cache
|
||||
- Not perpetuating stale information about resolved issues
|
||||
|
||||
## Guidelines
|
||||
|
||||
- **Be systematic**: Work through APIs methodically, don't skip around randomly
|
||||
|
|
@ -135,6 +156,8 @@ Store in cache memory:
|
|||
- **Be actionable**: Recommendations should be clear enough for a developer to implement
|
||||
- **Use Serena effectively**: Leverage Serena's language service integration for Java, Python, TypeScript, and C# to get accurate API information
|
||||
- **Cache your progress**: Always update cache memory so future runs build on previous work
|
||||
- **Keep cache fresh**: Re-verify cached issues before reporting them to ensure they haven't been resolved
|
||||
- **Don't report resolved issues**: Always check if a cached issue has been fixed before including it in the report
|
||||
- **Focus on quality over quantity**: 3-5 API families analyzed thoroughly is better than 20 analyzed superficially
|
||||
- **Consider developer experience**: Flag not just missing features but also confusing naming or parameter differences
|
||||
|
||||
|
|
@ -147,8 +170,14 @@ Store in cache memory:
|
|||
Analyzed: Solver APIs, BitVector operations, Context creation
|
||||
Total functions checked: 18
|
||||
Languages covered: 6
|
||||
Previously cached issues resolved: 2
|
||||
Inconsistencies found: 7
|
||||
|
||||
## Resolution Updates
|
||||
The following cached issues have been resolved since the last run:
|
||||
- ✅ BitVector Rotation in Java - Implemented in commit abc123
|
||||
- ✅ Solver Statistics API in C# - Fixed in PR #5678
|
||||
|
||||
## Progress
|
||||
- APIs analyzed so far: 45/~200 (22.5%)
|
||||
- This run: Solver APIs, BitVector operations, Context creation
|
||||
|
|
@ -156,14 +185,15 @@ Inconsistencies found: 7
|
|||
|
||||
## High Priority Issues
|
||||
|
||||
### 1. Missing BitVector Rotation in Java
|
||||
**What**: Bit rotation functions `Z3_mk_rotate_left` and `Z3_mk_rotate_right` are not exposed in Java
|
||||
**Available in**: C, C++, Python, .NET, TypeScript
|
||||
**Missing in**: Java
|
||||
**Fix**: Add `mkRotateLeft(int i)` and `mkRotateRight(int i)` methods to `BitVecExpr` class
|
||||
**File**: `src/api/java/BitVecExpr.java`
|
||||
### 1. Missing BitVector Sign Extension in TypeScript
|
||||
**What**: Bit sign extension function `Z3_mk_sign_ext` is not exposed in TypeScript
|
||||
**Available in**: C, C++, Python, .NET, Java
|
||||
**Missing in**: TypeScript
|
||||
**Fix**: Add `signExt(int i)` method to `BitVecExpr` class
|
||||
**File**: `src/api/js/src/high-level/`
|
||||
**Verified**: Checked current codebase on [Date] - still missing
|
||||
|
||||
### 2. Inconsistent Solver Statistics API
|
||||
### 2. Inconsistent Solver Timeout API
|
||||
...
|
||||
|
||||
## Medium Priority Issues
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue