mirrors/z3
3
0
Fork 0
mirror of https://github.com/Z3Prover/z3 synced 2026-05-25 11:26:21 +00:00
Code Activity

Fix api-coherence workflow to verify and filter resolved issues (#8201)

Browse source 
* Initial plan

* Update api-coherence workflow to verify and filter resolved issues

Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com>
This commit is contained in:
Copilot 2026-01-15 11:40:26 -08:00 • committed by GitHub
parent 2436943794
commit 7d4964a2f0
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 87 additions and 27 deletions
Download patch file Download diff file Expand all files Collapse all files

66
.github/workflows/api-coherence-checker.lock.yml generated vendored
View file

@ -83,14 +83,14 @@ jobs:
- name: Create gh-aw temp directory - name: Create gh-aw temp directory
run: bash /opt/gh-aw/actions/create_gh_aw_tmp_dir.sh run: bash /opt/gh-aw/actions/create_gh_aw_tmp_dir.sh
- name: Checkout repository - name: Checkout repository
uses: actions/checkout@0c366fd6a839edf440554fa01a7085ccba70ac98 # v5.0.1 uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd # v5.0.1
- name: Setup .NET - name: Setup .NET
uses: actions/setup-dotnet@2016bd2012dba4e32de620c46fe006a3ac9f0602 # v5.0.1 uses: actions/setup-dotnet@67a3573c9a986a3f9c594539f4ab511d57bb3ce9 # v4.3.1
with: with:
dotnet-version: '8.0' dotnet-version: '8.0'
- name: Setup Java - name: Setup Java
uses: actions/setup-java@f2beeb24e141e01a676f977032f5a29d81c9e27e # v5.1.0 uses: actions/setup-java@c1e323688fd81a25caa38c78aa6df2d33d3e20d9 # v4.8.0
with: with:
java-version: '21' java-version: '21'
distribution: temurin distribution: temurin
@ -104,7 +104,7 @@ jobs:
with: with:
python-version: '3.12' python-version: '3.12'
- name: Setup uv - name: Setup uv
uses: astral-sh/setup-uv@61cb8a9741eeb8a550a1b8544337180c0fc8476b # v7.2.0 uses: astral-sh/setup-uv@d4b2f3b6ecc6e67c4457f6d3e41ec42d3d0fcb86 # v5.4.2
- name: Install Python language service - name: Install Python language service
run: pip install --quiet python-lsp-server run: pip install --quiet python-lsp-server
- name: Install TypeScript language service - name: Install TypeScript language service
@ -469,6 +469,13 @@ jobs:
- Current progress through the API surface - Current progress through the API surface
- Any pending suggestions or issues found - Any pending suggestions or issues found
**Important**: If you have cached pending suggestions or issues:
- **Re-verify each cached issue** before including it in the report
- Check if the missing API has been implemented since the last run
- Use Serena, grep, or glob to verify the current state of the code
- **Mark issues as resolved** if the code now includes the previously missing functionality
- **Remove resolved issues** from the cache and do NOT include them in the report
If this is your first run or memory is empty, initialize a tracking structure to systematically cover all APIs over multiple runs. If this is your first run or memory is empty, initialize a tracking structure to systematically cover all APIs over multiple runs.
### 2. Select APIs to Analyze (Focus on a Few at a Time) ### 2. Select APIs to Analyze (Focus on a Few at a Time)
@ -525,6 +532,11 @@ jobs:
- **Suggested fix**: Specific recommendation (e.g., "Add `Z3_solver_get_reason_unknown` wrapper to Python API") - **Suggested fix**: Specific recommendation (e.g., "Add `Z3_solver_get_reason_unknown` wrapper to Python API")
- **Priority**: High (core functionality), Medium (useful feature), Low (nice-to-have) - **Priority**: High (core functionality), Medium (useful feature), Low (nice-to-have)
**Critical**: Before finalizing recommendations:
- **Verify each recommendation** is still valid by checking the current codebase
- **Do not report issues that have been resolved** - verify the code hasn't been updated to fix the gap
- Only include issues that are confirmed to still exist in the current codebase
### 6. Create Discussion with Results ### 6. Create Discussion with Results
Create a GitHub Discussion with: Create a GitHub Discussion with:
@ -532,19 +544,28 @@ jobs:
- **Content Structure**: - **Content Structure**:
- Summary of APIs analyzed in this run - Summary of APIs analyzed in this run
- Statistics (e.g., "Analyzed 15 functions across 6 languages") - Statistics (e.g., "Analyzed 15 functions across 6 languages")
- Coherence findings organized by priority - **Resolution status**: Number of previously cached issues now resolved (if any)
- Coherence findings organized by priority (only unresolved issues)
- Specific recommendations for each gap found - Specific recommendations for each gap found
- Progress tracker: what % of APIs have been analyzed so far - Progress tracker: what % of APIs have been analyzed so far
- Next areas to analyze in future runs - Next areas to analyze in future runs
**Important**: Only include issues that are confirmed to be unresolved in the current codebase. Do not report resolved issues as if they are still open or not started.
### 7. Update Cache Memory ### 7. Update Cache Memory
Store in cache memory: Store in cache memory:
- APIs analyzed in this run (add to cumulative list) - APIs analyzed in this run (add to cumulative list)
- Progress percentage through total API surface - Progress percentage through total API surface
- Any high-priority issues that need follow-up - **Only unresolved issues** that need follow-up (after re-verification)
- **Remove resolved issues** from the cache
- Next APIs to analyze in the next run - Next APIs to analyze in the next run
**Critical**: Keep cache fresh by:
- Re-verifying all cached issues periodically (at least every few runs)
- Removing issues that have been resolved from the cache
- Not perpetuating stale information about resolved issues
## Guidelines ## Guidelines
- **Be systematic**: Work through APIs methodically, don't skip around randomly - **Be systematic**: Work through APIs methodically, don't skip around randomly
@ -552,6 +573,8 @@ jobs:
- **Be actionable**: Recommendations should be clear enough for a developer to implement - **Be actionable**: Recommendations should be clear enough for a developer to implement
- **Use Serena effectively**: Leverage Serena's language service integration for Java, Python, TypeScript, and C# to get accurate API information - **Use Serena effectively**: Leverage Serena's language service integration for Java, Python, TypeScript, and C# to get accurate API information
- **Cache your progress**: Always update cache memory so future runs build on previous work - **Cache your progress**: Always update cache memory so future runs build on previous work
- **Keep cache fresh**: Re-verify cached issues before reporting them to ensure they haven't been resolved
- **Don't report resolved issues**: Always check if a cached issue has been fixed before including it in the report
- **Focus on quality over quantity**: 3-5 API families analyzed thoroughly is better than 20 analyzed superficially - **Focus on quality over quantity**: 3-5 API families analyzed thoroughly is better than 20 analyzed superficially
- **Consider developer experience**: Flag not just missing features but also confusing naming or parameter differences - **Consider developer experience**: Flag not just missing features but also confusing naming or parameter differences
@ -564,8 +587,14 @@ jobs:
Analyzed: Solver APIs, BitVector operations, Context creation Analyzed: Solver APIs, BitVector operations, Context creation
Total functions checked: 18 Total functions checked: 18
Languages covered: 6 Languages covered: 6
Previously cached issues resolved: 2
Inconsistencies found: 7 Inconsistencies found: 7
## Resolution Updates
The following cached issues have been resolved since the last run:
- ✅ BitVector Rotation in Java - Implemented in commit abc123
- ✅ Solver Statistics API in C# - Fixed in PR #5678
## Progress ## Progress
- APIs analyzed so far: 45/~200 (22.5%) - APIs analyzed so far: 45/~200 (22.5%)
- This run: Solver APIs, BitVector operations, Context creation - This run: Solver APIs, BitVector operations, Context creation
@ -573,14 +602,15 @@ jobs:
## High Priority Issues ## High Priority Issues
### 1. Missing BitVector Rotation in Java ### 1. Missing BitVector Sign Extension in TypeScript
**What**: Bit rotation functions `Z3_mk_rotate_left` and `Z3_mk_rotate_right` are not exposed in Java **What**: Bit sign extension function `Z3_mk_sign_ext` is not exposed in TypeScript
**Available in**: C, C++, Python, .NET, TypeScript **Available in**: C, C++, Python, .NET, Java
**Missing in**: Java **Missing in**: TypeScript
**Fix**: Add `mkRotateLeft(int i)` and `mkRotateRight(int i)` methods to `BitVecExpr` class **Fix**: Add `signExt(int i)` method to `BitVecExpr` class
**File**: `src/api/java/BitVecExpr.java` **File**: `src/api/js/src/high-level/`
**Verified**: Checked current codebase on [Date] - still missing
### 2. Inconsistent Solver Statistics API ### 2. Inconsistent Solver Timeout API
... ...
## Medium Priority Issues ## Medium Priority Issues
@ -922,7 +952,7 @@ jobs:
echo "Agent Conclusion: $AGENT_CONCLUSION" echo "Agent Conclusion: $AGENT_CONCLUSION"
- name: Download agent output artifact - name: Download agent output artifact
continue-on-error: true continue-on-error: true
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7.0.0 uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6.0.0
with: with:
name: agent-output name: agent-output
path: /tmp/gh-aw/safeoutputs/ path: /tmp/gh-aw/safeoutputs/
@ -994,13 +1024,13 @@ jobs:
destination: /opt/gh-aw/actions destination: /opt/gh-aw/actions
- name: Download agent artifacts - name: Download agent artifacts
continue-on-error: true continue-on-error: true
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7.0.0 uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6.0.0
with: with:
name: agent-artifacts name: agent-artifacts
path: /tmp/gh-aw/threat-detection/ path: /tmp/gh-aw/threat-detection/
- name: Download agent output artifact - name: Download agent output artifact
continue-on-error: true continue-on-error: true
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7.0.0 uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6.0.0
with: with:
name: agent-output name: agent-output
path: /tmp/gh-aw/threat-detection/ path: /tmp/gh-aw/threat-detection/
@ -1154,7 +1184,7 @@ jobs:
destination: /opt/gh-aw/actions destination: /opt/gh-aw/actions
- name: Download agent output artifact - name: Download agent output artifact
continue-on-error: true continue-on-error: true
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7.0.0 uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6.0.0
with: with:
name: agent-output name: agent-output
path: /tmp/gh-aw/safeoutputs/ path: /tmp/gh-aw/safeoutputs/
@ -1190,7 +1220,7 @@ jobs:
with: with:
destination: /opt/gh-aw/actions destination: /opt/gh-aw/actions
- name: Download cache-memory artifact (default) - name: Download cache-memory artifact (default)
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7.0.0 uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6.0.0
continue-on-error: true continue-on-error: true
with: with:
name: cache-memory name: cache-memory

48
.github/workflows/api-coherence-checker.md vendored
View file

@ -52,6 +52,13 @@ Check your cache memory for:
- Current progress through the API surface - Current progress through the API surface
- Any pending suggestions or issues found - Any pending suggestions or issues found
**Important**: If you have cached pending suggestions or issues:
- **Re-verify each cached issue** before including it in the report
- Check if the missing API has been implemented since the last run
- Use Serena, grep, or glob to verify the current state of the code
- **Mark issues as resolved** if the code now includes the previously missing functionality
- **Remove resolved issues** from the cache and do NOT include them in the report
If this is your first run or memory is empty, initialize a tracking structure to systematically cover all APIs over multiple runs. If this is your first run or memory is empty, initialize a tracking structure to systematically cover all APIs over multiple runs.
### 2. Select APIs to Analyze (Focus on a Few at a Time) ### 2. Select APIs to Analyze (Focus on a Few at a Time)
@ -108,6 +115,11 @@ For each inconsistency found, provide:
- **Suggested fix**: Specific recommendation (e.g., "Add `Z3_solver_get_reason_unknown` wrapper to Python API") - **Suggested fix**: Specific recommendation (e.g., "Add `Z3_solver_get_reason_unknown` wrapper to Python API")
- **Priority**: High (core functionality), Medium (useful feature), Low (nice-to-have) - **Priority**: High (core functionality), Medium (useful feature), Low (nice-to-have)
**Critical**: Before finalizing recommendations:
- **Verify each recommendation** is still valid by checking the current codebase
- **Do not report issues that have been resolved** - verify the code hasn't been updated to fix the gap
- Only include issues that are confirmed to still exist in the current codebase
### 6. Create Discussion with Results ### 6. Create Discussion with Results
Create a GitHub Discussion with: Create a GitHub Discussion with:
@ -115,19 +127,28 @@ Create a GitHub Discussion with:
- **Content Structure**: - **Content Structure**:
- Summary of APIs analyzed in this run - Summary of APIs analyzed in this run
- Statistics (e.g., "Analyzed 15 functions across 6 languages") - Statistics (e.g., "Analyzed 15 functions across 6 languages")
- Coherence findings organized by priority - **Resolution status**: Number of previously cached issues now resolved (if any)
- Coherence findings organized by priority (only unresolved issues)
- Specific recommendations for each gap found - Specific recommendations for each gap found
- Progress tracker: what % of APIs have been analyzed so far - Progress tracker: what % of APIs have been analyzed so far
- Next areas to analyze in future runs - Next areas to analyze in future runs
**Important**: Only include issues that are confirmed to be unresolved in the current codebase. Do not report resolved issues as if they are still open or not started.
### 7. Update Cache Memory ### 7. Update Cache Memory
Store in cache memory: Store in cache memory:
- APIs analyzed in this run (add to cumulative list) - APIs analyzed in this run (add to cumulative list)
- Progress percentage through total API surface - Progress percentage through total API surface
- Any high-priority issues that need follow-up - **Only unresolved issues** that need follow-up (after re-verification)
- **Remove resolved issues** from the cache
- Next APIs to analyze in the next run - Next APIs to analyze in the next run
**Critical**: Keep cache fresh by:
- Re-verifying all cached issues periodically (at least every few runs)
- Removing issues that have been resolved from the cache
- Not perpetuating stale information about resolved issues
## Guidelines ## Guidelines
- **Be systematic**: Work through APIs methodically, don't skip around randomly - **Be systematic**: Work through APIs methodically, don't skip around randomly
@ -135,6 +156,8 @@ Store in cache memory:
- **Be actionable**: Recommendations should be clear enough for a developer to implement - **Be actionable**: Recommendations should be clear enough for a developer to implement
- **Use Serena effectively**: Leverage Serena's language service integration for Java, Python, TypeScript, and C# to get accurate API information - **Use Serena effectively**: Leverage Serena's language service integration for Java, Python, TypeScript, and C# to get accurate API information
- **Cache your progress**: Always update cache memory so future runs build on previous work - **Cache your progress**: Always update cache memory so future runs build on previous work
- **Keep cache fresh**: Re-verify cached issues before reporting them to ensure they haven't been resolved
- **Don't report resolved issues**: Always check if a cached issue has been fixed before including it in the report
- **Focus on quality over quantity**: 3-5 API families analyzed thoroughly is better than 20 analyzed superficially - **Focus on quality over quantity**: 3-5 API families analyzed thoroughly is better than 20 analyzed superficially
- **Consider developer experience**: Flag not just missing features but also confusing naming or parameter differences - **Consider developer experience**: Flag not just missing features but also confusing naming or parameter differences
@ -147,8 +170,14 @@ Store in cache memory:
Analyzed: Solver APIs, BitVector operations, Context creation Analyzed: Solver APIs, BitVector operations, Context creation
Total functions checked: 18 Total functions checked: 18
Languages covered: 6 Languages covered: 6
Previously cached issues resolved: 2
Inconsistencies found: 7 Inconsistencies found: 7
## Resolution Updates
The following cached issues have been resolved since the last run:
- ✅ BitVector Rotation in Java - Implemented in commit abc123
- ✅ Solver Statistics API in C# - Fixed in PR #5678
## Progress ## Progress
- APIs analyzed so far: 45/~200 (22.5%) - APIs analyzed so far: 45/~200 (22.5%)
- This run: Solver APIs, BitVector operations, Context creation - This run: Solver APIs, BitVector operations, Context creation
@ -156,14 +185,15 @@ Inconsistencies found: 7
## High Priority Issues ## High Priority Issues
### 1. Missing BitVector Rotation in Java ### 1. Missing BitVector Sign Extension in TypeScript
**What**: Bit rotation functions `Z3_mk_rotate_left` and `Z3_mk_rotate_right` are not exposed in Java **What**: Bit sign extension function `Z3_mk_sign_ext` is not exposed in TypeScript
**Available in**: C, C++, Python, .NET, TypeScript **Available in**: C, C++, Python, .NET, Java
**Missing in**: Java **Missing in**: TypeScript
**Fix**: Add `mkRotateLeft(int i)` and `mkRotateRight(int i)` methods to `BitVecExpr` class **Fix**: Add `signExt(int i)` method to `BitVecExpr` class
**File**: `src/api/java/BitVecExpr.java` **File**: `src/api/js/src/high-level/`
**Verified**: Checked current codebase on [Date] - still missing
### 2. Inconsistent Solver Statistics API ### 2. Inconsistent Solver Timeout API
... ...
## Medium Priority Issues ## Medium Priority Issues