From 29c4e539f6f56d30b4464e9f40bf22b54d881d89 Mon Sep 17 00:00:00 2001
From: Copilot <198982749+Copilot@users.noreply.github.com>
Date: Tue, 26 May 2026 15:28:11 -0700
Subject: [PATCH] Add smtlib-benchmark-finder agentic workflow (#9629)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Monthly agentic workflow that discovers community-contributed SMT-LIB
benchmark repositories on GitHub, excluding official distributions from
smtlib.org and Zenodo, and posts a categorised summary as a GitHub
Discussion.
## Workflow steps
- **Exclusion list** — scrapes `smtlib.cs.uiowa.edu` + paginates the
[Zenodo SMT-LIB community](https://zenodo.org/communities/smt-lib/) (up
to 300 records) to extract official repo URLs; hard-excludes `SMT-LIB/*`
and `SMT-Competition/*` orgs
- **GitHub search** — 7 search strategies (topic tags
`smtlib`/`smt-lib`/`smt2`, filename/path patterns, README mentions)
scoped to repos pushed since the last run (90-day window on first run)
- **Filter + classify** — removes official sets and noise (homework
repos, empty repos); categorises survivors into: solver evaluation,
verification, security/CTF, theory/logic, tool output, education, other
- **Discussion report** — posts `[SMT-LIB Benchmarks] Community
Benchmark Repository Survey — [Month YYYY]`; older discussions
auto-closed after 90 days
- **Cache** — stores classified repo sets and run date so subsequent
runs are incremental
## Files
- `.github/workflows/smtlib-benchmark-finder.md` — workflow definition
- `.github/workflows/smtlib-benchmark-finder.lock.yml` — compiled GitHub
Actions YAML (`gh aw compile`)
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
---
.github/dependabot.yml | 12 +-
.../smtlib-benchmark-finder.lock.yml | 1442 +++++++++++++++++
.github/workflows/smtlib-benchmark-finder.md | 342 ++++
3 files changed, 1791 insertions(+), 5 deletions(-)
create mode 100644 .github/workflows/smtlib-benchmark-finder.lock.yml
create mode 100644 .github/workflows/smtlib-benchmark-finder.md
diff --git a/.github/dependabot.yml b/.github/dependabot.yml
index 5ace4600a..c00c619c4 100644
--- a/.github/dependabot.yml
+++ b/.github/dependabot.yml
@@ -1,6 +1,8 @@
-version: 2
updates:
- - package-ecosystem: "github-actions"
- directory: "/"
- schedule:
- interval: "weekly"
+- directory: /
+ ignore:
+ - dependency-name: "github/gh-aw-actions/**" # Managed by gh aw compile. Version-locked to the gh-aw compiler; do not bump.
+ package-ecosystem: github-actions
+ schedule:
+ interval: weekly
+version: 2
diff --git a/.github/workflows/smtlib-benchmark-finder.lock.yml b/.github/workflows/smtlib-benchmark-finder.lock.yml
new file mode 100644
index 000000000..30191b21c
--- /dev/null
+++ b/.github/workflows/smtlib-benchmark-finder.lock.yml
@@ -0,0 +1,1442 @@
+# gh-aw-metadata: {"schema_version":"v3","frontmatter_hash":"7fe62b76e1ec94b221dbb20fe5cd392a7a47a296f9da44c7405c343a2ca3c3e9","compiler_version":"v0.74.8","strict":true,"agent_id":"copilot"}
+# gh-aw-manifest: {"version":1,"secrets":["COPILOT_GITHUB_TOKEN","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/cache/restore","sha":"27d5ce7f107fe9357f9df03efb73ab90386fccae","version":"v5.0.5"},{"repo":"actions/cache/save","sha":"27d5ce7f107fe9357f9df03efb73ab90386fccae","version":"v5.0.5"},{"repo":"actions/checkout","sha":"de0fac2e4500dabe0009e67214ff5f5447ce83dd","version":"v6.0.2"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9.0.0"},{"repo":"actions/setup-node","sha":"48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e","version":"v6.4.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"v0.74.8","version":"v0.74.8"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.25.49"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.25.49"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.25.49"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.3.9","digest":"sha256:64828b42a4482f58fab16509d7f8f495a6d97c972a98a68aff20543531ac0388","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.3.9@sha256:64828b42a4482f58fab16509d7f8f495a6d97c972a98a68aff20543531ac0388"},{"image":"ghcr.io/github/github-mcp-server:v1.0.4"},{"image":"node:lts-alpine","digest":"sha256:d1b3b4da11eefd5941e7f0b9cf17783fc99d9c6fc34884a665f40a06dbdfc94f","pinned_image":"node:lts-alpine@sha256:d1b3b4da11eefd5941e7f0b9cf17783fc99d9c6fc34884a665f40a06dbdfc94f"}]}
+# ___ _ _
+# / _ \ | | (_)
+# | |_| | __ _ ___ _ __ | |_ _ ___
+# | _ |/ _` |/ _ \ '_ \| __| |/ __|
+# | | | | (_| | __/ | | | |_| | (__
+# \_| |_/\__, |\___|_| |_|\__|_|\___|
+# __/ |
+# _ _ |___/
+# | | | | / _| |
+# | | | | ___ _ __ _ __| |_| | _____ ____
+# | |/\| |/ _ \ '__| |/ /| _| |/ _ \ \ /\ / / ___|
+# \ /\ / (_) | | | | ( | | | | (_) \ V V /\__ \
+# \/ \/ \___/|_| |_|\_\|_| |_|\___/ \_/\_/ |___/
+#
+# This file was automatically generated by gh-aw (v0.74.8). DO NOT EDIT.
+#
+# To update this file, edit the corresponding .md file and run:
+# gh aw compile
+# Not all edits will cause changes to this file.
+#
+# For more information: https://github.github.com/gh-aw/introduction/overview/
+#
+# Monthly SMTLIB Benchmark Finder. Searches GitHub for repositories containing SMT-LIB benchmarks (.smt2 files), excludes repositories that belong to the official SMT-LIB benchmark sets (linked from smtlib.org and hosted on Zenodo), and posts a curated summary of community-contributed benchmark links as a GitHub Discussion.
+#
+# Secrets used:
+# - COPILOT_GITHUB_TOKEN
+# - GH_AW_GITHUB_MCP_SERVER_TOKEN
+# - GH_AW_GITHUB_TOKEN
+# - GITHUB_TOKEN
+#
+# Custom actions used:
+# - actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
+# - actions/cache/save@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
+# - actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+# - actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
+# - actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
+# - actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 (source v9)
+# - actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
+# - actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+# - github/gh-aw-actions/setup@v0.74.8
+#
+# Container images used:
+# - ghcr.io/github/gh-aw-firewall/agent:0.25.49
+# - ghcr.io/github/gh-aw-firewall/api-proxy:0.25.49
+# - ghcr.io/github/gh-aw-firewall/squid:0.25.49
+# - ghcr.io/github/gh-aw-mcpg:v0.3.9@sha256:64828b42a4482f58fab16509d7f8f495a6d97c972a98a68aff20543531ac0388
+# - ghcr.io/github/github-mcp-server:v1.0.4
+# - node:lts-alpine@sha256:d1b3b4da11eefd5941e7f0b9cf17783fc99d9c6fc34884a665f40a06dbdfc94f
+
+name: "SMTLIB Benchmark Finder"
+on:
+ schedule:
+ - cron: "0 8 1 * *"
+ workflow_dispatch:
+ inputs:
+ aw_context:
+ default: ""
+ description: "Agent caller context (used internally by Agentic Workflows)."
+ required: false
+ type: string
+
+permissions: {}
+
+concurrency:
+ group: "gh-aw-${{ github.workflow }}"
+
+run-name: "SMTLIB Benchmark Finder"
+
+jobs:
+ activation:
+ runs-on: ubuntu-slim
+ permissions:
+ actions: read
+ contents: read
+ outputs:
+ comment_id: ""
+ comment_repo: ""
+ engine_id: ${{ steps.generate_aw_info.outputs.engine_id }}
+ lockdown_check_failed: ${{ steps.generate_aw_info.outputs.lockdown_check_failed == 'true' }}
+ model: ${{ steps.generate_aw_info.outputs.model }}
+ secret_verification_result: ${{ steps.validate-secret.outputs.verification_result }}
+ setup-parent-span-id: ${{ steps.setup.outputs.parent-span-id || steps.setup.outputs.span-id }}
+ setup-span-id: ${{ steps.setup.outputs.span-id }}
+ setup-trace-id: ${{ steps.setup.outputs.trace-id }}
+ stale_lock_file_failed: ${{ steps.check-lock-file.outputs.stale_lock_file_failed == 'true' }}
+ steps:
+ - name: Setup Scripts
+ id: setup
+ uses: github/gh-aw-actions/setup@v0.74.8
+ with:
+ destination: ${{ runner.temp }}/gh-aw/actions
+ job-name: ${{ github.job }}
+ env:
+ GH_AW_SETUP_WORKFLOW_NAME: "SMTLIB Benchmark Finder"
+ GH_AW_CURRENT_WORKFLOW_REF: ${{ github.repository }}/.github/workflows/smtlib-benchmark-finder.lock.yml@${{ github.ref }}
+ GH_AW_INFO_VERSION: "1.0.48"
+ GH_AW_INFO_ENGINE_ID: "copilot"
+ - name: Generate agentic run info
+ id: generate_aw_info
+ env:
+ GH_AW_INFO_ENGINE_ID: "copilot"
+ GH_AW_INFO_ENGINE_NAME: "GitHub Copilot CLI"
+ GH_AW_INFO_MODEL: ${{ vars.GH_AW_MODEL_AGENT_COPILOT || 'claude-sonnet-4.5' }}
+ GH_AW_INFO_VERSION: "1.0.48"
+ GH_AW_INFO_AGENT_VERSION: "1.0.48"
+ GH_AW_INFO_CLI_VERSION: "v0.74.8"
+ GH_AW_INFO_WORKFLOW_NAME: "SMTLIB Benchmark Finder"
+ GH_AW_INFO_EXPERIMENTAL: "false"
+ GH_AW_INFO_SUPPORTS_TOOLS_ALLOWLIST: "true"
+ GH_AW_INFO_STAGED: "false"
+ GH_AW_INFO_ALLOWED_DOMAINS: '["defaults","github","smtlib.cs.uiowa.edu","zenodo.org"]'
+ GH_AW_INFO_FIREWALL_ENABLED: "true"
+ GH_AW_INFO_AWF_VERSION: "v0.25.49"
+ GH_AW_INFO_AWMG_VERSION: ""
+ GH_AW_INFO_FIREWALL_TYPE: "squid"
+ GH_AW_COMPILED_STRICT: "true"
+ uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
+ with:
+ script: |
+ const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
+ setupGlobals(core, github, context, exec, io, getOctokit);
+ const { main } = require('${{ runner.temp }}/gh-aw/actions/generate_aw_info.cjs');
+ await main(core, context);
+ - name: Validate COPILOT_GITHUB_TOKEN secret
+ id: validate-secret
+ run: bash "${RUNNER_TEMP}/gh-aw/actions/validate_multi_secret.sh" COPILOT_GITHUB_TOKEN 'GitHub Copilot CLI' https://github.github.com/gh-aw/reference/engines/#github-copilot-default
+ env:
+ COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }}
+ - name: Checkout .github and .agents folders
+ uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+ with:
+ persist-credentials: false
+ sparse-checkout: |
+ .github
+ .agents
+ .claude
+ .codex
+ .crush
+ .gemini
+ .opencode
+ .pi
+ sparse-checkout-cone-mode: true
+ fetch-depth: 1
+ - name: Save agent config folders for base branch restoration
+ env:
+ GH_AW_AGENT_FOLDERS: ".agents .claude .codex .crush .gemini .github .opencode .pi"
+ GH_AW_AGENT_FILES: ".crush.json AGENTS.md CLAUDE.md GEMINI.md PI.md opencode.jsonc"
+ # poutine:ignore untrusted_checkout_exec
+ run: bash "${RUNNER_TEMP}/gh-aw/actions/save_base_github_folders.sh"
+ - name: Check workflow lock file
+ id: check-lock-file
+ uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
+ env:
+ GH_AW_WORKFLOW_FILE: "smtlib-benchmark-finder.lock.yml"
+ GH_AW_CONTEXT_WORKFLOW_REF: "${{ github.workflow_ref }}"
+ with:
+ script: |
+ const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
+ setupGlobals(core, github, context, exec, io, getOctokit);
+ const { main } = require('${{ runner.temp }}/gh-aw/actions/check_workflow_timestamp_api.cjs');
+ await main();
+ - name: Check compile-agentic version
+ uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
+ env:
+ GH_AW_COMPILED_VERSION: "v0.74.8"
+ with:
+ script: |
+ const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
+ setupGlobals(core, github, context, exec, io, getOctokit);
+ const { main } = require('${{ runner.temp }}/gh-aw/actions/check_version_updates.cjs');
+ await main();
+ - name: Create prompt with built-in context
+ env:
+ GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt
+ GH_AW_SAFE_OUTPUTS: ${{ runner.temp }}/gh-aw/safeoutputs/outputs.jsonl
+ GH_AW_EXPR_1A3A194A: ${{ github.event.discussion.number || (fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').item_type == 'discussion' && fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').item_number) }}
+ GH_AW_EXPR_463A214A: ${{ github.event.pull_request.number || (fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').item_type == 'pull_request' && fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').item_number) }}
+ GH_AW_EXPR_802A9F6A: ${{ github.event.issue.number || (fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').item_type == 'issue' && fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').item_number) }}
+ GH_AW_EXPR_FF1D34CE: ${{ github.event.comment.id || fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').comment_id }}
+ GH_AW_GITHUB_ACTOR: ${{ github.actor }}
+ GH_AW_GITHUB_REPOSITORY: ${{ github.repository }}
+ GH_AW_GITHUB_RUN_ID: ${{ github.run_id }}
+ GH_AW_GITHUB_WORKFLOW: ${{ github.workflow }}
+ GH_AW_GITHUB_WORKSPACE: ${{ github.workspace }}
+ # poutine:ignore untrusted_checkout_exec
+ run: |
+ bash "${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh"
+ {
+ cat << 'GH_AW_PROMPT_04598545a7d250b1_EOF'
+
+ GH_AW_PROMPT_04598545a7d250b1_EOF
+ cat "${RUNNER_TEMP}/gh-aw/prompts/xpia.md"
+ cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md"
+ cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md"
+ cat "${RUNNER_TEMP}/gh-aw/prompts/cache_memory_prompt.md"
+ cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_prompt.md"
+ cat << 'GH_AW_PROMPT_04598545a7d250b1_EOF'
+
+ Tools: create_discussion, missing_tool, missing_data, noop
+
+ GH_AW_PROMPT_04598545a7d250b1_EOF
+ cat "${RUNNER_TEMP}/gh-aw/prompts/mcp_cli_tools_prompt.md"
+ cat << 'GH_AW_PROMPT_04598545a7d250b1_EOF'
+
+ The following GitHub context information is available for this workflow:
+ {{#if github.actor}}
+ - **actor**: __GH_AW_GITHUB_ACTOR__
+ {{/if}}
+ {{#if github.repository}}
+ - **repository**: __GH_AW_GITHUB_REPOSITORY__
+ {{/if}}
+ {{#if github.workspace}}
+ - **workspace**: __GH_AW_GITHUB_WORKSPACE__
+ {{/if}}
+ {{#if github.event.issue.number || (github.aw.context.item_type == 'issue' && github.aw.context.item_number)}}
+ - **issue-number**: #__GH_AW_EXPR_802A9F6A__
+ {{/if}}
+ {{#if github.event.discussion.number || (github.aw.context.item_type == 'discussion' && github.aw.context.item_number)}}
+ - **discussion-number**: #__GH_AW_EXPR_1A3A194A__
+ {{/if}}
+ {{#if github.event.pull_request.number || (github.aw.context.item_type == 'pull_request' && github.aw.context.item_number)}}
+ - **pull-request-number**: #__GH_AW_EXPR_463A214A__
+ {{/if}}
+ {{#if github.event.comment.id || github.aw.context.comment_id}}
+ - **comment-id**: __GH_AW_EXPR_FF1D34CE__
+ {{/if}}
+ {{#if github.run_id}}
+ - **workflow-run-id**: __GH_AW_GITHUB_RUN_ID__
+ {{/if}}
+
+
+ GH_AW_PROMPT_04598545a7d250b1_EOF
+ cat "${RUNNER_TEMP}/gh-aw/prompts/github_mcp_tools_with_safeoutputs_prompt.md"
+ cat << 'GH_AW_PROMPT_04598545a7d250b1_EOF'
+
+ {{#runtime-import .github/workflows/smtlib-benchmark-finder.md}}
+ GH_AW_PROMPT_04598545a7d250b1_EOF
+ } > "$GH_AW_PROMPT"
+ - name: Interpolate variables and render templates
+ uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
+ env:
+ GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt
+ GH_AW_ENGINE_ID: "copilot"
+ GH_AW_GITHUB_REPOSITORY: ${{ github.repository }}
+ GH_AW_GITHUB_WORKFLOW: ${{ github.workflow }}
+ with:
+ script: |
+ const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
+ setupGlobals(core, github, context, exec, io, getOctokit);
+ const { main } = require('${{ runner.temp }}/gh-aw/actions/interpolate_prompt.cjs');
+ await main();
+ - name: Substitute placeholders
+ uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
+ env:
+ GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt
+ GH_AW_ALLOWED_EXTENSIONS: ''
+ GH_AW_CACHE_DESCRIPTION: ''
+ GH_AW_CACHE_DIR: '/tmp/gh-aw/cache-memory/'
+ GH_AW_EXPR_1A3A194A: ${{ github.event.discussion.number || (fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').item_type == 'discussion' && fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').item_number) }}
+ GH_AW_EXPR_463A214A: ${{ github.event.pull_request.number || (fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').item_type == 'pull_request' && fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').item_number) }}
+ GH_AW_EXPR_802A9F6A: ${{ github.event.issue.number || (fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').item_type == 'issue' && fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').item_number) }}
+ GH_AW_EXPR_FF1D34CE: ${{ github.event.comment.id || fromJSON(github.event.inputs.aw_context || github.event.client_payload.aw_context || '{}').comment_id }}
+ GH_AW_GITHUB_ACTOR: ${{ github.actor }}
+ GH_AW_GITHUB_REPOSITORY: ${{ github.repository }}
+ GH_AW_GITHUB_RUN_ID: ${{ github.run_id }}
+ GH_AW_GITHUB_WORKFLOW: ${{ github.workflow }}
+ GH_AW_GITHUB_WORKSPACE: ${{ github.workspace }}
+ GH_AW_MCP_CLI_SERVERS_LIST: '- `safeoutputs` — run `safeoutputs --help` to see available tools'
+ with:
+ script: |
+ const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
+ setupGlobals(core, github, context, exec, io, getOctokit);
+
+ const substitutePlaceholders = require('${{ runner.temp }}/gh-aw/actions/substitute_placeholders.cjs');
+
+ // Call the substitution function
+ return await substitutePlaceholders({
+ file: process.env.GH_AW_PROMPT,
+ substitutions: {
+ GH_AW_ALLOWED_EXTENSIONS: process.env.GH_AW_ALLOWED_EXTENSIONS,
+ GH_AW_CACHE_DESCRIPTION: process.env.GH_AW_CACHE_DESCRIPTION,
+ GH_AW_CACHE_DIR: process.env.GH_AW_CACHE_DIR,
+ GH_AW_EXPR_1A3A194A: process.env.GH_AW_EXPR_1A3A194A,
+ GH_AW_EXPR_463A214A: process.env.GH_AW_EXPR_463A214A,
+ GH_AW_EXPR_802A9F6A: process.env.GH_AW_EXPR_802A9F6A,
+ GH_AW_EXPR_FF1D34CE: process.env.GH_AW_EXPR_FF1D34CE,
+ GH_AW_GITHUB_ACTOR: process.env.GH_AW_GITHUB_ACTOR,
+ GH_AW_GITHUB_REPOSITORY: process.env.GH_AW_GITHUB_REPOSITORY,
+ GH_AW_GITHUB_RUN_ID: process.env.GH_AW_GITHUB_RUN_ID,
+ GH_AW_GITHUB_WORKFLOW: process.env.GH_AW_GITHUB_WORKFLOW,
+ GH_AW_GITHUB_WORKSPACE: process.env.GH_AW_GITHUB_WORKSPACE,
+ GH_AW_MCP_CLI_SERVERS_LIST: process.env.GH_AW_MCP_CLI_SERVERS_LIST
+ }
+ });
+ - name: Validate prompt placeholders
+ env:
+ GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt
+ # poutine:ignore untrusted_checkout_exec
+ run: bash "${RUNNER_TEMP}/gh-aw/actions/validate_prompt_placeholders.sh"
+ - name: Print prompt
+ env:
+ GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt
+ # poutine:ignore untrusted_checkout_exec
+ run: bash "${RUNNER_TEMP}/gh-aw/actions/print_prompt_summary.sh"
+ - name: Upload activation artifact
+ if: success()
+ uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+ with:
+ name: activation
+ include-hidden-files: true
+ path: |
+ /tmp/gh-aw/aw_info.json
+ /tmp/gh-aw/aw-prompts/prompt.txt
+ /tmp/gh-aw/aw-prompts/prompt-template.txt
+ /tmp/gh-aw/aw-prompts/prompt-import-tree.json
+ /tmp/gh-aw/github_rate_limits.jsonl
+ /tmp/gh-aw/base
+ /tmp/gh-aw/.github/agents
+ if-no-files-found: ignore
+ retention-days: 1
+
+ agent:
+ needs: activation
+ runs-on: ubuntu-latest
+ permissions: read-all
+ concurrency:
+ group: "gh-aw-copilot-${{ github.workflow }}"
+ env:
+ DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
+ GH_AW_ASSETS_ALLOWED_EXTS: ""
+ GH_AW_ASSETS_BRANCH: ""
+ GH_AW_ASSETS_MAX_SIZE_KB: 0
+ GH_AW_MCP_LOG_DIR: /tmp/gh-aw/mcp-logs/safeoutputs
+ GH_AW_WORKFLOW_ID_SANITIZED: smtlibbenchmarkfinder
+ outputs:
+ agentic_engine_timeout: ${{ steps.detect-copilot-errors.outputs.agentic_engine_timeout || 'false' }}
+ checkout_pr_success: ${{ steps.checkout-pr.outputs.checkout_pr_success || 'true' }}
+ effective_tokens: ${{ steps.parse-mcp-gateway.outputs.effective_tokens }}
+ effective_tokens_rate_limit_error: ${{ steps.parse-mcp-gateway.outputs.effective_tokens_rate_limit_error || 'false' }}
+ has_patch: ${{ steps.collect_output.outputs.has_patch }}
+ inference_access_error: ${{ steps.detect-copilot-errors.outputs.inference_access_error || 'false' }}
+ mcp_policy_error: ${{ steps.detect-copilot-errors.outputs.mcp_policy_error || 'false' }}
+ model: ${{ needs.activation.outputs.model }}
+ model_not_supported_error: ${{ steps.detect-copilot-errors.outputs.model_not_supported_error || 'false' }}
+ output: ${{ steps.collect_output.outputs.output }}
+ output_types: ${{ steps.collect_output.outputs.output_types }}
+ setup-parent-span-id: ${{ steps.setup.outputs.parent-span-id || steps.setup.outputs.span-id }}
+ setup-span-id: ${{ steps.setup.outputs.span-id }}
+ setup-trace-id: ${{ steps.setup.outputs.trace-id }}
+ steps:
+ - name: Setup Scripts
+ id: setup
+ uses: github/gh-aw-actions/setup@v0.74.8
+ with:
+ destination: ${{ runner.temp }}/gh-aw/actions
+ job-name: ${{ github.job }}
+ trace-id: ${{ needs.activation.outputs.setup-trace-id }}
+ parent-span-id: ${{ needs.activation.outputs.setup-parent-span-id || needs.activation.outputs.setup-span-id }}
+ env:
+ GH_AW_SETUP_WORKFLOW_NAME: "SMTLIB Benchmark Finder"
+ GH_AW_CURRENT_WORKFLOW_REF: ${{ github.repository }}/.github/workflows/smtlib-benchmark-finder.lock.yml@${{ github.ref }}
+ GH_AW_INFO_VERSION: "1.0.48"
+ GH_AW_INFO_ENGINE_ID: "copilot"
+ - name: Set runtime paths
+ id: set-runtime-paths
+ run: |
+ {
+ echo "GH_AW_SAFE_OUTPUTS=${RUNNER_TEMP}/gh-aw/safeoutputs/outputs.jsonl"
+ echo "GH_AW_SAFE_OUTPUTS_CONFIG_PATH=${RUNNER_TEMP}/gh-aw/safeoutputs/config.json"
+ echo "GH_AW_SAFE_OUTPUTS_TOOLS_PATH=${RUNNER_TEMP}/gh-aw/safeoutputs/tools.json"
+ } >> "$GITHUB_OUTPUT"
+ - name: Checkout repository
+ uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+ with:
+ persist-credentials: false
+ - name: Create gh-aw temp directory
+ run: bash "${RUNNER_TEMP}/gh-aw/actions/create_gh_aw_tmp_dir.sh"
+ - name: Configure gh CLI for GitHub Enterprise
+ run: bash "${RUNNER_TEMP}/gh-aw/actions/configure_gh_for_ghe.sh"
+ env:
+ GH_TOKEN: ${{ github.token }}
+ # Cache memory file share configuration from frontmatter processed below
+ - name: Create cache-memory directory
+ run: bash "${RUNNER_TEMP}/gh-aw/actions/create_cache_memory_dir.sh"
+ - name: Restore cache-memory file share data
+ uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
+ with:
+ key: memory-none-nopolicy-${{ env.GH_AW_WORKFLOW_ID_SANITIZED }}-${{ github.run_id }}
+ path: /tmp/gh-aw/cache-memory
+ restore-keys: |
+ memory-none-nopolicy-${{ env.GH_AW_WORKFLOW_ID_SANITIZED }}-
+ - name: Setup cache-memory git repository
+ env:
+ GH_AW_CACHE_DIR: /tmp/gh-aw/cache-memory
+ GH_AW_MIN_INTEGRITY: none
+ run: bash "${RUNNER_TEMP}/gh-aw/actions/setup_cache_memory_git.sh"
+ - name: Configure Git credentials
+ env:
+ REPO_NAME: ${{ github.repository }}
+ SERVER_URL: ${{ github.server_url }}
+ GITHUB_TOKEN: ${{ github.token }}
+ run: |
+ git config --global user.email "github-actions[bot]@users.noreply.github.com"
+ git config --global user.name "github-actions[bot]"
+ git config --global am.keepcr true
+ # Re-authenticate git with GitHub token
+ SERVER_URL_STRIPPED="${SERVER_URL#https://}"
+ git remote set-url origin "https://x-access-token:${GITHUB_TOKEN}@${SERVER_URL_STRIPPED}/${REPO_NAME}.git"
+ echo "Git configured with standard GitHub Actions identity"
+ - name: Checkout PR branch
+ id: checkout-pr
+ if: |
+ github.event.pull_request || github.event.issue.pull_request
+ uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
+ env:
+ GH_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }}
+ with:
+ github-token: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }}
+ script: |
+ const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
+ setupGlobals(core, github, context, exec, io, getOctokit);
+ const { main } = require('${{ runner.temp }}/gh-aw/actions/checkout_pr_branch.cjs');
+ await main();
+ - name: Install GitHub Copilot CLI
+ run: bash "${RUNNER_TEMP}/gh-aw/actions/install_copilot_cli.sh" 1.0.48
+ env:
+ GH_HOST: github.com
+ - name: Install AWF binary
+ run: bash "${RUNNER_TEMP}/gh-aw/actions/install_awf_binary.sh" v0.25.49
+ - name: Determine automatic lockdown mode for GitHub MCP Server
+ id: determine-automatic-lockdown
+ uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 (source v9)
+ env:
+ GH_AW_GITHUB_TOKEN: ${{ secrets.GH_AW_GITHUB_TOKEN }}
+ GH_AW_GITHUB_MCP_SERVER_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN }}
+ with:
+ script: |
+ const determineAutomaticLockdown = require('${{ runner.temp }}/gh-aw/actions/determine_automatic_lockdown.cjs');
+ await determineAutomaticLockdown(github, context, core);
+ - name: Download activation artifact
+ uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
+ with:
+ name: activation
+ path: /tmp/gh-aw
+ - name: Restore agent config folders from base branch
+ if: steps.checkout-pr.outcome == 'success'
+ env:
+ GH_AW_AGENT_FOLDERS: ".agents .claude .codex .crush .gemini .github .opencode .pi"
+ GH_AW_AGENT_FILES: ".crush.json AGENTS.md CLAUDE.md GEMINI.md PI.md opencode.jsonc"
+ run: bash "${RUNNER_TEMP}/gh-aw/actions/restore_base_github_folders.sh"
+ - name: Restore inline sub-agents from activation artifact
+ env:
+ GH_AW_SUB_AGENT_DIR: ".github/agents"
+ GH_AW_SUB_AGENT_EXT: ".agent.md"
+ run: bash "${RUNNER_TEMP}/gh-aw/actions/restore_inline_sub_agents.sh"
+ - name: Download container images
+ run: bash "${RUNNER_TEMP}/gh-aw/actions/download_docker_images.sh" ghcr.io/github/gh-aw-firewall/agent:0.25.49 ghcr.io/github/gh-aw-firewall/api-proxy:0.25.49 ghcr.io/github/gh-aw-firewall/squid:0.25.49 ghcr.io/github/gh-aw-mcpg:v0.3.9@sha256:64828b42a4482f58fab16509d7f8f495a6d97c972a98a68aff20543531ac0388 ghcr.io/github/github-mcp-server:v1.0.4 node:lts-alpine@sha256:d1b3b4da11eefd5941e7f0b9cf17783fc99d9c6fc34884a665f40a06dbdfc94f
+ - name: Generate Safe Outputs Config
+ run: |
+ mkdir -p "${RUNNER_TEMP}/gh-aw/safeoutputs"
+ mkdir -p /tmp/gh-aw/safeoutputs
+ mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs
+ cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_1c2429a4603946af_EOF'
+ {"create_discussion":{"category":"agentic workflows","close_older_discussions":true,"expires":2160,"fallback_to_issue":true,"max":1,"title_prefix":"[SMT-LIB Benchmarks] "},"create_report_incomplete_issue":{},"max_bot_mentions":1,"mentions":{"enabled":false},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"false"},"report_incomplete":{}}
+ GH_AW_SAFE_OUTPUTS_CONFIG_1c2429a4603946af_EOF
+ - name: Generate Safe Outputs Tools
+ env:
+ GH_AW_TOOLS_META_JSON: |
+ {
+ "description_suffixes": {
+ "create_discussion": " CONSTRAINTS: Maximum 1 discussion(s) can be created. Title will be prefixed with \"[SMT-LIB Benchmarks] \". Discussions will be created in category \"agentic workflows\"."
+ },
+ "repo_params": {},
+ "dynamic_tools": []
+ }
+ GH_AW_VALIDATION_JSON: |
+ {
+ "create_discussion": {
+ "defaultMax": 1,
+ "fields": {
+ "body": {
+ "required": true,
+ "type": "string",
+ "sanitize": true,
+ "maxLength": 65000
+ },
+ "category": {
+ "type": "string",
+ "sanitize": true,
+ "maxLength": 128
+ },
+ "repo": {
+ "type": "string",
+ "maxLength": 256
+ },
+ "title": {
+ "required": true,
+ "type": "string",
+ "sanitize": true,
+ "maxLength": 128
+ }
+ }
+ },
+ "missing_data": {
+ "defaultMax": 20,
+ "fields": {
+ "alternatives": {
+ "type": "string",
+ "sanitize": true,
+ "maxLength": 256
+ },
+ "context": {
+ "type": "string",
+ "sanitize": true,
+ "maxLength": 256
+ },
+ "data_type": {
+ "type": "string",
+ "sanitize": true,
+ "maxLength": 128
+ },
+ "reason": {
+ "type": "string",
+ "sanitize": true,
+ "maxLength": 256
+ }
+ }
+ },
+ "missing_tool": {
+ "defaultMax": 20,
+ "fields": {
+ "alternatives": {
+ "type": "string",
+ "sanitize": true,
+ "maxLength": 512
+ },
+ "reason": {
+ "required": true,
+ "type": "string",
+ "sanitize": true,
+ "maxLength": 256
+ },
+ "tool": {
+ "type": "string",
+ "sanitize": true,
+ "maxLength": 128
+ }
+ }
+ },
+ "noop": {
+ "defaultMax": 1,
+ "fields": {
+ "message": {
+ "required": true,
+ "type": "string",
+ "sanitize": true,
+ "maxLength": 65000
+ }
+ }
+ },
+ "report_incomplete": {
+ "defaultMax": 5,
+ "fields": {
+ "details": {
+ "type": "string",
+ "sanitize": true,
+ "maxLength": 65000
+ },
+ "reason": {
+ "required": true,
+ "type": "string",
+ "sanitize": true,
+ "maxLength": 1024
+ }
+ }
+ }
+ }
+ uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
+ with:
+ script: |
+ const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
+ setupGlobals(core, github, context, exec, io, getOctokit);
+ const { main } = require('${{ runner.temp }}/gh-aw/actions/generate_safe_outputs_tools.cjs');
+ await main();
+ - name: Generate Safe Outputs MCP Server Config
+ id: safe-outputs-config
+ run: |
+ # Generate a secure random API key (360 bits of entropy, 40+ chars)
+ # Mask immediately to prevent timing vulnerabilities
+ API_KEY=$(openssl rand -base64 45 | tr -d '/+=')
+ echo "::add-mask::${API_KEY}"
+
+ PORT=3001
+
+ # Set outputs for next steps
+ {
+ echo "safe_outputs_api_key=${API_KEY}"
+ echo "safe_outputs_port=${PORT}"
+ } >> "$GITHUB_OUTPUT"
+
+ echo "Safe Outputs MCP server will run on port ${PORT}"
+
+ - name: Start Safe Outputs MCP HTTP Server
+ id: safe-outputs-start
+ env:
+ DEBUG: '*'
+ GH_AW_SAFE_OUTPUTS: ${{ steps.set-runtime-paths.outputs.GH_AW_SAFE_OUTPUTS }}
+ GH_AW_SAFE_OUTPUTS_PORT: ${{ steps.safe-outputs-config.outputs.safe_outputs_port }}
+ GH_AW_SAFE_OUTPUTS_API_KEY: ${{ steps.safe-outputs-config.outputs.safe_outputs_api_key }}
+ GH_AW_SAFE_OUTPUTS_TOOLS_PATH: ${{ runner.temp }}/gh-aw/safeoutputs/tools.json
+ GH_AW_SAFE_OUTPUTS_CONFIG_PATH: ${{ runner.temp }}/gh-aw/safeoutputs/config.json
+ GH_AW_MCP_LOG_DIR: /tmp/gh-aw/mcp-logs/safeoutputs
+ run: |
+ # Environment variables are set above to prevent template injection
+ export DEBUG
+ export GH_AW_SAFE_OUTPUTS
+ export GH_AW_SAFE_OUTPUTS_PORT
+ export GH_AW_SAFE_OUTPUTS_API_KEY
+ export GH_AW_SAFE_OUTPUTS_TOOLS_PATH
+ export GH_AW_SAFE_OUTPUTS_CONFIG_PATH
+ export GH_AW_MCP_LOG_DIR
+
+ bash "${RUNNER_TEMP}/gh-aw/actions/start_safe_outputs_server.sh"
+
+ - name: Start MCP Gateway
+ id: start-mcp-gateway
+ env:
+ GH_AW_SAFE_OUTPUTS: ${{ steps.set-runtime-paths.outputs.GH_AW_SAFE_OUTPUTS }}
+ GH_AW_SAFE_OUTPUTS_API_KEY: ${{ steps.safe-outputs-start.outputs.api_key }}
+ GH_AW_SAFE_OUTPUTS_PORT: ${{ steps.safe-outputs-start.outputs.port }}
+ GITHUB_MCP_GUARD_MIN_INTEGRITY: ${{ steps.determine-automatic-lockdown.outputs.min_integrity }}
+ GITHUB_MCP_GUARD_REPOS: ${{ steps.determine-automatic-lockdown.outputs.repos }}
+ GITHUB_MCP_SERVER_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }}
+ run: |
+ set -eo pipefail
+ mkdir -p "${RUNNER_TEMP}/gh-aw/mcp-config"
+
+ # Export gateway environment variables for MCP config and gateway script
+ export MCP_GATEWAY_PORT="8080"
+ export MCP_GATEWAY_DOMAIN="host.docker.internal"
+ export MCP_GATEWAY_HOST_DOMAIN="localhost"
+ MCP_GATEWAY_API_KEY=$(openssl rand -base64 45 | tr -d '/+=')
+ echo "::add-mask::${MCP_GATEWAY_API_KEY}"
+ export MCP_GATEWAY_API_KEY
+ export MCP_GATEWAY_PAYLOAD_DIR="/tmp/gh-aw/mcp-payloads"
+ mkdir -p "${MCP_GATEWAY_PAYLOAD_DIR}"
+ export MCP_GATEWAY_PAYLOAD_SIZE_THRESHOLD="524288"
+ export DEBUG="*"
+
+ export GH_AW_ENGINE="copilot"
+ MCP_GATEWAY_UID=$(id -u 2>/dev/null || echo '0')
+ MCP_GATEWAY_GID=$(id -g 2>/dev/null || echo '0')
+ case "${DOCKER_HOST:-}" in
+ unix://* ) DOCKER_SOCK_PATH="${DOCKER_HOST#unix://}" ;;
+ /* ) DOCKER_SOCK_PATH="$DOCKER_HOST" ;;
+ * ) DOCKER_SOCK_PATH=/var/run/docker.sock ;;
+ esac
+ DOCKER_SOCK_GID=$(stat -c '%g' "$DOCKER_SOCK_PATH" 2>/dev/null || echo '0')
+ export MCP_GATEWAY_DOCKER_COMMAND='docker run -i --rm --network host --add-host host.docker.internal:127.0.0.1 --user '"${MCP_GATEWAY_UID}"':'"${MCP_GATEWAY_GID}"' --group-add '"${DOCKER_SOCK_GID}"' -v '"${DOCKER_SOCK_PATH}"':/var/run/docker.sock -e MCP_GATEWAY_PORT -e MCP_GATEWAY_DOMAIN -e MCP_GATEWAY_API_KEY -e MCP_GATEWAY_PAYLOAD_DIR -e MCP_GATEWAY_PAYLOAD_SIZE_THRESHOLD -e DOCKER_HOST=unix:///var/run/docker.sock -e DEBUG -e MCP_GATEWAY_LOG_DIR -e GH_AW_MCP_LOG_DIR -e GH_AW_SAFE_OUTPUTS -e GH_AW_SAFE_OUTPUTS_CONFIG_PATH -e GH_AW_SAFE_OUTPUTS_TOOLS_PATH -e GH_AW_ASSETS_BRANCH -e GH_AW_ASSETS_MAX_SIZE_KB -e GH_AW_ASSETS_ALLOWED_EXTS -e DEFAULT_BRANCH -e GITHUB_MCP_SERVER_TOKEN -e GITHUB_MCP_GUARD_MIN_INTEGRITY -e GITHUB_MCP_GUARD_REPOS -e GITHUB_REPOSITORY -e GITHUB_SERVER_URL -e GITHUB_SHA -e GITHUB_WORKSPACE -e GITHUB_TOKEN -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e GITHUB_JOB -e GITHUB_ACTION -e GITHUB_EVENT_NAME -e GITHUB_EVENT_PATH -e GITHUB_ACTOR -e GITHUB_ACTOR_ID -e GITHUB_TRIGGERING_ACTOR -e GITHUB_WORKFLOW -e GITHUB_WORKFLOW_REF -e GITHUB_WORKFLOW_SHA -e GITHUB_REF -e GITHUB_REF_NAME -e GITHUB_REF_TYPE -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GH_AW_SAFE_OUTPUTS_PORT -e GH_AW_SAFE_OUTPUTS_API_KEY -v /tmp/gh-aw/mcp-payloads:/tmp/gh-aw/mcp-payloads:rw -v /opt:/opt:ro -v /tmp:/tmp:rw -v '"${GITHUB_WORKSPACE}"':'"${GITHUB_WORKSPACE}"':rw ghcr.io/github/gh-aw-mcpg:v0.3.9'
+
+ mkdir -p /home/runner/.copilot
+ GH_AW_NODE=$(which node 2>/dev/null || command -v node 2>/dev/null || echo node)
+ cat << GH_AW_MCP_CONFIG_07d3b8ec96f0806f_EOF | "$GH_AW_NODE" "${RUNNER_TEMP}/gh-aw/actions/start_mcp_gateway.cjs"
+ {
+ "mcpServers": {
+ "github": {
+ "type": "stdio",
+ "container": "ghcr.io/github/github-mcp-server:v1.0.4",
+ "env": {
+ "GITHUB_HOST": "\${GITHUB_SERVER_URL}",
+ "GITHUB_PERSONAL_ACCESS_TOKEN": "\${GITHUB_MCP_SERVER_TOKEN}",
+ "GITHUB_READ_ONLY": "1",
+ "GITHUB_TOOLSETS": "context,repos,issues,pull_requests"
+ },
+ "guard-policies": {
+ "allow-only": {
+ "min-integrity": "$GITHUB_MCP_GUARD_MIN_INTEGRITY",
+ "repos": "$GITHUB_MCP_GUARD_REPOS"
+ }
+ }
+ },
+ "safeoutputs": {
+ "type": "http",
+ "url": "http://host.docker.internal:$GH_AW_SAFE_OUTPUTS_PORT",
+ "headers": {
+ "Authorization": "\${GH_AW_SAFE_OUTPUTS_API_KEY}"
+ },
+ "guard-policies": {
+ "write-sink": {
+ "accept": [
+ "*"
+ ]
+ }
+ }
+ }
+ },
+ "gateway": {
+ "port": $MCP_GATEWAY_PORT,
+ "domain": "${MCP_GATEWAY_DOMAIN}",
+ "apiKey": "${MCP_GATEWAY_API_KEY}",
+ "payloadDir": "${MCP_GATEWAY_PAYLOAD_DIR}"
+ }
+ }
+ GH_AW_MCP_CONFIG_07d3b8ec96f0806f_EOF
+ - name: Mount MCP servers as CLIs
+ id: mount-mcp-clis
+ continue-on-error: true
+ env:
+ MCP_GATEWAY_API_KEY: ${{ steps.start-mcp-gateway.outputs.gateway-api-key }}
+ MCP_GATEWAY_DOMAIN: ${{ steps.start-mcp-gateway.outputs.gateway-domain }}
+ MCP_GATEWAY_PORT: ${{ steps.start-mcp-gateway.outputs.gateway-port }}
+ uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
+ with:
+ script: |
+ const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
+ setupGlobals(core, github, context, exec, io);
+ const { main } = require('${{ runner.temp }}/gh-aw/actions/mount_mcp_as_cli.cjs');
+ await main();
+ - name: Clean credentials
+ continue-on-error: true
+ run: bash "${RUNNER_TEMP}/gh-aw/actions/clean_git_credentials.sh"
+ - name: Audit pre-agent workspace
+ id: pre_agent_audit
+ continue-on-error: true
+ run: bash "${RUNNER_TEMP}/gh-aw/actions/audit_pre_agent_workspace.sh"
+ - name: Execute GitHub Copilot CLI
+ id: agentic_execution
+ # Copilot CLI tool arguments (sorted):
+ timeout-minutes: 60
+ run: |
+ set -o pipefail
+ printf '%s' "$(date +%s%3N)" > /tmp/gh-aw/agent_cli_start_ms.txt
+ touch /tmp/gh-aw/agent-step-summary.md
+ GH_AW_NODE_BIN=$(command -v node 2>/dev/null || true)
+ export GH_AW_NODE_BIN
+ export COPILOT_API_KEY="$COPILOT_DUMMY_BYOK"
+ (umask 177 && touch /tmp/gh-aw/agent-stdio.log)
+ printf '%s\n' '{"$schema":"https://github.com/github/gh-aw-firewall/releases/download/v0.25.49/awf-config.schema.json","network":{"allowDomains":["*.githubusercontent.com","api.business.githubcopilot.com","api.enterprise.githubcopilot.com","api.github.com","api.githubcopilot.com","api.individual.githubcopilot.com","api.snapcraft.io","archive.ubuntu.com","azure.archive.ubuntu.com","codeload.github.com","crl.geotrust.com","crl.globalsign.com","crl.identrust.com","crl.sectigo.com","crl.thawte.com","crl.usertrust.com","crl.verisign.com","crl3.digicert.com","crl4.digicert.com","crls.ssl.com","docs.github.com","github-cloud.githubusercontent.com","github-cloud.s3.amazonaws.com","github.blog","github.com","github.githubassets.com","host.docker.internal","json-schema.org","json.schemastore.org","keyserver.ubuntu.com","lfs.github.com","objects.githubusercontent.com","ocsp.digicert.com","ocsp.geotrust.com","ocsp.globalsign.com","ocsp.identrust.com","ocsp.sectigo.com","ocsp.ssl.com","ocsp.thawte.com","ocsp.usertrust.com","ocsp.verisign.com","packagecloud.io","packages.cloud.google.com","packages.microsoft.com","patch-diff.githubusercontent.com","ppa.launchpad.net","raw.githubusercontent.com","registry.npmjs.org","s.symcb.com","s.symcd.com","security.ubuntu.com","smtlib.cs.uiowa.edu","telemetry.enterprise.githubcopilot.com","ts-crl.ws.symantec.com","ts-ocsp.ws.symantec.com","www.googleapis.com","zenodo.org"]},"apiProxy":{"enabled":true,"enableTokenSteering":true,"maxRuns":500,"maxEffectiveTokens":25000000,"models":{"agent":["sonnet-6x","gpt-5.4","gpt-5","gemini-pro","haiku","any"],"any":["copilot/*","anthropic/*","openai/*","google/*","gemini/*"],"auto":["large"],"claude":["agent","sonnet-6x","haiku","any"],"codex":["agent","gpt-5-codex","gpt-5","any"],"coding":["copilot/gpt-5*codex*","openai/gpt-5*codex*","gpt-5-codex"],"copilot":["agent","gpt-5.4","sonnet","gpt-5","any"],"deep-research":["copilot/deep-research*","copilot/o3-deep-research*","copilot/o4-mini-deep-research*","google/deep-research*","gemini/deep-research*","openai/o3-deep-research*","openai/o4-mini-deep-research*"],"gemini":["agent","gemini-pro","gemini-flash","any"],"gemini-flash":["copilot/gemini-*flash*","google/gemini-*flash*","gemini/gemini-*flash*"],"gemini-flash-lite":["copilot/gemini-*flash*lite*","google/gemini-*flash*lite*","gemini/gemini-*flash*lite*"],"gemini-pro":["copilot/gemini-*pro*","google/gemini-*pro*","gemini/gemini-*pro*"],"gemma":["copilot/gemma*","google/gemma*","gemini/gemma*"],"gpt-4.1":["copilot/gpt-4.1*","openai/gpt-4.1*"],"gpt-5":["copilot/gpt-5*","openai/gpt-5*"],"gpt-5-codex":["copilot/gpt-5*codex*","openai/gpt-5*codex*"],"gpt-5-mini":["copilot/gpt-5*mini*","openai/gpt-5*mini*"],"gpt-5-nano":["copilot/gpt-5*nano*","openai/gpt-5*nano*"],"gpt-5-pro":["copilot/gpt-5*pro*","openai/gpt-5*pro*"],"haiku":["copilot/*haiku*","anthropic/*haiku*"],"large":["sonnet","gpt-5-pro","gpt-5","gemini-pro"],"mini":["haiku","gpt-5-mini","gpt-5-nano","gemini-flash-lite","copilot/raptor*mini*"],"opus":["copilot/*opus*","anthropic/*opus*"],"reasoning":["copilot/o1*","copilot/o3*","copilot/o4*","openai/o1*","openai/o3*","openai/o4*"],"small":["mini"],"sonnet":["copilot/*sonnet*","anthropic/*sonnet*"],"sonnet-6x":["copilot/*sonnet-4.5*","copilot/*sonnet-4-5*","anthropic/*sonnet-4.5*","anthropic/*sonnet-4-5*","copilot/*sonnet-3.7*","copilot/*sonnet-3-7*","anthropic/*sonnet-3.7*","anthropic/*sonnet-3-7*","copilot/*sonnet-3.5*","copilot/*sonnet-3-5*","anthropic/*sonnet-3.5*","anthropic/*sonnet-3-5*"],"vision":["copilot/gemini-*image*","gemini/gemini-*image*","copilot/gemini-*flash*","gemini/gemini-*flash*"]}},"container":{"imageTag":"0.25.49"}}' > "${RUNNER_TEMP}/gh-aw/awf-config.json"
+ cp "${RUNNER_TEMP}/gh-aw/awf-config.json" /tmp/gh-aw/awf-config.json
+ GH_AW_DOCKER_HOST_PATH_PREFIX_ARGS=""
+ if [[ "${DOCKER_HOST:-}" =~ ^tcp:// ]]; then
+ GH_AW_DOCKER_HOST_PATH_PREFIX_ARGS="--docker-host-path-prefix /tmp/gh-aw"
+ fi
+ # shellcheck disable=SC1003
+ sudo -E awf --config "${RUNNER_TEMP}/gh-aw/awf-config.json" --container-workdir "${GITHUB_WORKSPACE}" --mount "${RUNNER_TEMP}/gh-aw:${RUNNER_TEMP}/gh-aw:ro" --mount "${RUNNER_TEMP}/gh-aw:/host${RUNNER_TEMP}/gh-aw:ro" ${GH_AW_DOCKER_HOST_PATH_PREFIX_ARGS} --env-all --exclude-env COPILOT_GITHUB_TOKEN --exclude-env GITHUB_MCP_SERVER_TOKEN --exclude-env MCP_GATEWAY_API_KEY --log-level info --proxy-logs-dir /tmp/gh-aw/sandbox/firewall/logs --audit-dir /tmp/gh-aw/sandbox/firewall/audit --enable-host-access --allow-host-ports 80,443,8080 --skip-pull \
+ -- /bin/bash -c 'export PATH="${RUNNER_TEMP}/gh-aw/mcp-cli/bin:$PATH" && export PATH="$(find /opt/hostedtoolcache /home/runner/work/_tool -maxdepth 5 -type d -name bin 2>/dev/null | tr '\''\n'\'' '\'':'\'')$PATH"; [ -n "$GOROOT" ] && export PATH="$GOROOT/bin:$PATH" || true && GH_AW_NODE_EXEC="${GH_AW_NODE_BIN:-}"; if [ -z "$GH_AW_NODE_EXEC" ] || [ ! -x "$GH_AW_NODE_EXEC" ]; then GH_AW_NODE_EXEC="$(command -v node 2>/dev/null || true)"; fi; if [ -z "$GH_AW_NODE_EXEC" ]; then echo "node runtime missing on this runner — check runtimes.node in workflow YAML" >&2; exit 127; fi; "$GH_AW_NODE_EXEC" ${RUNNER_TEMP}/gh-aw/actions/copilot_harness.cjs /usr/local/bin/copilot --add-dir /tmp/gh-aw/ --log-level all --log-dir /tmp/gh-aw/sandbox/agent/logs/ --disable-builtin-mcps --no-ask-user --allow-all-tools --add-dir /tmp/gh-aw/cache-memory/ --allow-all-paths --add-dir "${GITHUB_WORKSPACE}" --prompt-file /tmp/gh-aw/aw-prompts/prompt.txt' 2>&1 | tee -a /tmp/gh-aw/agent-stdio.log
+ env:
+ AWF_REFLECT_ENABLED: 1
+ COPILOT_AGENT_RUNNER_TYPE: STANDALONE
+ COPILOT_DUMMY_BYOK: dummy-byok-key-for-offline-mode
+ COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }}
+ COPILOT_MODEL: ${{ vars.GH_AW_MODEL_AGENT_COPILOT || 'claude-sonnet-4.5' }}
+ GH_AW_MCP_CONFIG: /home/runner/.copilot/mcp-config.json
+ GH_AW_PHASE: agent
+ GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt
+ GH_AW_SAFE_OUTPUTS: ${{ steps.set-runtime-paths.outputs.GH_AW_SAFE_OUTPUTS }}
+ GH_AW_VERSION: v0.74.8
+ GITHUB_API_URL: ${{ github.api_url }}
+ GITHUB_AW: true
+ GITHUB_COPILOT_INTEGRATION_ID: agentic-workflows
+ GITHUB_HEAD_REF: ${{ github.head_ref }}
+ GITHUB_MCP_SERVER_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }}
+ GITHUB_REF_NAME: ${{ github.ref_name }}
+ GITHUB_SERVER_URL: ${{ github.server_url }}
+ GITHUB_STEP_SUMMARY: /tmp/gh-aw/agent-step-summary.md
+ GITHUB_WORKSPACE: ${{ github.workspace }}
+ GIT_AUTHOR_EMAIL: github-actions[bot]@users.noreply.github.com
+ GIT_AUTHOR_NAME: github-actions[bot]
+ GIT_COMMITTER_EMAIL: github-actions[bot]@users.noreply.github.com
+ GIT_COMMITTER_NAME: github-actions[bot]
+ XDG_CONFIG_HOME: /home/runner
+ - name: Detect Copilot errors
+ id: detect-copilot-errors
+ if: always()
+ continue-on-error: true
+ run: node "${RUNNER_TEMP}/gh-aw/actions/detect_copilot_errors.cjs"
+ - name: Configure Git credentials
+ env:
+ REPO_NAME: ${{ github.repository }}
+ SERVER_URL: ${{ github.server_url }}
+ GITHUB_TOKEN: ${{ github.token }}
+ run: |
+ git config --global user.email "github-actions[bot]@users.noreply.github.com"
+ git config --global user.name "github-actions[bot]"
+ git config --global am.keepcr true
+ # Re-authenticate git with GitHub token
+ SERVER_URL_STRIPPED="${SERVER_URL#https://}"
+ git remote set-url origin "https://x-access-token:${GITHUB_TOKEN}@${SERVER_URL_STRIPPED}/${REPO_NAME}.git"
+ echo "Git configured with standard GitHub Actions identity"
+ - name: Copy Copilot session state files to logs
+ if: always()
+ continue-on-error: true
+ run: bash "${RUNNER_TEMP}/gh-aw/actions/copy_copilot_session_state.sh"
+ - name: Stop MCP Gateway
+ if: always()
+ continue-on-error: true
+ env:
+ MCP_GATEWAY_PORT: ${{ steps.start-mcp-gateway.outputs.gateway-port }}
+ MCP_GATEWAY_API_KEY: ${{ steps.start-mcp-gateway.outputs.gateway-api-key }}
+ GATEWAY_PID: ${{ steps.start-mcp-gateway.outputs.gateway-pid }}
+ run: |
+ bash "${RUNNER_TEMP}/gh-aw/actions/stop_mcp_gateway.sh" "$GATEWAY_PID"
+ - name: Redact secrets in logs
+ if: always()
+ uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
+ with:
+ script: |
+ const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
+ setupGlobals(core, github, context, exec, io, getOctokit);
+ const { main } = require('${{ runner.temp }}/gh-aw/actions/redact_secrets.cjs');
+ await main();
+ env:
+ GH_AW_SECRET_NAMES: 'COPILOT_GITHUB_TOKEN,GH_AW_GITHUB_MCP_SERVER_TOKEN,GH_AW_GITHUB_TOKEN,GITHUB_TOKEN'
+ SECRET_COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }}
+ SECRET_GH_AW_GITHUB_MCP_SERVER_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN }}
+ SECRET_GH_AW_GITHUB_TOKEN: ${{ secrets.GH_AW_GITHUB_TOKEN }}
+ SECRET_GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+ - name: Append agent step summary
+ if: always()
+ run: bash "${RUNNER_TEMP}/gh-aw/actions/append_agent_step_summary.sh"
+ - name: Copy Safe Outputs
+ if: always()
+ env:
+ GH_AW_SAFE_OUTPUTS: ${{ steps.set-runtime-paths.outputs.GH_AW_SAFE_OUTPUTS }}
+ run: |
+ mkdir -p /tmp/gh-aw
+ cp "$GH_AW_SAFE_OUTPUTS" /tmp/gh-aw/safeoutputs.jsonl 2>/dev/null || true
+ - name: Ingest agent output
+ id: collect_output
+ if: always()
+ uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
+ env:
+ GH_AW_SAFE_OUTPUTS: ${{ steps.set-runtime-paths.outputs.GH_AW_SAFE_OUTPUTS }}
+ GH_AW_ALLOWED_DOMAINS: "*.githubusercontent.com,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,codeload.github.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,docs.github.com,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.blog,github.com,github.githubassets.com,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,lfs.github.com,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,patch-diff.githubusercontent.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,smtlib.cs.uiowa.edu,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com,zenodo.org"
+ GH_AW_ALLOWED_GITHUB_REFS: ""
+ GITHUB_SERVER_URL: ${{ github.server_url }}
+ GITHUB_API_URL: ${{ github.api_url }}
+ with:
+ script: |
+ const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
+ setupGlobals(core, github, context, exec, io, getOctokit);
+ const { main } = require('${{ runner.temp }}/gh-aw/actions/collect_ndjson_output.cjs');
+ await main();
+ - name: Parse agent logs for step summary
+ if: always()
+ uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
+ env:
+ GH_AW_AGENT_OUTPUT: /tmp/gh-aw/sandbox/agent/logs/
+ with:
+ script: |
+ const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
+ setupGlobals(core, github, context, exec, io, getOctokit);
+ const { main } = require('${{ runner.temp }}/gh-aw/actions/parse_copilot_log.cjs');
+ await main();
+ - name: Parse MCP Gateway logs for step summary
+ if: always()
+ id: parse-mcp-gateway
+ uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
+ with:
+ script: |
+ const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
+ setupGlobals(core, github, context, exec, io, getOctokit);
+ const { main } = require('${{ runner.temp }}/gh-aw/actions/parse_mcp_gateway_log.cjs');
+ await main();
+ - name: Print firewall logs
+ if: always()
+ continue-on-error: true
+ env:
+ AWF_LOGS_DIR: /tmp/gh-aw/sandbox/firewall/logs
+ run: |
+ # Fix permissions on firewall logs/audit dirs so they can be uploaded as artifacts
+ # AWF runs with sudo, creating files owned by root
+ sudo chmod -R a+rX /tmp/gh-aw/sandbox/firewall 2>/dev/null || true
+ # Only run awf logs summary if awf command exists (it may not be installed if workflow failed before install step)
+ if command -v awf &> /dev/null; then
+ awf logs summary | tee -a "$GITHUB_STEP_SUMMARY"
+ else
+ echo 'AWF binary not installed, skipping firewall log summary'
+ fi
+ - name: Parse token usage for step summary
+ if: always()
+ continue-on-error: true
+ uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
+ with:
+ script: |
+ const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
+ setupGlobals(core, github, context, exec, io, getOctokit);
+ const { main } = require('${{ runner.temp }}/gh-aw/actions/parse_token_usage.cjs');
+ await main();
+ - name: Print AWF reflect summary
+ if: always()
+ continue-on-error: true
+ uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
+ with:
+ script: |
+ const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
+ setupGlobals(core, github, context, exec, io, getOctokit);
+ const { main } = require('${{ runner.temp }}/gh-aw/actions/awf_reflect_summary.cjs');
+ await main();
+ - name: Write agent output placeholder if missing
+ if: always()
+ run: |
+ if [ ! -f /tmp/gh-aw/agent_output.json ]; then
+ echo '{"items":[]}' > /tmp/gh-aw/agent_output.json
+ fi
+ - name: Commit cache-memory changes
+ if: always()
+ env:
+ GH_AW_CACHE_DIR: /tmp/gh-aw/cache-memory
+ run: bash "${RUNNER_TEMP}/gh-aw/actions/commit_cache_memory_git.sh"
+ - name: Upload cache-memory data as artifact
+ uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+ if: always()
+ with:
+ name: cache-memory
+ include-hidden-files: true
+ path: /tmp/gh-aw/cache-memory
+ - name: Upload agent artifacts
+ if: always()
+ continue-on-error: true
+ uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+ with:
+ name: agent
+ path: |
+ /tmp/gh-aw/aw-prompts/prompt.txt
+ /tmp/gh-aw/sandbox/agent/logs/
+ /tmp/gh-aw/redacted-urls.log
+ /tmp/gh-aw/mcp-logs/
+ /tmp/gh-aw/agent_usage.json
+ /tmp/gh-aw/agent-stdio.log
+ /tmp/gh-aw/pre-agent-audit.txt
+ /tmp/gh-aw/agent/
+ /tmp/gh-aw/github_rate_limits.jsonl
+ /tmp/gh-aw/safeoutputs.jsonl
+ /tmp/gh-aw/agent_output.json
+ /tmp/gh-aw/aw-*.patch
+ /tmp/gh-aw/aw-*.bundle
+ /tmp/gh-aw/awf-config.json
+ /tmp/gh-aw/sandbox/firewall/logs/
+ /tmp/gh-aw/sandbox/firewall/audit/
+ /tmp/gh-aw/sandbox/firewall/awf-reflect.json
+ if-no-files-found: ignore
+
+ conclusion:
+ needs:
+ - activation
+ - agent
+ - detection
+ - safe_outputs
+ - update_cache_memory
+ if: >
+ always() && (needs.agent.result != 'skipped' || needs.activation.outputs.lockdown_check_failed == 'true' ||
+ needs.activation.outputs.stale_lock_file_failed == 'true')
+ runs-on: ubuntu-slim
+ permissions:
+ contents: read
+ discussions: write
+ issues: write
+ concurrency:
+ group: "gh-aw-conclusion-smtlib-benchmark-finder"
+ cancel-in-progress: false
+ queue: max
+ outputs:
+ incomplete_count: ${{ steps.report_incomplete.outputs.incomplete_count }}
+ noop_message: ${{ steps.noop.outputs.noop_message }}
+ tools_reported: ${{ steps.missing_tool.outputs.tools_reported }}
+ total_count: ${{ steps.missing_tool.outputs.total_count }}
+ steps:
+ - name: Setup Scripts
+ id: setup
+ uses: github/gh-aw-actions/setup@v0.74.8
+ with:
+ destination: ${{ runner.temp }}/gh-aw/actions
+ job-name: ${{ github.job }}
+ trace-id: ${{ needs.activation.outputs.setup-trace-id }}
+ parent-span-id: ${{ needs.activation.outputs.setup-parent-span-id || needs.activation.outputs.setup-span-id }}
+ env:
+ GH_AW_SETUP_WORKFLOW_NAME: "SMTLIB Benchmark Finder"
+ GH_AW_CURRENT_WORKFLOW_REF: ${{ github.repository }}/.github/workflows/smtlib-benchmark-finder.lock.yml@${{ github.ref }}
+ GH_AW_INFO_VERSION: "1.0.48"
+ GH_AW_INFO_ENGINE_ID: "copilot"
+ - name: Download agent output artifact
+ id: download-agent-output
+ continue-on-error: true
+ uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
+ with:
+ name: agent
+ path: /tmp/gh-aw/
+ - name: Setup agent output environment variable
+ id: setup-agent-output-env
+ if: steps.download-agent-output.outcome == 'success'
+ run: |
+ mkdir -p /tmp/gh-aw/
+ find "/tmp/gh-aw/" -type f -print
+ echo "GH_AW_AGENT_OUTPUT=/tmp/gh-aw/agent_output.json" >> "$GITHUB_OUTPUT"
+ - name: Process no-op messages
+ id: noop
+ uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
+ env:
+ GH_AW_AGENT_OUTPUT: ${{ steps.setup-agent-output-env.outputs.GH_AW_AGENT_OUTPUT }}
+ GH_AW_NOOP_MAX: "1"
+ GH_AW_WORKFLOW_NAME: "SMTLIB Benchmark Finder"
+ GH_AW_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
+ GH_AW_AGENT_CONCLUSION: ${{ needs.agent.result }}
+ GH_AW_NOOP_REPORT_AS_ISSUE: "false"
+ with:
+ github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }}
+ script: |
+ const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
+ setupGlobals(core, github, context, exec, io, getOctokit);
+ const { main } = require('${{ runner.temp }}/gh-aw/actions/handle_noop_message.cjs');
+ await main();
+ - name: Log detection run
+ id: detection_runs
+ uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
+ env:
+ GH_AW_AGENT_OUTPUT: ${{ steps.setup-agent-output-env.outputs.GH_AW_AGENT_OUTPUT }}
+ GH_AW_WORKFLOW_NAME: "SMTLIB Benchmark Finder"
+ GH_AW_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
+ GH_AW_DETECTION_CONCLUSION: ${{ needs.detection.outputs.detection_conclusion }}
+ GH_AW_DETECTION_REASON: ${{ needs.detection.outputs.detection_reason }}
+ with:
+ github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }}
+ script: |
+ const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
+ setupGlobals(core, github, context, exec, io, getOctokit);
+ const { main } = require('${{ runner.temp }}/gh-aw/actions/handle_detection_runs.cjs');
+ await main();
+ - name: Record missing tool
+ id: missing_tool
+ uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
+ env:
+ GH_AW_AGENT_OUTPUT: ${{ steps.setup-agent-output-env.outputs.GH_AW_AGENT_OUTPUT }}
+ GH_AW_MISSING_TOOL_CREATE_ISSUE: "true"
+ GH_AW_MISSING_TOOL_TITLE_PREFIX: "[missing tool]"
+ GH_AW_WORKFLOW_NAME: "SMTLIB Benchmark Finder"
+ with:
+ github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }}
+ script: |
+ const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
+ setupGlobals(core, github, context, exec, io, getOctokit);
+ const { main } = require('${{ runner.temp }}/gh-aw/actions/missing_tool.cjs');
+ await main();
+ - name: Record incomplete
+ id: report_incomplete
+ uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
+ env:
+ GH_AW_AGENT_OUTPUT: ${{ steps.setup-agent-output-env.outputs.GH_AW_AGENT_OUTPUT }}
+ GH_AW_REPORT_INCOMPLETE_CREATE_ISSUE: "true"
+ GH_AW_WORKFLOW_NAME: "SMTLIB Benchmark Finder"
+ with:
+ github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }}
+ script: |
+ const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
+ setupGlobals(core, github, context, exec, io, getOctokit);
+ const { main } = require('${{ runner.temp }}/gh-aw/actions/report_incomplete_handler.cjs');
+ await main();
+ - name: Handle agent failure
+ id: handle_agent_failure
+ if: always()
+ uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
+ env:
+ GH_AW_AGENT_OUTPUT: ${{ steps.setup-agent-output-env.outputs.GH_AW_AGENT_OUTPUT }}
+ GH_AW_WORKFLOW_NAME: "SMTLIB Benchmark Finder"
+ GH_AW_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
+ GH_AW_AGENT_CONCLUSION: ${{ needs.agent.result }}
+ GH_AW_WORKFLOW_ID: "smtlib-benchmark-finder"
+ GH_AW_ACTION_FAILURE_ISSUE_EXPIRES_HOURS: "168"
+ GH_AW_ENGINE_ID: "copilot"
+ GH_AW_SECRET_VERIFICATION_RESULT: ${{ needs.activation.outputs.secret_verification_result }}
+ GH_AW_CHECKOUT_PR_SUCCESS: ${{ needs.agent.outputs.checkout_pr_success }}
+ GH_AW_EFFECTIVE_TOKENS: ${{ needs.agent.outputs.effective_tokens || '' }}
+ GH_AW_EFFECTIVE_TOKENS_RATE_LIMIT_ERROR: ${{ needs.agent.outputs.effective_tokens_rate_limit_error || 'false' }}
+ GH_AW_INFERENCE_ACCESS_ERROR: ${{ needs.agent.outputs.inference_access_error }}
+ GH_AW_MCP_POLICY_ERROR: ${{ needs.agent.outputs.mcp_policy_error }}
+ GH_AW_AGENTIC_ENGINE_TIMEOUT: ${{ needs.agent.outputs.agentic_engine_timeout }}
+ GH_AW_MODEL_NOT_SUPPORTED_ERROR: ${{ needs.agent.outputs.model_not_supported_error }}
+ GH_AW_ENGINE_API_HOSTS: "api.enterprise.githubcopilot.com,api.githubcopilot.com,api.business.githubcopilot.com,api.individual.githubcopilot.com"
+ GH_AW_CREATE_DISCUSSION_ERRORS: ${{ needs.safe_outputs.outputs.create_discussion_errors }}
+ GH_AW_CREATE_DISCUSSION_ERROR_COUNT: ${{ needs.safe_outputs.outputs.create_discussion_error_count }}
+ GH_AW_LOCKDOWN_CHECK_FAILED: ${{ needs.activation.outputs.lockdown_check_failed }}
+ GH_AW_STALE_LOCK_FILE_FAILED: ${{ needs.activation.outputs.stale_lock_file_failed }}
+ GH_AW_GROUP_REPORTS: "false"
+ GH_AW_FAILURE_REPORT_AS_ISSUE: "true"
+ GH_AW_MISSING_TOOL_REPORT_AS_FAILURE: "true"
+ GH_AW_MISSING_DATA_REPORT_AS_FAILURE: "true"
+ GH_AW_TIMEOUT_MINUTES: "60"
+ GH_AW_MAX_EFFECTIVE_TOKENS: "25000000"
+ GH_AW_CACHE_MEMORY_ENABLED: "true"
+ with:
+ github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }}
+ script: |
+ const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
+ setupGlobals(core, github, context, exec, io, getOctokit);
+ const { main } = require('${{ runner.temp }}/gh-aw/actions/handle_agent_failure.cjs');
+ await main();
+
+ detection:
+ needs:
+ - activation
+ - agent
+ if: >
+ always() && needs.agent.result != 'skipped' && (needs.agent.outputs.output_types != '' || needs.agent.outputs.has_patch == 'true')
+ runs-on: ubuntu-latest
+ permissions:
+ contents: read
+ outputs:
+ detection_conclusion: ${{ steps.detection_conclusion.outputs.conclusion }}
+ detection_reason: ${{ steps.detection_conclusion.outputs.reason }}
+ detection_success: ${{ steps.detection_conclusion.outputs.success }}
+ steps:
+ - name: Setup Scripts
+ id: setup
+ uses: github/gh-aw-actions/setup@v0.74.8
+ with:
+ destination: ${{ runner.temp }}/gh-aw/actions
+ job-name: ${{ github.job }}
+ trace-id: ${{ needs.activation.outputs.setup-trace-id }}
+ parent-span-id: ${{ needs.activation.outputs.setup-parent-span-id || needs.activation.outputs.setup-span-id }}
+ env:
+ GH_AW_SETUP_WORKFLOW_NAME: "SMTLIB Benchmark Finder"
+ GH_AW_CURRENT_WORKFLOW_REF: ${{ github.repository }}/.github/workflows/smtlib-benchmark-finder.lock.yml@${{ github.ref }}
+ GH_AW_INFO_VERSION: "1.0.48"
+ GH_AW_INFO_ENGINE_ID: "copilot"
+ - name: Download agent output artifact
+ id: download-agent-output
+ continue-on-error: true
+ uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
+ with:
+ name: agent
+ path: /tmp/gh-aw/
+ - name: Setup agent output environment variable
+ id: setup-agent-output-env
+ if: steps.download-agent-output.outcome == 'success'
+ run: |
+ mkdir -p /tmp/gh-aw/
+ find "/tmp/gh-aw/" -type f -print
+ echo "GH_AW_AGENT_OUTPUT=/tmp/gh-aw/agent_output.json" >> "$GITHUB_OUTPUT"
+ - name: Checkout repository for patch context
+ if: needs.agent.outputs.has_patch == 'true'
+ uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+ with:
+ persist-credentials: false
+ # --- Threat Detection ---
+ - name: Clean stale firewall files from agent artifact
+ run: |
+ rm -rf /tmp/gh-aw/sandbox/firewall/logs
+ rm -rf /tmp/gh-aw/sandbox/firewall/audit
+ - name: Download container images
+ run: bash "${RUNNER_TEMP}/gh-aw/actions/download_docker_images.sh" ghcr.io/github/gh-aw-firewall/agent:0.25.49 ghcr.io/github/gh-aw-firewall/api-proxy:0.25.49 ghcr.io/github/gh-aw-firewall/squid:0.25.49
+ - name: Check if detection needed
+ id: detection_guard
+ if: always()
+ env:
+ OUTPUT_TYPES: ${{ needs.agent.outputs.output_types }}
+ HAS_PATCH: ${{ needs.agent.outputs.has_patch }}
+ run: |
+ if [[ -n "$OUTPUT_TYPES" || "$HAS_PATCH" == "true" ]]; then
+ echo "run_detection=true" >> "$GITHUB_OUTPUT"
+ echo "Detection will run: output_types=$OUTPUT_TYPES, has_patch=$HAS_PATCH"
+ else
+ echo "run_detection=false" >> "$GITHUB_OUTPUT"
+ echo "Detection skipped: no agent outputs or patches to analyze"
+ fi
+ - name: Clear MCP Config for detection
+ if: always() && steps.detection_guard.outputs.run_detection == 'true'
+ run: |
+ rm -f "${RUNNER_TEMP}/gh-aw/mcp-config/mcp-servers.json"
+ rm -f /home/runner/.copilot/mcp-config.json
+ rm -f "$GITHUB_WORKSPACE/.gemini/settings.json"
+ - name: Prepare threat detection files
+ if: always() && steps.detection_guard.outputs.run_detection == 'true'
+ run: |
+ mkdir -p /tmp/gh-aw/threat-detection/aw-prompts
+ cp /tmp/gh-aw/aw-prompts/prompt.txt /tmp/gh-aw/threat-detection/aw-prompts/prompt.txt 2>/dev/null || true
+ cp /tmp/gh-aw/agent_output.json /tmp/gh-aw/threat-detection/agent_output.json 2>/dev/null || true
+ for f in /tmp/gh-aw/aw-*.patch; do
+ [ -f "$f" ] && cp "$f" /tmp/gh-aw/threat-detection/ 2>/dev/null || true
+ done
+ for f in /tmp/gh-aw/aw-*.bundle; do
+ [ -f "$f" ] && cp "$f" /tmp/gh-aw/threat-detection/ 2>/dev/null || true
+ done
+ echo "Prepared threat detection files:"
+ ls -la /tmp/gh-aw/threat-detection/ 2>/dev/null || true
+ - name: Setup threat detection
+ if: always() && steps.detection_guard.outputs.run_detection == 'true'
+ uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
+ env:
+ WORKFLOW_NAME: "SMTLIB Benchmark Finder"
+ WORKFLOW_DESCRIPTION: "Monthly SMTLIB Benchmark Finder. Searches GitHub for repositories containing SMT-LIB benchmarks (.smt2 files), excludes repositories that belong to the official SMT-LIB benchmark sets (linked from smtlib.org and hosted on Zenodo), and posts a curated summary of community-contributed benchmark links as a GitHub Discussion."
+ HAS_PATCH: ${{ needs.agent.outputs.has_patch }}
+ with:
+ script: |
+ const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
+ setupGlobals(core, github, context, exec, io, getOctokit);
+ const { main } = require('${{ runner.temp }}/gh-aw/actions/setup_threat_detection.cjs');
+ await main();
+ - name: Ensure threat-detection directory and log
+ if: always() && steps.detection_guard.outputs.run_detection == 'true'
+ run: |
+ mkdir -p /tmp/gh-aw/threat-detection
+ touch /tmp/gh-aw/threat-detection/detection.log
+ - name: Setup Node.js
+ uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
+ with:
+ node-version: '24'
+ package-manager-cache: false
+ - name: Install GitHub Copilot CLI
+ run: bash "${RUNNER_TEMP}/gh-aw/actions/install_copilot_cli.sh" 1.0.48
+ env:
+ GH_HOST: github.com
+ - name: Install AWF binary
+ run: bash "${RUNNER_TEMP}/gh-aw/actions/install_awf_binary.sh" v0.25.49
+ - name: Execute GitHub Copilot CLI
+ if: always() && steps.detection_guard.outputs.run_detection == 'true'
+ continue-on-error: true
+ id: detection_agentic_execution
+ # Copilot CLI tool arguments (sorted):
+ timeout-minutes: 20
+ run: |
+ set -o pipefail
+ printf '%s' "$(date +%s%3N)" > /tmp/gh-aw/agent_cli_start_ms.txt
+ touch /tmp/gh-aw/agent-step-summary.md
+ GH_AW_NODE_BIN=$(command -v node 2>/dev/null || true)
+ export GH_AW_NODE_BIN
+ export COPILOT_API_KEY="$COPILOT_DUMMY_BYOK"
+ (umask 177 && touch /tmp/gh-aw/threat-detection/detection.log)
+ printf '%s\n' '{"$schema":"https://github.com/github/gh-aw-firewall/releases/download/v0.25.49/awf-config.schema.json","network":{"allowDomains":["api.business.githubcopilot.com","api.enterprise.githubcopilot.com","api.github.com","api.githubcopilot.com","api.individual.githubcopilot.com","github.com","host.docker.internal","telemetry.enterprise.githubcopilot.com"]},"apiProxy":{"enabled":true,"enableTokenSteering":true,"maxRuns":500,"maxEffectiveTokens":25000000},"container":{"imageTag":"0.25.49"}}' > "${RUNNER_TEMP}/gh-aw/awf-config.json"
+ cp "${RUNNER_TEMP}/gh-aw/awf-config.json" /tmp/gh-aw/awf-config.json
+ GH_AW_DOCKER_HOST_PATH_PREFIX_ARGS=""
+ if [[ "${DOCKER_HOST:-}" =~ ^tcp:// ]]; then
+ GH_AW_DOCKER_HOST_PATH_PREFIX_ARGS="--docker-host-path-prefix /tmp/gh-aw"
+ fi
+ # shellcheck disable=SC1003
+ sudo -E awf --config "${RUNNER_TEMP}/gh-aw/awf-config.json" --container-workdir "${GITHUB_WORKSPACE}" --mount "${RUNNER_TEMP}/gh-aw:${RUNNER_TEMP}/gh-aw:ro" --mount "${RUNNER_TEMP}/gh-aw:/host${RUNNER_TEMP}/gh-aw:ro" ${GH_AW_DOCKER_HOST_PATH_PREFIX_ARGS} --env-all --exclude-env COPILOT_GITHUB_TOKEN --log-level info --proxy-logs-dir /tmp/gh-aw/sandbox/firewall/logs --audit-dir /tmp/gh-aw/sandbox/firewall/audit --enable-host-access --allow-host-ports 80,443,8080 --skip-pull \
+ -- /bin/bash -c 'export PATH="$(find /opt/hostedtoolcache /home/runner/work/_tool -maxdepth 5 -type d -name bin 2>/dev/null | tr '\''\n'\'' '\'':'\'')$PATH"; [ -n "$GOROOT" ] && export PATH="$GOROOT/bin:$PATH" || true && GH_AW_NODE_EXEC="${GH_AW_NODE_BIN:-}"; if [ -z "$GH_AW_NODE_EXEC" ] || [ ! -x "$GH_AW_NODE_EXEC" ]; then GH_AW_NODE_EXEC="$(command -v node 2>/dev/null || true)"; fi; if [ -z "$GH_AW_NODE_EXEC" ]; then echo "node runtime missing on this runner — check runtimes.node in workflow YAML" >&2; exit 127; fi; "$GH_AW_NODE_EXEC" ${RUNNER_TEMP}/gh-aw/actions/copilot_harness.cjs /usr/local/bin/copilot --add-dir /tmp/gh-aw/ --log-level all --log-dir /tmp/gh-aw/sandbox/agent/logs/ --disable-builtin-mcps --no-ask-user --allow-all-tools --add-dir "${GITHUB_WORKSPACE}" --prompt-file /tmp/gh-aw/aw-prompts/prompt.txt' 2>&1 | tee -a /tmp/gh-aw/threat-detection/detection.log
+ env:
+ AWF_REFLECT_ENABLED: 1
+ COPILOT_AGENT_RUNNER_TYPE: STANDALONE
+ COPILOT_DUMMY_BYOK: dummy-byok-key-for-offline-mode
+ COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }}
+ COPILOT_MODEL: ${{ vars.GH_AW_MODEL_DETECTION_COPILOT || 'claude-sonnet-4.5' }}
+ GH_AW_PHASE: detection
+ GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt
+ GH_AW_VERSION: v0.74.8
+ GITHUB_API_URL: ${{ github.api_url }}
+ GITHUB_AW: true
+ GITHUB_COPILOT_INTEGRATION_ID: agentic-workflows
+ GITHUB_HEAD_REF: ${{ github.head_ref }}
+ GITHUB_REF_NAME: ${{ github.ref_name }}
+ GITHUB_SERVER_URL: ${{ github.server_url }}
+ GITHUB_STEP_SUMMARY: /tmp/gh-aw/agent-step-summary.md
+ GITHUB_WORKSPACE: ${{ github.workspace }}
+ GIT_AUTHOR_EMAIL: github-actions[bot]@users.noreply.github.com
+ GIT_AUTHOR_NAME: github-actions[bot]
+ GIT_COMMITTER_EMAIL: github-actions[bot]@users.noreply.github.com
+ GIT_COMMITTER_NAME: github-actions[bot]
+ XDG_CONFIG_HOME: /home/runner
+ - name: Upload threat detection log
+ if: always() && steps.detection_guard.outputs.run_detection == 'true'
+ uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+ with:
+ name: detection
+ path: /tmp/gh-aw/threat-detection/detection.log
+ if-no-files-found: ignore
+ - name: Parse and conclude threat detection
+ id: detection_conclusion
+ if: always()
+ continue-on-error: true
+ uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
+ env:
+ RUN_DETECTION: ${{ steps.detection_guard.outputs.run_detection }}
+ DETECTION_AGENTIC_EXECUTION_OUTCOME: ${{ steps.detection_agentic_execution.outcome }}
+ GH_AW_DETECTION_CONTINUE_ON_ERROR: "true"
+ with:
+ script: |
+ try {
+ const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
+ setupGlobals(core, github, context, exec, io, getOctokit);
+ const { main } = require('${{ runner.temp }}/gh-aw/actions/parse_threat_detection_results.cjs');
+ await main();
+ } catch (loadErr) {
+ const continueOnError = process.env.GH_AW_DETECTION_CONTINUE_ON_ERROR !== 'false';
+ const detectionExecutionFailed = process.env.DETECTION_AGENTIC_EXECUTION_OUTCOME === 'failure';
+ const msg = 'ERR_SYSTEM: \u274C Unexpected error loading threat detection module: ' + (loadErr && loadErr.message ? loadErr.message : String(loadErr));
+ core.error(msg);
+ core.setOutput('reason', 'parse_error');
+ if (continueOnError && !detectionExecutionFailed) {
+ core.warning('\u26A0\uFE0F ' + msg);
+ core.setOutput('conclusion', 'warning');
+ core.setOutput('success', 'false');
+ } else {
+ core.setOutput('conclusion', 'failure');
+ core.setOutput('success', 'false');
+ core.setFailed(msg);
+ }
+ }
+
+ safe_outputs:
+ needs:
+ - activation
+ - agent
+ - detection
+ if: (!cancelled()) && needs.agent.result != 'skipped' && needs.detection.result == 'success'
+ runs-on: ubuntu-slim
+ permissions:
+ contents: read
+ discussions: write
+ issues: write
+ timeout-minutes: 15
+ env:
+ GH_AW_CALLER_WORKFLOW_ID: "${{ github.repository }}/smtlib-benchmark-finder"
+ GH_AW_DETECTION_CONCLUSION: ${{ needs.detection.outputs.detection_conclusion }}
+ GH_AW_DETECTION_REASON: ${{ needs.detection.outputs.detection_reason }}
+ GH_AW_EFFECTIVE_TOKENS: ${{ needs.agent.outputs.effective_tokens }}
+ GH_AW_ENGINE_ID: "copilot"
+ GH_AW_ENGINE_MODEL: ${{ needs.agent.outputs.model }}
+ GH_AW_ENGINE_VERSION: "1.0.48"
+ GH_AW_WORKFLOW_ID: "smtlib-benchmark-finder"
+ GH_AW_WORKFLOW_NAME: "SMTLIB Benchmark Finder"
+ outputs:
+ code_push_failure_count: ${{ steps.process_safe_outputs.outputs.code_push_failure_count }}
+ code_push_failure_errors: ${{ steps.process_safe_outputs.outputs.code_push_failure_errors }}
+ create_discussion_error_count: ${{ steps.process_safe_outputs.outputs.create_discussion_error_count }}
+ create_discussion_errors: ${{ steps.process_safe_outputs.outputs.create_discussion_errors }}
+ process_safe_outputs_processed_count: ${{ steps.process_safe_outputs.outputs.processed_count }}
+ process_safe_outputs_temporary_id_map: ${{ steps.process_safe_outputs.outputs.temporary_id_map }}
+ steps:
+ - name: Setup Scripts
+ id: setup
+ uses: github/gh-aw-actions/setup@v0.74.8
+ with:
+ destination: ${{ runner.temp }}/gh-aw/actions
+ job-name: ${{ github.job }}
+ trace-id: ${{ needs.activation.outputs.setup-trace-id }}
+ parent-span-id: ${{ needs.activation.outputs.setup-parent-span-id || needs.activation.outputs.setup-span-id }}
+ env:
+ GH_AW_SETUP_WORKFLOW_NAME: "SMTLIB Benchmark Finder"
+ GH_AW_CURRENT_WORKFLOW_REF: ${{ github.repository }}/.github/workflows/smtlib-benchmark-finder.lock.yml@${{ github.ref }}
+ GH_AW_INFO_VERSION: "1.0.48"
+ GH_AW_INFO_ENGINE_ID: "copilot"
+ - name: Download agent output artifact
+ id: download-agent-output
+ continue-on-error: true
+ uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
+ with:
+ name: agent
+ path: /tmp/gh-aw/
+ - name: Setup agent output environment variable
+ id: setup-agent-output-env
+ if: steps.download-agent-output.outcome == 'success'
+ run: |
+ mkdir -p /tmp/gh-aw/
+ find "/tmp/gh-aw/" -type f -print
+ echo "GH_AW_AGENT_OUTPUT=/tmp/gh-aw/agent_output.json" >> "$GITHUB_OUTPUT"
+ - name: Configure GH_HOST for enterprise compatibility
+ id: ghes-host-config
+ shell: bash
+ run: |
+ # Derive GH_HOST from GITHUB_SERVER_URL so the gh CLI targets the correct
+ # GitHub instance (GHES/GHEC). On github.com this is a harmless no-op.
+ GH_HOST="${GITHUB_SERVER_URL#https://}"
+ GH_HOST="${GH_HOST#http://}"
+ echo "GH_HOST=${GH_HOST}" >> "$GITHUB_ENV"
+ - name: Process Safe Outputs
+ id: process_safe_outputs
+ uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
+ env:
+ GH_AW_AGENT_OUTPUT: ${{ steps.setup-agent-output-env.outputs.GH_AW_AGENT_OUTPUT }}
+ GH_AW_COMMENT_ID: ${{ needs.activation.outputs.comment_id }}
+ GH_AW_ALLOWED_DOMAINS: "*.githubusercontent.com,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,codeload.github.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,docs.github.com,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.blog,github.com,github.githubassets.com,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,lfs.github.com,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,patch-diff.githubusercontent.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,smtlib.cs.uiowa.edu,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com,zenodo.org"
+ GITHUB_SERVER_URL: ${{ github.server_url }}
+ GITHUB_API_URL: ${{ github.api_url }}
+ GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"create_discussion\":{\"category\":\"agentic workflows\",\"close_older_discussions\":true,\"expires\":2160,\"fallback_to_issue\":true,\"max\":1,\"title_prefix\":\"[SMT-LIB Benchmarks] \"},\"create_report_incomplete_issue\":{},\"mentions\":{\"enabled\":false},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"false\"},\"report_incomplete\":{}}"
+ with:
+ github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }}
+ script: |
+ const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
+ setupGlobals(core, github, context, exec, io, getOctokit);
+ const { main } = require('${{ runner.temp }}/gh-aw/actions/safe_output_handler_manager.cjs');
+ await main();
+ - name: Upload Safe Outputs Items
+ if: always()
+ uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+ with:
+ name: safe-outputs-items
+ path: |
+ /tmp/gh-aw/safe-output-items.jsonl
+ /tmp/gh-aw/temporary-id-map.json
+ if-no-files-found: ignore
+
+ update_cache_memory:
+ needs:
+ - activation
+ - agent
+ - detection
+ if: >
+ always() && (needs.detection.result == 'success' || needs.detection.result == 'skipped') &&
+ needs.agent.result == 'success'
+ runs-on: ubuntu-slim
+ permissions: {}
+ env:
+ GH_AW_WORKFLOW_ID_SANITIZED: smtlibbenchmarkfinder
+ steps:
+ - name: Setup Scripts
+ id: setup
+ uses: github/gh-aw-actions/setup@v0.74.8
+ with:
+ destination: ${{ runner.temp }}/gh-aw/actions
+ job-name: ${{ github.job }}
+ trace-id: ${{ needs.activation.outputs.setup-trace-id }}
+ parent-span-id: ${{ needs.activation.outputs.setup-parent-span-id || needs.activation.outputs.setup-span-id }}
+ env:
+ GH_AW_SETUP_WORKFLOW_NAME: "SMTLIB Benchmark Finder"
+ GH_AW_CURRENT_WORKFLOW_REF: ${{ github.repository }}/.github/workflows/smtlib-benchmark-finder.lock.yml@${{ github.ref }}
+ GH_AW_INFO_VERSION: "1.0.48"
+ GH_AW_INFO_ENGINE_ID: "copilot"
+ - name: Download cache-memory artifact (default)
+ id: download_cache_default
+ uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
+ continue-on-error: true
+ with:
+ name: cache-memory
+ path: /tmp/gh-aw/cache-memory
+ - name: Check if cache-memory folder has content (default)
+ id: check_cache_default
+ shell: bash
+ run: |
+ if [ -d "/tmp/gh-aw/cache-memory" ] && [ "$(ls -A /tmp/gh-aw/cache-memory 2>/dev/null)" ]; then
+ echo "has_content=true" >> "$GITHUB_OUTPUT"
+ else
+ echo "has_content=false" >> "$GITHUB_OUTPUT"
+ fi
+ - name: Save cache-memory to cache (default)
+ if: steps.check_cache_default.outputs.has_content == 'true'
+ uses: actions/cache/save@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
+ with:
+ key: memory-none-nopolicy-${{ env.GH_AW_WORKFLOW_ID_SANITIZED }}-${{ github.run_id }}
+ path: /tmp/gh-aw/cache-memory
+
diff --git a/.github/workflows/smtlib-benchmark-finder.md b/.github/workflows/smtlib-benchmark-finder.md
new file mode 100644
index 000000000..19c615493
--- /dev/null
+++ b/.github/workflows/smtlib-benchmark-finder.md
@@ -0,0 +1,342 @@
+---
+description: >
+ Monthly SMTLIB Benchmark Finder.
+ Searches GitHub for repositories containing SMT-LIB benchmarks (.smt2 files),
+ excludes repositories that belong to the official SMT-LIB benchmark sets
+ (linked from smtlib.org and hosted on Zenodo), and posts a curated summary
+ of community-contributed benchmark links as a GitHub Discussion.
+
+on:
+ schedule:
+ - cron: "0 8 1 * *"
+ workflow_dispatch:
+
+timeout-minutes: 60
+
+permissions: read-all
+
+network:
+ allowed:
+ - defaults
+ - github
+ - smtlib.cs.uiowa.edu
+ - zenodo.org
+
+tools:
+ cache-memory: true
+ web-fetch: {}
+ github:
+ toolsets: [default, repos]
+ bash: [":*"]
+
+safe-outputs:
+ mentions: false
+ allowed-github-references: []
+ max-bot-mentions: 1
+ create-discussion:
+ title-prefix: "[SMT-LIB Benchmarks] "
+ category: "Agentic Workflows"
+ close-older-discussions: true
+ expires: 90d
+ missing-tool:
+ create-issue: true
+ noop:
+ report-as-issue: false
+
+---
+
+# SMTLIB Benchmark Finder
+
+## Job Description
+
+Your name is ${{ github.workflow }}. You are a research analyst for the Z3 theorem
+prover repository `${{ github.repository }}`. Your mission is to discover GitHub
+repositories that host SMT-LIB benchmarks, exclude the ones that are already part
+of the official SMT-LIB benchmark distribution (linked from smtlib.org and published
+on Zenodo), and post a curated summary of community-contributed benchmark links as a
+GitHub Discussion.
+
+## Step 1: Load Cache and Determine Run Mode
+
+Check cache memory for:
+- `official_repos`: set of GitHub repository full names (`owner/repo`) and Zenodo
+ record IDs already identified as official SMT-LIB benchmark sets
+- `known_community_repos`: set of repo full names already listed in a previous report
+- `last_run_date`: ISO-8601 date string of the previous run
+
+Use the cache to skip repos already classified. On the very first run (no cache),
+perform a full discovery pass. On subsequent runs focus on repos pushed or created
+since `last_run_date`.
+
+## Step 2: Collect Official SMT-LIB Benchmark Sets to Exclude
+
+### 2.1 Scrape smtlib.org
+
+Fetch the SMT-LIB benchmarks page and extract all linked Zenodo DOIs and GitHub URLs:
+
+```bash
+curl -s "https://smtlib.cs.uiowa.edu/benchmarks.shtml" -o /tmp/smtlib-benchmarks.html
+# Also try the main page and any mirror
+curl -s "https://smtlib.cs.uiowa.edu/" -o /tmp/smtlib-home.html
+```
+
+Parse both files for:
+- Zenodo DOI links (`doi.org/10.5281/zenodo.*` or `zenodo.org/record/*`)
+- GitHub repository URLs (`github.com/...`)
+- Any other hosted benchmark archive links
+
+### 2.2 Enumerate Zenodo SMT-LIB Community Records
+
+Query the Zenodo API for all records in the SMT-LIB community:
+
+```bash
+curl -s "https://zenodo.org/api/records?communities=smt-lib&size=100&page=1" \
+ -o /tmp/zenodo-smtlib-page1.json
+
+# Check if there are more pages (paginate until empty)
+curl -s "https://zenodo.org/api/records?communities=smt-lib&size=100&page=2" \
+ -o /tmp/zenodo-smtlib-page2.json 2>/dev/null || true
+
+curl -s "https://zenodo.org/api/records?communities=smt-lib&size=100&page=3" \
+ -o /tmp/zenodo-smtlib-page3.json 2>/dev/null || true
+```
+
+For each Zenodo record extract:
+- Record ID (e.g. `5827900`)
+- Title
+- Any GitHub repository URLs listed in the description or related identifiers
+
+```bash
+python3 - <<'PYEOF'
+import json, re
+
+def extract_github_repos(text):
+ pattern = r'github\.com/([A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+)'
+ return set(re.findall(pattern, text or ''))
+
+official_repos = set()
+official_zenodo_ids = set()
+
+for fname in ['/tmp/zenodo-smtlib-page1.json', '/tmp/zenodo-smtlib-page2.json',
+ '/tmp/zenodo-smtlib-page3.json']:
+ try:
+ data = json.load(open(fname))
+ except Exception:
+ continue
+ for hit in data.get('hits', {}).get('hits', []):
+ rid = str(hit.get('id', ''))
+ official_zenodo_ids.add(rid)
+ metadata = hit.get('metadata', {})
+ description = metadata.get('description', '')
+ related = ' '.join(
+ r.get('identifier', '')
+ for r in metadata.get('related_identifiers', [])
+ )
+ title = metadata.get('title', '')
+ for repo in extract_github_repos(description + ' ' + related + ' ' + title):
+ official_repos.add(repo.lower().rstrip('.'))
+
+with open('/tmp/smtlib-benchmarks.html') as f:
+ html = f.read()
+for repo in extract_github_repos(html):
+ official_repos.add(repo.lower().rstrip('.'))
+
+print("OFFICIAL_ZENODO_IDS:", ','.join(sorted(official_zenodo_ids)) or '(none)')
+print("OFFICIAL_GITHUB_REPOS:", ','.join(sorted(official_repos)) or '(none)')
+PYEOF
+```
+
+### 2.3 Well-Known Official Repository Patterns
+
+Regardless of the above scrape, always exclude:
+- Any repo under the `SMT-LIB` GitHub organization (`SMT-LIB/*`)
+- Any repo whose name matches `smt-comp-*` that is under `SMT-Competition` org
+- The Z3 repo itself (`Z3Prover/z3`) and its forks
+
+Combine all official sources into a single exclusion set stored at
+`/tmp/official_exclusions.txt` (one `owner/repo` pattern per line, lowercase).
+
+## Step 3: Search GitHub for Community SMT-LIB Benchmark Repositories
+
+Use multiple GitHub search strategies to find repos containing `.smt2` benchmark
+files that are NOT part of the official distribution. Search for repos updated
+since the last run date (or the last 90 days for the initial run).
+
+Compute the cutoff date first:
+```bash
+CUTOFF=$(date -d "90 days ago" +%Y-%m-%d 2>/dev/null || date -v-90d +%Y-%m-%d)
+echo "Using cutoff date: $CUTOFF"
+```
+
+### Search Strategies
+
+Use the GitHub MCP server tools to run these searches:
+
+1. **Topic search**: `topic:smtlib pushed:>$CUTOFF`
+2. **Topic search variant**: `topic:smt-lib pushed:>$CUTOFF`
+3. **Topic search variant**: `topic:smt2 pushed:>$CUTOFF`
+4. **Benchmark filename pattern**: `filename:*.smt2 pushed:>$CUTOFF` (limit to top results)
+5. **Benchmarks directory pattern**: `path:benchmarks *.smt2 in:path pushed:>$CUTOFF`
+6. **README mention**: `SMT-LIB benchmarks in:readme stars:>2 pushed:>$CUTOFF`
+7. **Organization-level search**: repos under `SMT-Competition` org, if any are not already excluded
+
+For each search, collect: `full_name`, `html_url`, `description`, `stargazers_count`,
+`updated_at`, `pushed_at`, `default_branch`, `topics`.
+
+Deduplicate by `full_name`. Limit to 200 total candidates before filtering.
+
+## Step 4: Filter Out Official Benchmark Sets
+
+For each candidate repo:
+
+1. Check if `full_name.lower()` is in the exclusion set from Step 2.
+2. Check if the repo is owned by `SMT-LIB` or `SMT-Competition` org (case-insensitive).
+3. Check if the repo is a fork of a known official repo (if `fork: true` and parent is
+ in the exclusion set).
+
+Discard repos that match any of the above. Keep the rest as community benchmarks.
+
+Also apply quality filters to reduce noise — skip repos that:
+- Have 0 stars and fewer than 3 `.smt2` files (likely a student homework or test repo
+ with minimal public value); use your judgement — if the repo description clearly
+ describes a research benchmark, keep it regardless of star count.
+- Were created but never pushed after creation (empty repos).
+- Have names that are clearly course assignment repositories
+ (e.g. contain `homework`, `assignment`, `hw[0-9]`, `cs[0-9]{3}`).
+
+## Step 5: Classify Remaining Repos
+
+For each repo that survives filtering, classify it into one of these categories:
+
+| Category | Description |
+|----------|-------------|
+| **Solver evaluation** | Benchmarks used to evaluate or compare SMT solvers |
+| **Verification** | Benchmarks from program verification or model checking |
+| **Security / CTF** | Benchmarks from security research or CTF challenges |
+| **Theory / logic** | Benchmarks exploring specific SMT theories or logics |
+| **Tool output** | Benchmarks generated by another tool (e.g. a compiler, fuzzer) |
+| **Education** | Course materials or tutorials with benchmark examples |
+| **Other / unknown** | Does not fit another category |
+
+Base the classification on: repo description, README (fetch if web-fetch is available),
+topics, and directory structure. A single brief `web-fetch` of the repo's README is
+sufficient; do not fetch individual `.smt2` files.
+
+Note the dominant SMT logic(s) present, if discernible from the description or topics
+(e.g. QF_BV, QF_LIA, QF_S, NIA, …).
+
+## Step 6: Generate the Discussion Report
+
+Create a GitHub Discussion. Use heading level 3 or deeper (`###`, `####`, …) for all
+section headers; never use `##` or `#` in the body.
+Wrap long tables in `` tags to keep the report scannable.
+
+Title: `[SMT-LIB Benchmarks] Community Benchmark Repository Survey — [Month YYYY]`
+
+Structure the report as follows:
+
+```markdown
+**Period covered**: [cutoff date] – [today's date]
+**Repositories found**: N community repos (after excluding M official sets)
+**New this run**: N (not listed in previous report)
+
+### Overview
+
+1–2 sentences summarising the breadth of community SMT-LIB benchmarks found.
+
+### Community Benchmark Repositories
+
+Use `###` for category headers. Within each category, list repos as a markdown table.
+For each repo include:
+- Repo link (`[owner/repo](html_url)`)
+- Stars
+- Last pushed date
+- Dominant logic(s) (if known)
+- Brief description (from repo description or README, max 120 chars)
+
+#### Solver Evaluation
+
+| Repository | ⭐ | Last pushed | Logic(s) | Description |
+|------------|-----|------------|---------|-------------|
+| [owner/repo](url) | N | YYYY-MM-DD | QF_BV, QF_LIA | … |
+
+#### Verification
+
+[same table structure]
+
+#### Security / CTF
+
+[same table structure]
+
+#### Theory / Logic
+
+[same table structure]
+
+#### Tool Output
+
+[same table structure]
+
+#### Education
+
+[same table structure]
+
+#### Other / Unknown
+
+[same table structure]
+
+---
+
+### Exclusions Applied
+
+
+Official SMT-LIB sets excluded from this report
+
+List Zenodo record IDs and GitHub repos identified as official distributions.
+
+| Source | Identifier | Notes |
+|--------|-----------|-------|
+| Zenodo | [10.5281/zenodo.XXXXXXX](https://zenodo.org/record/XXXXXXX) | Official QF_BV benchmark set |
+| GitHub | [SMT-LIB/benchmarks-non-incremental](https://github.com/SMT-LIB/benchmarks-non-incremental) | |
+
+
+
+---
+
+### Methodology
+
+Brief note on search queries used, cutoff date, and any quality filters applied.
+```
+
+## Step 7: Update Cache Memory
+
+After posting the discussion, update cache memory with:
+- `official_repos`: updated exclusion set (union of previous + newly found)
+- `known_community_repos`: union of previous + repos listed in this report
+- `last_run_date`: today's ISO-8601 date
+- `report_url`: URL of the GitHub Discussion created
+
+## Guidelines
+
+- **Be conservative with exclusions**: when in doubt whether a repo is "official",
+ keep it in the community list rather than silently dropping it.
+- **Be accurate**: only include repos that genuinely contain SMT-LIB `.smt2` files
+ or clearly describe themselves as SMT-LIB benchmark collections.
+- **Avoid noise**: student homework repos and trivially small repos add clutter;
+ apply the quality filters from Step 4 judiciously.
+- **No source code changes**: DO NOT create pull requests or modify any source files.
+- **No copyrighted content**: DO NOT reproduce benchmark file contents; only post
+ links and metadata.
+- **Always cite sources**: include the full GitHub URL for every listed repository.
+- **Use cache**: skip repos already classified in a previous run to keep runtime short.
+- **Fail gracefully**: if GitHub search rate-limits the workflow, post whatever was
+ collected so far with a note that the search was incomplete.
+
+## Important Notes
+
+- DO NOT create pull requests or modify source files.
+- DO close older SMT-LIB Benchmarks discussions automatically (configured).
+- DO always call `create_discussion` or `noop` before the workflow ends.
+ Failing to produce any safe output triggers an automatic failure issue.
+- DO use cache memory to avoid re-processing repos already surveyed.
+- DO limit individual `web-fetch` calls (README fetches) to repos where the
+ description alone is insufficient for classification.