3
0
Fork 0
mirror of https://github.com/Z3Prover/z3 synced 2025-11-20 12:46:39 +00:00

Setting up param tuning infrastructure in C++ (#8006)

* draft attempt at optimizing cube tree with resolvents. have not tested/ran yet

* adding comments

* fix bug about needing to bubble resolvent upwards to highest ancestor

* fix bug where we need to cover the whole resolvent in the path when bubbling up

* clean up comments

* Bump actions/checkout from 4 to 5 (#7954)

Bumps [actions/checkout](https://github.com/actions/checkout) from 4 to 5.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v4...v5)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '5'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* close entire tree when sibling resolvent is empty

* integrate asms directly into cube tree, remove separate tracking

* try to fix bug about redundant resolutions, merging close and try_resolve_upwards into once function

* separate the logic again to avoid mutual recursion

* [WIP] Add a mutex to warning.cpp to ensure that warning messages from different threads don't interfere (#7963)

* Initial plan

* Add mutex to warning.cpp for thread safety

Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com>

* Remove unused variable 'first' in mpz.cpp

Removed unused variable 'first' from the function.

* fixing the order

Signed-off-by: Lev Nachmanson <levnach@hotmail.com>

* fixing the order

Signed-off-by: Lev Nachmanson <levnach@hotmail.com>

* fix the order of parameter evaluation

Signed-off-by: Lev Nachmanson <levnach@hotmail.com>

* remove AI slop

Signed-off-by: Nikolaj Bjorner <nbjorner@microsoft.com>

* param order

Signed-off-by: Lev Nachmanson <levnach@hotmail.com>

* param order

Signed-off-by: Lev Nachmanson <levnach@hotmail.com>

* param order evaluation

* parameter eval order

* parameter evaluation order

* param eval

* param eval order

* parameter eval order

Signed-off-by: Lev Nachmanson <levnach@hotmail.com>

* parameter eval order

Signed-off-by: Lev Nachmanson <levnach@hotmail.com>

* parameter eval order

Signed-off-by: Lev Nachmanson <levnach@hotmail.com>

* parameter eval order

Signed-off-by: Lev Nachmanson <levnach@hotmail.com>

* parameter eval order

Signed-off-by: Lev Nachmanson <levnach@hotmail.com>

* parameter eval order

Signed-off-by: Lev Nachmanson <levnach@hotmail.com>

* parameter eval order

Signed-off-by: Lev Nachmanson <levnach@hotmail.com>

* Bump github/codeql-action from 3 to 4 (#7971)

Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3 to 4.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update Z3_mk_datatype_sort API to accept array of sort parameters and add Z3_mk_polymorphic_datatype (#7966)

* Initial plan

* Update Z3_mk_datatype_sort API to accept array of parameters

Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com>

* Update language bindings for parametric datatype sort API

- Python: Updated DatatypeSort() to accept optional params list
- OCaml: Added mk_sort_ref_p/mk_sort_ref_ps for parametric datatypes
- .NET: Added MkDatatypeSortRef() methods with optional params
- Java: Added mkDatatypeSortRef() methods with optional params
- All changes maintain backward compatibility

Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com>

* Fix .NET build error - rename params to parameters

The 'params' keyword is reserved in C#, causing compilation errors.
Renamed parameter from 'params' to 'parameters' in MkDatatypeSortRef methods.

Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com>

* Add unit test for parametric datatypes

Added test_parametric_datatype.cpp to demonstrate polymorphic datatypes.
The test creates two concrete instantiations of a generic pair concept:
- pair_int_real with fields (first:Int, second:Real)
- pair_real_int with fields (first:Real, second:Int)
Then verifies that accessors work correctly and type checking is enforced.

Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com>

* Implement polymorphic type variables support for parametric datatypes

- Modified datatype_decl_plugin to allow type variables as parameters
- Updated mk_datatype_decl to extract type variables from field sorts
- Type variables are collected in order of first appearance
- Revised unit test to use Z3_mk_type_variable for polymorphic datatypes
- Test creates pair datatype with type variables alpha and beta
- Successfully instantiates with concrete types (pair Int Real) and (pair Real Int)
- Verifies accessor types match and equality terms are well-typed
- All tests pass

Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com>

* Add Z3_mk_polymorphic_datatype API and refactor datatype creation

- Added new API Z3_mk_polymorphic_datatype to z3_api.h
- Renamed static mk_datatype_decl to api_datatype_decl in api_datatype.cpp
- Modified api_datatype_decl to accept explicit type parameters
- Updated all callers to use renamed function
- Added test_polymorphic_datatype_api demonstrating new API usage
- Both tests pass successfully

Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com>

* Remove type variable collection logic from constructors

Removed the logic for collecting type variables from field sorts based on constructors.

* Update comments on parameter handling in api_datatype.cpp

Clarify usage of parameters in API documentation.

* Fix OCaml build error - use list instead of array for mk_datatype_sort

Changed mk_sort_ref to pass empty list [] instead of empty array [||].
Changed mk_sort_ref_p to pass params list directly instead of converting to array.
Z3native.mk_datatype_sort expects a list, not an array.

Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com>

* Add polymorphic datatype example to C++ examples

Added polymorphic_datatype_example() demonstrating:
- Creating type variables alpha and beta with Z3_mk_type_variable
- Defining parametric Pair datatype with fields of type alpha and beta
- Instantiating with concrete types (Pair Int Real) and (Pair Real Int)
- Getting constructors and accessors from instantiated datatypes
- Creating constants and expressions using the polymorphic types
- Verifying type correctness with equality (= (first p1) (second p2))

Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com>
Co-authored-by: Nikolaj Bjorner <nbjorner@microsoft.com>

* trim parametric datatype test

Signed-off-by: Nikolaj Bjorner <nbjorner@microsoft.com>

* restore single cell

Signed-off-by: Lev Nachmanson <levnach@hotmail.com>

* restore the method behavior

Signed-off-by: Lev Nachmanson <levnach@hotmail.com>

* setting up python tuning experiment, not done

* Add finite_set_value_factory for creating finite set values in model generation (#7981)

* Initial plan

* Add finite_set_value_factory implementation

Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com>

* Remove unused dl_decl_plugin variable and include

Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com>

* Update copyright and add TODOs in finite_set_value_factory

Updated copyright information and added TODO comments for handling in finite_set_value_factory methods.

* Update copyright information in finite_set_value_factory.h

Updated copyright year from 2006 to 2025.

* Implement finite_set_value_factory using array_util to create singleton sets

Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com>

* Simplify empty set creation in finite_set_value_factory

Refactor finite_set_value_factory to simplify empty set handling and remove array-specific logic.

* Change family ID for finite_set_value_factory

* Fix build error by restoring array_decl_plugin include and implementation

Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com>

* Update finite_set_value_factory.h

* Add SASSERT for finite set check in factory

Added assertion to check if the sort is a finite set.

* Rename member variable from m_util to u

* Refactor finite_set_value_factory for value handling

* Use register_value instead of direct set insertion

Replaced direct insertion into set with register_value calls.

* Update finite_set_value_factory.cpp

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com>
Co-authored-by: Nikolaj Bjorner <nbjorner@microsoft.com>

* Revert "Add finite_set_value_factory for creating finite set values in model …" (#7985)

This reverts commit 05ffc0a77b.

* Update arith_rewriter.cpp

fix memory leak introduced by update to ensure determinism

* update pythonnn prototyping experiment, need to add a couple more things

* add explicit constructors for nightly mac build failure

Signed-off-by: Nikolaj Bjorner <nbjorner@microsoft.com>

* build fixes

Signed-off-by: Nikolaj Bjorner <nbjorner@microsoft.com>

* fixes

* fix some more things but now it hangs

* change multithread to multiprocess seems to have resolved current deadlock

* fix some bugs, it seems to run now

* fix logic about checking clauses individually, and add proof prefix clause selection (naively) via the OnClause hook

* disable manylinux until segfault is resolved

Signed-off-by: Nikolaj Bjorner <nbjorner@microsoft.com>

* add the  "noexcept" keyword to value_score=(value_score&&) declaration

* expose a status flag for clauses but every single one is being coded as an assumption...

* Add a fast-path to _coerce_exprs. (#7995)

When the inputs are already the same sort, we can skip most of the
coercion logic and just return.

Currently, `_coerce_exprs` is by far the most expensive part of
building up many common Z3 ASTs, so this fast-path is a substantial
speedup for many use-cases.

* Bump actions/setup-node from 5 to 6 (#7994)

Bumps [actions/setup-node](https://github.com/actions/setup-node) from 5 to 6.
- [Release notes](https://github.com/actions/setup-node/releases)
- [Commits](https://github.com/actions/setup-node/compare/v5...v6)

---
updated-dependencies:
- dependency-name: actions/setup-node
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Enabling Control Flow Guard (CFG) by default for MSVC on Windows, with options to disable CFG. (#7988)

* Enabling Control Flow Guard by default for MSVC on Windows, with options to disable it.

* Fix configuration error for non-MSVC compilers.

* Reviewed and updated configuration for Python build and added comment for CFG.

* try exponential delay in grobner

Signed-off-by: Lev Nachmanson <levnach@hotmail.com>

* throttle grobner method more actively

Signed-off-by: Lev Nachmanson <levnach@hotmail.com>

* enable always add all coeffs in nlsat

Signed-off-by: Lev Nachmanson <levnach@hotmail.com>

* initial parameter probe thread setup in C++

* more param tuning setup

* setting up the param probe solvers and mutation generator

* adding the learned clauses from the internalizer

* fix some things for clause replay

* score the param probes, but i can't figure out how to access the relevant solver statistics fields from the statistics obj

* set up pattern to notify batch manager so worker threads can update their params according
ly

---------

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Lev Nachmanson <levnach@hotmail.com>
Signed-off-by: Nikolaj Bjorner <nbjorner@microsoft.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com>
Co-authored-by: Nikolaj Bjorner <nbjorner@microsoft.com>
Co-authored-by: Lev Nachmanson <levnach@hotmail.com>
Co-authored-by: Nelson Elhage <nelhage@nelhage.com>
Co-authored-by: hwisungi <hwisungi@users.noreply.github.com>
This commit is contained in:
Ilana Shapiro 2025-10-30 03:15:32 -07:00 committed by GitHub
parent 9369a14825
commit e4a285187b
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
17 changed files with 454 additions and 46 deletions

View file

@ -362,34 +362,75 @@ endif()
include(${PROJECT_SOURCE_DIR}/cmake/compiler_lto.cmake)
################################################################################
# Control flow integrity
# Control flow integrity (Clang only)
################################################################################
option(Z3_ENABLE_CFI "Enable control flow integrity checking" OFF)
option(Z3_ENABLE_CFI "Enable Control Flow Integrity security checks" OFF)
if (Z3_ENABLE_CFI)
set(build_types_with_cfi "RELEASE" "RELWITHDEBINFO")
if (NOT CMAKE_CXX_COMPILER_ID MATCHES "Clang")
message(FATAL_ERROR "Z3_ENABLE_CFI is only supported with Clang compiler. "
"Current compiler: ${CMAKE_CXX_COMPILER_ID}. "
"You should set Z3_ENABLE_CFI to OFF or use Clang to compile.")
endif()
if (NOT Z3_LINK_TIME_OPTIMIZATION)
message(FATAL_ERROR "Cannot enable control flow integrity checking without link-time optimization."
message(FATAL_ERROR "Cannot enable Control Flow Integrity without link-time optimization. "
"You should set Z3_LINK_TIME_OPTIMIZATION to ON or Z3_ENABLE_CFI to OFF.")
endif()
set(build_types_with_cfi "RELEASE" "RELWITHDEBINFO")
if (DEFINED CMAKE_CONFIGURATION_TYPES)
# Multi configuration generator
message(STATUS "Note CFI is only enabled for the following configurations: ${build_types_with_cfi}")
# No need for else because this is the same as the set that LTO requires.
endif()
if ("${CMAKE_CXX_COMPILER_ID}" MATCHES "Clang")
z3_add_cxx_flag("-fsanitize=cfi" REQUIRED)
z3_add_cxx_flag("-fsanitize-cfi-cross-dso" REQUIRED)
elseif (CMAKE_CXX_COMPILER_ID STREQUAL "MSVC")
z3_add_cxx_flag("/guard:cf" REQUIRED)
message(STATUS "Enabling CFI for MSVC")
foreach (_build_type ${build_types_with_cfi})
message(STATUS "Enabling CFI for MSVC")
string(APPEND CMAKE_EXE_LINKER_FLAGS_${_build_type} " /GUARD:CF")
string(APPEND CMAKE_SHARED_LINKER_FLAGS_${_build_type} " /GUARD:CF")
endforeach()
message(STATUS "Enabling Control Flow Integrity (CFI) for Clang")
z3_add_cxx_flag("-fsanitize=cfi" REQUIRED)
z3_add_cxx_flag("-fsanitize-cfi-cross-dso" REQUIRED)
endif()
# End CFI section
################################################################################
# Control Flow Guard (MSVC only)
################################################################################
# Default CFG to ON for MSVC, OFF for other compilers.
if (CMAKE_CXX_COMPILER_ID STREQUAL "MSVC")
option(Z3_ENABLE_CFG "Enable Control Flow Guard security checks" ON)
else()
option(Z3_ENABLE_CFG "Enable Control Flow Guard security checks" OFF)
endif()
if (Z3_ENABLE_CFG)
if (NOT CMAKE_CXX_COMPILER_ID STREQUAL "MSVC")
message(FATAL_ERROR "Z3_ENABLE_CFG is only supported with MSVC compiler. "
"Current compiler: ${CMAKE_CXX_COMPILER_ID}. "
"You should remove Z3_ENABLE_CFG or set it to OFF or use MSVC to compile.")
endif()
# Check for incompatible options (handle both / and - forms for robustness)
string(REGEX MATCH "[-/]ZI" _has_ZI "${CMAKE_CXX_FLAGS} ${CMAKE_CXX_FLAGS_DEBUG} ${CMAKE_CXX_FLAGS_RELEASE} ${CMAKE_CXX_FLAGS_RELWITHDEBINFO} ${CMAKE_CXX_FLAGS_MINSIZEREL}")
string(REGEX MATCH "[-/]clr" _has_clr "${CMAKE_CXX_FLAGS} ${CMAKE_CXX_FLAGS_DEBUG} ${CMAKE_CXX_FLAGS_RELEASE} ${CMAKE_CXX_FLAGS_RELWITHDEBINFO} ${CMAKE_CXX_FLAGS_MINSIZEREL}")
if(_has_ZI)
message(WARNING "/guard:cf is incompatible with /ZI (Edit and Continue debug information). "
"Control Flow Guard will be disabled due to /ZI option.")
elseif(_has_clr)
message(WARNING "/guard:cf is incompatible with /clr (Common Language Runtime compilation). "
"Control Flow Guard will be disabled due to /clr option.")
else()
message(FATAL_ERROR "Can't enable control flow integrity for compiler \"${CMAKE_CXX_COMPILER_ID}\"."
"You should set Z3_ENABLE_CFI to OFF or use Clang or MSVC to compile.")
# Enable Control Flow Guard if no incompatible options are present
message(STATUS "Enabling Control Flow Guard (/guard:cf) and ASLR (/DYNAMICBASE) for MSVC")
z3_add_cxx_flag("/guard:cf" REQUIRED)
string(APPEND CMAKE_EXE_LINKER_FLAGS " /GUARD:CF /DYNAMICBASE")
string(APPEND CMAKE_SHARED_LINKER_FLAGS " /GUARD:CF /DYNAMICBASE")
endif()
else()
if (CMAKE_CXX_COMPILER_ID STREQUAL "MSVC")
# Explicitly disable Control Flow Guard when Z3_ENABLE_CFG is OFF
message(STATUS "Disabling Control Flow Guard (/guard:cf-) for MSVC")
z3_add_cxx_flag("/guard:cf-" REQUIRED)
string(APPEND CMAKE_EXE_LINKER_FLAGS " /GUARD:NO")
string(APPEND CMAKE_SHARED_LINKER_FLAGS " /GUARD:NO")
endif()
endif()