mirror of
https://github.com/Z3Prover/z3
synced 2026-02-19 07:04:22 +00:00
Setting up param tuning python prototyping experiment (#7993)
* draft attempt at optimizing cube tree with resolvents. have not tested/ran yet
* adding comments
* fix bug about needing to bubble resolvent upwards to highest ancestor
* fix bug where we need to cover the whole resolvent in the path when bubbling up
* clean up comments
* Bump actions/checkout from 4 to 5 (#7954)
Bumps [actions/checkout](https://github.com/actions/checkout) from 4 to 5.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v4...v5)
---
updated-dependencies:
- dependency-name: actions/checkout
dependency-version: '5'
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* close entire tree when sibling resolvent is empty
* integrate asms directly into cube tree, remove separate tracking
* try to fix bug about redundant resolutions, merging close and try_resolve_upwards into once function
* separate the logic again to avoid mutual recursion
* [WIP] Add a mutex to warning.cpp to ensure that warning messages from different threads don't interfere (#7963)
* Initial plan
* Add mutex to warning.cpp for thread safety
Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com>
* Remove unused variable 'first' in mpz.cpp
Removed unused variable 'first' from the function.
* fixing the order
Signed-off-by: Lev Nachmanson <levnach@hotmail.com>
* fixing the order
Signed-off-by: Lev Nachmanson <levnach@hotmail.com>
* fix the order of parameter evaluation
Signed-off-by: Lev Nachmanson <levnach@hotmail.com>
* remove AI slop
Signed-off-by: Nikolaj Bjorner <nbjorner@microsoft.com>
* param order
Signed-off-by: Lev Nachmanson <levnach@hotmail.com>
* param order
Signed-off-by: Lev Nachmanson <levnach@hotmail.com>
* param order evaluation
* parameter eval order
* parameter evaluation order
* param eval
* param eval order
* parameter eval order
Signed-off-by: Lev Nachmanson <levnach@hotmail.com>
* parameter eval order
Signed-off-by: Lev Nachmanson <levnach@hotmail.com>
* parameter eval order
Signed-off-by: Lev Nachmanson <levnach@hotmail.com>
* parameter eval order
Signed-off-by: Lev Nachmanson <levnach@hotmail.com>
* parameter eval order
Signed-off-by: Lev Nachmanson <levnach@hotmail.com>
* parameter eval order
Signed-off-by: Lev Nachmanson <levnach@hotmail.com>
* parameter eval order
Signed-off-by: Lev Nachmanson <levnach@hotmail.com>
* Bump github/codeql-action from 3 to 4 (#7971)
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3 to 4.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/v3...v4)
---
updated-dependencies:
- dependency-name: github/codeql-action
dependency-version: '4'
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Update Z3_mk_datatype_sort API to accept array of sort parameters and add Z3_mk_polymorphic_datatype (#7966)
* Initial plan
* Update Z3_mk_datatype_sort API to accept array of parameters
Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com>
* Update language bindings for parametric datatype sort API
- Python: Updated DatatypeSort() to accept optional params list
- OCaml: Added mk_sort_ref_p/mk_sort_ref_ps for parametric datatypes
- .NET: Added MkDatatypeSortRef() methods with optional params
- Java: Added mkDatatypeSortRef() methods with optional params
- All changes maintain backward compatibility
Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com>
* Fix .NET build error - rename params to parameters
The 'params' keyword is reserved in C#, causing compilation errors.
Renamed parameter from 'params' to 'parameters' in MkDatatypeSortRef methods.
Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com>
* Add unit test for parametric datatypes
Added test_parametric_datatype.cpp to demonstrate polymorphic datatypes.
The test creates two concrete instantiations of a generic pair concept:
- pair_int_real with fields (first:Int, second:Real)
- pair_real_int with fields (first:Real, second:Int)
Then verifies that accessors work correctly and type checking is enforced.
Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com>
* Implement polymorphic type variables support for parametric datatypes
- Modified datatype_decl_plugin to allow type variables as parameters
- Updated mk_datatype_decl to extract type variables from field sorts
- Type variables are collected in order of first appearance
- Revised unit test to use Z3_mk_type_variable for polymorphic datatypes
- Test creates pair datatype with type variables alpha and beta
- Successfully instantiates with concrete types (pair Int Real) and (pair Real Int)
- Verifies accessor types match and equality terms are well-typed
- All tests pass
Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com>
* Add Z3_mk_polymorphic_datatype API and refactor datatype creation
- Added new API Z3_mk_polymorphic_datatype to z3_api.h
- Renamed static mk_datatype_decl to api_datatype_decl in api_datatype.cpp
- Modified api_datatype_decl to accept explicit type parameters
- Updated all callers to use renamed function
- Added test_polymorphic_datatype_api demonstrating new API usage
- Both tests pass successfully
Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com>
* Remove type variable collection logic from constructors
Removed the logic for collecting type variables from field sorts based on constructors.
* Update comments on parameter handling in api_datatype.cpp
Clarify usage of parameters in API documentation.
* Fix OCaml build error - use list instead of array for mk_datatype_sort
Changed mk_sort_ref to pass empty list [] instead of empty array [||].
Changed mk_sort_ref_p to pass params list directly instead of converting to array.
Z3native.mk_datatype_sort expects a list, not an array.
Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com>
* Add polymorphic datatype example to C++ examples
Added polymorphic_datatype_example() demonstrating:
- Creating type variables alpha and beta with Z3_mk_type_variable
- Defining parametric Pair datatype with fields of type alpha and beta
- Instantiating with concrete types (Pair Int Real) and (Pair Real Int)
- Getting constructors and accessors from instantiated datatypes
- Creating constants and expressions using the polymorphic types
- Verifying type correctness with equality (= (first p1) (second p2))
Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com>
Co-authored-by: Nikolaj Bjorner <nbjorner@microsoft.com>
* trim parametric datatype test
Signed-off-by: Nikolaj Bjorner <nbjorner@microsoft.com>
* restore single cell
Signed-off-by: Lev Nachmanson <levnach@hotmail.com>
* restore the method behavior
Signed-off-by: Lev Nachmanson <levnach@hotmail.com>
* setting up python tuning experiment, not done
* Add finite_set_value_factory for creating finite set values in model generation (#7981)
* Initial plan
* Add finite_set_value_factory implementation
Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com>
* Remove unused dl_decl_plugin variable and include
Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com>
* Update copyright and add TODOs in finite_set_value_factory
Updated copyright information and added TODO comments for handling in finite_set_value_factory methods.
* Update copyright information in finite_set_value_factory.h
Updated copyright year from 2006 to 2025.
* Implement finite_set_value_factory using array_util to create singleton sets
Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com>
* Simplify empty set creation in finite_set_value_factory
Refactor finite_set_value_factory to simplify empty set handling and remove array-specific logic.
* Change family ID for finite_set_value_factory
* Fix build error by restoring array_decl_plugin include and implementation
Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com>
* Update finite_set_value_factory.h
* Add SASSERT for finite set check in factory
Added assertion to check if the sort is a finite set.
* Rename member variable from m_util to u
* Refactor finite_set_value_factory for value handling
* Use register_value instead of direct set insertion
Replaced direct insertion into set with register_value calls.
* Update finite_set_value_factory.cpp
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com>
Co-authored-by: Nikolaj Bjorner <nbjorner@microsoft.com>
* Revert "Add finite_set_value_factory for creating finite set values in model …" (#7985)
This reverts commit 05ffc0a77b.
* Update arith_rewriter.cpp
fix memory leak introduced by update to ensure determinism
* update pythonnn prototyping experiment, need to add a couple more things
* add explicit constructors for nightly mac build failure
Signed-off-by: Nikolaj Bjorner <nbjorner@microsoft.com>
* build fixes
Signed-off-by: Nikolaj Bjorner <nbjorner@microsoft.com>
* fixes
* fix some more things but now it hangs
* change multithread to multiprocess seems to have resolved current deadlock
* fix some bugs, it seems to run now
* fix logic about checking clauses individually, and add proof prefix clause selection (naively) via the OnClause hook
* disable manylinux until segfault is resolved
Signed-off-by: Nikolaj Bjorner <nbjorner@microsoft.com>
* add the "noexcept" keyword to value_score=(value_score&&) declaration
* expose a status flag for clauses but every single one is being coded as an assumption...
* Add a fast-path to _coerce_exprs. (#7995)
When the inputs are already the same sort, we can skip most of the
coercion logic and just return.
Currently, `_coerce_exprs` is by far the most expensive part of
building up many common Z3 ASTs, so this fast-path is a substantial
speedup for many use-cases.
* Bump actions/setup-node from 5 to 6 (#7994)
Bumps [actions/setup-node](https://github.com/actions/setup-node) from 5 to 6.
- [Release notes](https://github.com/actions/setup-node/releases)
- [Commits](https://github.com/actions/setup-node/compare/v5...v6)
---
updated-dependencies:
- dependency-name: actions/setup-node
dependency-version: '6'
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
---------
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Lev Nachmanson <levnach@hotmail.com>
Signed-off-by: Nikolaj Bjorner <nbjorner@microsoft.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: NikolajBjorner <3085284+NikolajBjorner@users.noreply.github.com>
Co-authored-by: Nikolaj Bjorner <nbjorner@microsoft.com>
Co-authored-by: Lev Nachmanson <levnach@hotmail.com>
Co-authored-by: Nelson Elhage <nelhage@nelhage.com>
This commit is contained in:
Ilana Shapiro
GitHub
parent
c5d65cdedd
commit
a4bcd74ba5
33 changed files with 989 additions and 118 deletions
254
param-tuning-experiment.py
Normal file
254
param-tuning-experiment.py
Normal file
|
|
@ -0,0 +1,254 @@
|
|||
from multiprocessing import Process
|
||||
import math, random
|
||||
|
||||
import sys, os
|
||||
sys.path.insert(0, os.path.abspath("build/python"))
|
||||
os.environ["Z3_LIBRARY_PATH"] = os.path.abspath("build")
|
||||
|
||||
# import z3
|
||||
# print("Using z3 from:", z3.__file__)
|
||||
|
||||
from z3 import *
|
||||
|
||||
MAX_CONFLICTS = 100
|
||||
MAX_EXAMPLES = 5
|
||||
bench_dir = "../z3-poly-testing/inputs/QF_NIA_small"
|
||||
|
||||
BASE_PARAM_CANDIDATES = [
|
||||
("smt.arith.eager_eq_axioms", False),
|
||||
("smt.restart_factor", 1.2),
|
||||
("smt.relevancy", 0),
|
||||
("smt.phase_caching_off", 200),
|
||||
("smt.phase_caching_on", 600),
|
||||
]
|
||||
|
||||
# --------------------------
|
||||
# One class: BatchManager
|
||||
# --------------------------
|
||||
class BatchManager:
|
||||
def __init__(self):
|
||||
self.best_param_state = None
|
||||
self.best_score = (math.inf, math.inf, math.inf)
|
||||
self.search_complete = False
|
||||
|
||||
def mark_complete(self):
|
||||
self.search_complete = True
|
||||
|
||||
def maybe_update_best(self, param_state, triple):
|
||||
if self._better(triple, self.best_score):
|
||||
self.best_param_state = list(param_state)
|
||||
self.best_score = triple
|
||||
|
||||
@staticmethod
|
||||
def _better(a, b):
|
||||
return a < b # lexicographic compare
|
||||
|
||||
|
||||
# -------------------
|
||||
# Helpers
|
||||
# -------------------
|
||||
|
||||
def solver_from_file(filepath):
|
||||
s = Solver()
|
||||
s.set("smt.auto_config", False)
|
||||
s.from_file(filepath)
|
||||
return s
|
||||
|
||||
|
||||
def apply_param_state(s, param_state):
|
||||
print(f"Applying param state: {param_state}")
|
||||
for name, value in param_state:
|
||||
s.set(name, value)
|
||||
|
||||
|
||||
def stats_tuple(st):
|
||||
def get(key):
|
||||
return int(st.get_key_value(key)) if key in st.keys() else 0
|
||||
return (get("conflicts"), get("decisions"), get("rlimit count"))
|
||||
|
||||
|
||||
# --------------------------
|
||||
# Protocol steps
|
||||
# --------------------------
|
||||
|
||||
def run_prefix_step(S, K, clause_limit):
|
||||
clauses = []
|
||||
|
||||
def on_clause(premises, deps, clause, status):
|
||||
print(f" [OnClause] collected clause status: {status}, clause: {clause}")
|
||||
if len(clauses) < clause_limit:
|
||||
clauses.append(clause)
|
||||
|
||||
OnClause(S, on_clause)
|
||||
S.set("max_conflicts", K)
|
||||
r = S.check()
|
||||
return r, clauses
|
||||
|
||||
# Replay proof prefix on an existing PPS_solver (no solver recreation)
|
||||
# Solver continues from its current state.
|
||||
def replay_prefix_on_pps(PPS_solver, clauses, param_state, budget):
|
||||
print(f"[Replaying] on PPS with params={param_state} and budget={budget}")
|
||||
apply_param_state(PPS_solver, param_state)
|
||||
|
||||
total_conflicts = total_decisions = total_rlimit = 0
|
||||
|
||||
# For each learned clause Cj = [l1, l2, ...], check ¬(l1 ∨ l2 ∨ ...)
|
||||
for idx, Cj in enumerate(clauses):
|
||||
lits = [l.translate(PPS_solver.ctx) for l in Cj]
|
||||
|
||||
negated_lits = []
|
||||
for l in lits:
|
||||
negated_lits.append(Not(l))
|
||||
|
||||
PPS_solver.set("max_conflicts", budget)
|
||||
r = PPS_solver.check(negated_lits)
|
||||
st = PPS_solver.statistics()
|
||||
|
||||
c, d, rl = stats_tuple(st)
|
||||
total_conflicts += c
|
||||
total_decisions += d
|
||||
total_rlimit += rl
|
||||
|
||||
print(f" [C{idx}] result={r}, conflicts={c}, decisions={d}, rlimit={rl}")
|
||||
|
||||
return (total_conflicts, total_decisions, total_rlimit)
|
||||
|
||||
|
||||
# For each PPS_i, replay the proof prefix of S
|
||||
def replay_proof_prefixes(clauses, param_states, PPS_solvers, K, eps=200):
|
||||
budget = K + eps
|
||||
base_param_state, candidate_param_states = param_states[0], param_states[1:]
|
||||
|
||||
# PPS_0 (baseline)
|
||||
score0 = replay_prefix_on_pps(PPS_solvers[0], clauses, base_param_state, budget)
|
||||
best_param_state, best_score = base_param_state, score0
|
||||
|
||||
# PPS_i, i > 0
|
||||
for i, p_state in enumerate(candidate_param_states, start=1):
|
||||
score = replay_prefix_on_pps(PPS_solvers[i], clauses, p_state, budget)
|
||||
if score < best_score:
|
||||
best_param_state, best_score = p_state, score
|
||||
|
||||
return best_param_state, best_score
|
||||
|
||||
# return a variant of the given param state
|
||||
def perturbate(param_state):
|
||||
new_state = []
|
||||
for name, val in param_state:
|
||||
if isinstance(val, (int, float)) and "restart_factor" in name:
|
||||
# perturb multiplicatively +/-10%
|
||||
factor = random.choice([0.9, 1.1])
|
||||
new_state.append((name, round(val * factor, 3)))
|
||||
elif isinstance(val, int) and "phase_caching" in name:
|
||||
# pick half or double
|
||||
new_val = random.choice([max(1, val // 2), val * 2])
|
||||
new_state.append((name, new_val))
|
||||
elif name == "smt.relevancy":
|
||||
# pick random alternative from {0,1,2}
|
||||
new_val = random.choice([0, 1, 2])
|
||||
new_state.append((name, new_val))
|
||||
else:
|
||||
# unchanged
|
||||
new_state.append((name, val))
|
||||
return new_state
|
||||
|
||||
# --------------------------
|
||||
# Protocol iteration
|
||||
# --------------------------
|
||||
|
||||
def protocol_iteration(filepath, manager, S, PPS_solvers, PPS_states, K, eps=200):
|
||||
# --- Proof Prefix Solver (S) ---
|
||||
P = manager.best_param_state or BASE_PARAM_CANDIDATES
|
||||
apply_param_state(S, P)
|
||||
|
||||
# Run S with max conflicts K
|
||||
# Simultaneously, collect subset of conflict clauses from the bounded run of S.
|
||||
# Right now clause collection is pretty naive as we just take the first clause_limit clauses from OnClause
|
||||
print(f"[S] Running proof prefix solver with params={P} and max_conflicts={K}")
|
||||
r, C_list = run_prefix_step(S, K, clause_limit=MAX_EXAMPLES)
|
||||
|
||||
# If S returns SAT or UNSAT we have a verdict
|
||||
# Tell the central dispatch that search is complete and exit
|
||||
if r == sat or r == unsat:
|
||||
print(f"[S] {os.path.basename(filepath)} → {r} (within max_conflicts={K}). Search complete.")
|
||||
manager.mark_complete()
|
||||
return
|
||||
|
||||
# For each PPS_i, replay the proof prefix of S
|
||||
print(f"[Replaying] Replaying proof prefix on PPS solvers with budget={K + eps}")
|
||||
best_state, best_score = replay_proof_prefixes(C_list, PPS_states, PPS_solvers, K, eps)
|
||||
|
||||
if best_state != P:
|
||||
print(f"[Dispatch] updating best param state")
|
||||
manager.maybe_update_best(best_state, best_score)
|
||||
P = best_state
|
||||
|
||||
# Update PPS_0 to use P (if it changed), and update all PPS_i > 0 with new perturbations of P
|
||||
PPS_states[0] = P
|
||||
for i in range(1, len(PPS_states)):
|
||||
PPS_states[i] = perturbate(P)
|
||||
|
||||
return PPS_states
|
||||
|
||||
|
||||
# --------------------------
|
||||
# Prefix probing thread
|
||||
# --------------------------
|
||||
|
||||
def prefix_probe_thread(filepath, manager):
|
||||
# Proof prefix solver S
|
||||
S = solver_from_file(filepath)
|
||||
apply_param_state(S, BASE_PARAM_CANDIDATES)
|
||||
|
||||
PPS_solvers = []
|
||||
PPS_states = []
|
||||
|
||||
# set up the 4 variant parameter probe solvers PPS_1 ... PPS_4 as new contexts on the proof prefix solver S
|
||||
for i in range(4):
|
||||
st = BASE_PARAM_CANDIDATES if i == 0 else perturbate(BASE_PARAM_CANDIDATES) # PPS_0 uses base params
|
||||
ctx = Context()
|
||||
PPS_solver = S.translate(ctx) # clone S (proof prefix) into new context
|
||||
apply_param_state(PPS_solver, st)
|
||||
PPS_solvers.append(PPS_solver)
|
||||
PPS_states.append(st)
|
||||
print(f"[Init] PPS_{i} inherited prefix in new context with params={st}")
|
||||
|
||||
# Reuse the same solvers each iteration
|
||||
iteration = 0
|
||||
while not manager.search_complete:
|
||||
print(f"\n[PrefixThread] Iteration {iteration}")
|
||||
PPS_states = protocol_iteration(filepath, manager, S, PPS_solvers, PPS_states, K=MAX_CONFLICTS, eps=200)
|
||||
iteration += 1
|
||||
|
||||
|
||||
# --------------------------
|
||||
# Main
|
||||
# --------------------------
|
||||
|
||||
def run_main_solver(filepath):
|
||||
set_param("parallel.enable", True)
|
||||
main_solver = solver_from_file(filepath)
|
||||
apply_param_state(main_solver, BASE_PARAM_CANDIDATES)
|
||||
print(f"[Main] Started main solver on {os.path.basename(filepath)} with parallel.enable=True")
|
||||
r = main_solver.check()
|
||||
print(f"[Main] {os.path.basename(filepath)} → {r}")
|
||||
|
||||
def main():
|
||||
manager = BatchManager()
|
||||
for benchmark in os.listdir(bench_dir):
|
||||
if benchmark != "From_T2__hqr.t2_fixed__term_unfeasibility_1_0.smt2":
|
||||
continue
|
||||
|
||||
filepath = os.path.join(bench_dir, benchmark)
|
||||
prefix_proc = Process(target=prefix_probe_thread, args=(filepath, manager))
|
||||
main_proc = Process(target=run_main_solver, args=(filepath,))
|
||||
prefix_proc.start()
|
||||
main_proc.start()
|
||||
prefix_proc.join()
|
||||
main_proc.join()
|
||||
|
||||
if manager.best_param_state:
|
||||
print(f"\n[GLOBAL] Best parameter state: {manager.best_param_state} with score {manager.best_score}")
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Loading…
Add table
Add a link
Reference in a new issue