* Add rewrite for array selects of chain of stores of a same value
Example:
```smt
(declare-fun mem () (Array (_ BitVec 4) (_ BitVec 4)))
(declare-const x (_ BitVec 4))
(declare-const y (_ BitVec 4))
; simplifies to #x1
(simplify (select (store (store (store mem #x1 #x1) y #x1) x #x1) #x1))
```
* Update array_rewriter.cpp
* Update array_rewriter.cpp
This update includes an experimental feature to access a congruence closure data-structure after search.
It comes with several caveats as pre-processing is free to eliminate terms. It is therefore necessary to use a solver that does not eliminate the terms you want to track for congruence of. This is partially addressed by using SimpleSolver or incremental mode solving.
```python
from z3 import *
s = SimpleSolver()
x, y, z = Ints('x y z')
s.add(x == y)
s.add(y == z)
s.check()
print(s.root(x), s.root(y), s.root(z))
print(s.next(x), s.next(y), s.next(z))
```
This would crash before:
(declare-fun x () (_ BitVec 4))
(assert (not (bvule x #x1)))
(apply elim-uncnstr2)
That's because the index_set iterator was querying qtail to compute the end of the iteration
But the problem is that elim-uncnstr2 may add new fmls to the goal, as in this case.
The bvule is replaced with an 'or', but since it's negated, it turns into 2 goals
Solve the issue by freezing the qtail for the iteration loop.
This is the right behavior for elim-uncnstr2, as it can't rewrite exprs that haven't been analyzed before
@NikolajBjorner please check if this the right behavior for the other simplifiers. Thank you
- enable sat.smt in smt_tactic that
is invoked by default on first goals
add flatten-clauses
add push-ite
have tptp5 front-end pretty print SMT2 formulas a little nicer.
rename size() to qtail() and introduce shortcuts
ensure tactic goals are not updated if they are in inconsistent state (because indices could be invalidated)
- add sat.smt option to enable the new incremental core (it is not ready for mainstream consumption as cloning and other features are not implemented and it hasn't been tested in any detail yet).
- move "name" into attribute on simplifier so it can be reused for diagnostics by the seq-simplifier.
other updates:
- change signature of advance_qhead to simplify call sites
- have model reconstruction replay work on a tail of dependent_expr state, while adding formulas to the tail.
inc_sat_solver uses bit-blaster, card2bv and max_bv_sharing.
By turning these into simplifiers it will be possible to remove
dependencies on tactics and goals in inc_sat_simplifier and instead use a modular and general incremental pre-processing infrastructure.
literals true/false are not necessarily created when the merge flag is set.
Also disable merge_tf for if-then-else expressions
Perhaps even not insert children of if expressions into congruence table?
perf fix for propagation behavior for equalities in the new core.
The old behavior was not to allow congruence closure on equalities.
The new behavior is to just not allow merging tf with equalities unless they appear somewhere in a foreign context (not under a Boolean operator)
The change re-surfaces merge_tf and enable_cgc distinction from the old core.
They can both be turned on or off.
merge_enabled renamed to cgc_enabled
The change is highly likely to introduce regressions in the new core.
Change propagation of literals from congruence:
- track antecedent enode. There are four ways to propagate
literals from the egraph.
- the literal is an equality and the two arguments are congruent
- the antecedent is merged with node n and the antecedent has a Boolean variable assignment.
- the antecedent is true or false, they are merged.
- the merge_tf flag is toggled to true but the node n has not been merged with true/false
- remove reduce_invertible. It is subsumed by reduce_uncstr(2)
- introduce a simplifier for reduce_unconstrained. It uses reference counting to deal with inefficiency bug of legacy reduce_uncstr. It decomposes theory plugins into expr_inverter.
reduce_invertible is a tactic used in most built-in scenarios. It is useful for removing subterms that can be eliminated using "cheap" quantifier elimination. Specifically variables that occur only once can be removed in many cases by computing an expression that represents the effect computing a value for the eliminated occurrence.
The theory plugins for variable elimination are very partial and should be augmented by extensions, esp. for the case of bit-vectors where the invertibility conditions are thoroughly documented by Niemetz and Preiner.
the solve_eqs_tactic is to be replaced by a re-implementation that uses solve_eqs in the simplifiers directory.
The re-implementation should address efficiency issues with the previous code.
At this point it punts on low level proofs. The plan is to use coarser
dependency tracking instead of low level proofs for pre-processing. Dependencies can be converted into a proof hint representation that can be checked using a stronger checker.
simplifiers layer is a common substrate for global non-incremental and incremental processing.
The first two layers are new, but others are to be ported form tactics.
- bv::slice - rewrites equations to cut-dice-slice bit-vector extractions until they align. It creates opportunities for rewriting portions of bit-vectors to common sub-expressions, including values.
- euf::completion - generalizes the KB simplifcation from asserted formulas to use the E-graph to establish a global and order-independent canonization.
The interface dependent_expr_simplifier is amenable to forming tactics. Plugins for asserted-formulas is also possible but not yet realized.
- ensure mk_extract performs simplification to distribute over extract and removing extract if the range is the entire bit-vector
- ensure bool_rewriter simplifeis disjunctions when applicable.
move self-checking functionality to inside sat/smt so it can be used on-line and not just off-line.
when self-validation fails, use vs, not clause, to check. It allows self-validation without checking and maintaining RUP validation.
new options sat.smt.proof.check_rup, sat.smt.proof.check for online validation.
z3 sat.smt.proof.check=true sat.euf=true /v:1 sat.smt.proof.check_rup=true /st file.smt2 sat.smt.proof=p.smt2
EUF proofs are checked modulo union-find.
Equalities are added to to union-find if they are assumptions or if they can be derived using congruence closure. The congruence closure assumptions are added as proof-hints.
Note that this proof format does not track equality inferences, symmetry and transitivity. Instead they are handled by assuming a union-find based checker.
The bug was that axiom generation was not enabled on last_index, so no axioms got created to constrain last-index.
With default settings the solver is now very slow on this example. It is related to that the smallest size of a satisfying assignment is above 24. Pending a good heuristic to find initial seeds and increments for iterative deepening, I am adding another parameter smt.seq.min_unfolding that when set to 30 helps for this example.
The literal "emp" can be true in the current assignment, in which case the clause
cnt or emp or ~postf is true and does not contribute to propagation.
This saves, potentially, for generating lemmas for postf.
Add a lemma a = "" or |s| >= idx when a = tail(s, idx)
The lemma ensures that length bounding on s is enforced
(the branch that expands not-contains for long sequences s is closed).
* Make spacer_sem_matcher::reset() public
* Add .clang-format for src/muz/spacer
* Mark substitution::get_bindings() as const
* Fix in spacer_antiunify
* Various helper methods in spacer_util
Minor functions to compute number of free variables, detect presence of certain
sub-expressions, etc.
The diff is ugly because of clang-format
* Add spacer_cluster for clustering lemmas
A cluster of lemmas is a set of lemmas that are all instances of the same
pattern, where a pattern is a qff formula with free variables.
Currently, the instances are required to be explicit, that is, they are all
obtained by substituting concrete values (i.e., numbers) for free variables of
the pattern.
Lemmas are clustered in cluster_db in each predicate transformer.
* Integrate spacer_cluster into spacer_context
* Custom clang-format pragmas for spacer_context
spacer_context.(cpp|h) are large and have inconsistent formatting. Disable
clang-format for them until merge with main z3 branch and re-format.
* Computation of convex closure and matrix kernel
Various LA functions. The implementations are somewhat preliminary.
Convex closure is simplemented via syntactic convex closure procedure.
Kernel computation considers many common cases.
spacer_arith_kernel_sage implements kernel computation by call external
Sage binary. It is used only for debugging and experiments. There is no
link dependence on Sage. If desired, it can be removed.
* Add spacer_concretize
* Utility methods for spacer conjecture rule
* Add spacer_expand_bnd_generalizer
Generalizes arithmetic inequality literals of the form x <= c,
by changing constant c to other constants found in the problem.
* Add spacer_global_generalizer
Global generalizer checks every new lemma against a cluster
of previously learned lemmas, and, if possible, conjectures
a new pob, that, when blocked, generalizes multiple existing
lemmas.
* Remove fp.spacer.print_json option
The option is used to dump state of spacer into json for debugging.
It has been replaced by `fp.spacer.trace_file` that allows dumping an execution
of spacer. The json file can be reconstructed from the trace file elsewhere.
* Workaround for segfault in spacer_proof_utils
Issue #3 in hgvk94/z3
Segfault in some proof reduction. Avoid by bailing out on reduction.
* Revert bug for incomplete models
* Use local fresh variables in spacer_global_generalizer
* Cleanup of spacer_convex_closure
* Allow arbitrary expressions to name cols in convex_closure
* WIP: convex closure
* WIP: convex closure
* Fix bindings order in spacer_global_generalizer
The matcher creates substitution using std_order, which is
reverse of expected order (variable 0 is last). Adjust the code
appropriately for that.
* Increase verbosity level for smt_context stats
* Dead code in qe_mbp
* bug fixes in spacer_global_generalizer::subsumer
* Partially remove dependence of size of m_alphas
I want m_alphas to potentially be greater than currently used alpha variables.
This is helpful for reusing them across multiple calls to convex closure
* Subtle bug in kernel computation
Coefficient was being passed by reference and, therefore, was
being changed indirectly.
In the process, updated the code to be more generic to avoid rational
computation in the middle of matrix manipulation.
* another test for sparse_matrix_ops::kernel
* Implementation of matrix kernel using Fraction Free Elimination
Ensures that the kernel is int for int matrices. All divisions are exact.
* clang-format sparse_matrix_ops.h
* another implementation of ffe kernel in sparse_matrix_ops
* Re-do arith_kernel and convex_closure
* update spacer_global_generalization for new subsumer
* remove spacer.gg.use_sage parameter
* cleanup of spacer_global_generalizer
* Removed dependency on sage
* fix in spacer_convex_closure
* spacer_sem_matcher: consider an additional semantic matching
disabled until it is shown useful
* spacer_global_generalizer: improve do_conjecture
- if conjecture does not apply to pob, use lemma instead
- better normalization
- improve debug prints
* spacer_conjecture: formatting
* spacer_cluster: improve debug prints
* spacer_context: improve debug prints
* spacer_context: re-queue may pobs
enabled even if global re-queue is disabled
* spacer_cluster print formatting
* reset methods on pob
* cleanup of print and local variable names
* formatting
* reset generalization data once it has been used
* refactored extra pob creation during global guidance
* fix bug copying sparse matrix into spacer matrix
* bug fix in spacer_convex_closure
* formatting change in spacer_context
* spacer_cluster: get_min_lvl
chose level based on pob as well as lemmas
* spacer_context: add desired_level to pob
desired_level indicates at which level pob should be proved.
A pob will be pushed to desired_level if necessary
* spacer_context: renamed subsume stats
the name of success/failed was switched
* spacer_convex_closure: fix prototype of is_congruent_mod()
* spacer_convex_closure: hacks in infer_div_pred()
* spacer_util: do not expand literals with mod
By default, equality literal t=p is expanded into t<=p && t>=p
Disable the expansion in case t contains 'mod' operator since such
expansion is usually not helpful for divisibility
* spacer_util: rename m_util into m_arith
* spacer_util: cleanup normalize()
* spacer_util: formatting
* spacer_context: formatting cleanup on subsume and conjecture
* spacer_context: fix handling may pobs when abs_weakness is enabled
A pob might be undef, so weakness must be bumped up
* spacer_arith_kernel: enhance debug print
* spacer_global_generalizer: improve matching on conjecture
* spacer_global_generalizer: set desired level on conjecture pob
* spacer_global_generalizer: debug print
* spacer_global_generalizer: set min level on new pobs
the new level should not be higher than the pob that was generalized
* spacer_global_generalizer: do no re-create closed pobs
If a generalized pob exist and closed, do not re-create it.
* spacer_context: normalize twice
* spacer_context: forward propagate only same kind of pobs
* sketch of inductive generalizer
A better implementation of inductive generalizer that in addition to dropping
literals also attempts to weaken them.
Current implementation is a sketch to be extended based on examples/requirements.
* fix ordering in spacer_cluster_util
* fix resetting of substitution matcher in spacer_conjecture
Old code would forget to reset the substitution provided to the sem_matcher.
Thus, if the substitution was matched once (i.e., one literal of interest is
found), no other literal would be matched.
* add spacer_util is_normalized() method
used for debugging only
* simplify normalization of pob expressions
pob expressions are normalized to increase syntactic matching.
Some of the normalization rules seem out of place, so removing them for now.
* fix in spacer_global_generalizer
If conjecture fails, do not try other generalization strategies -- they will not apply.
* fix in spacer_context
do not check that may pob is blocked by existing lemmas.
It is likely to be blocked. Our goal is to block it again and generalize
to a new lemma.
This can be further improved by moving directly to generalization when pob is
blocked by existing lemmas...
Co-authored-by: hgvk94 <hgvk94@gmail.com>
Conditionals are used to guard unfolding of recursive functions.
This is, as shown in #6304, incompatible with the case where recursive functions are used inside if-then-else guards.
We address this by disabling if-conditions as guards if they contain a recursive definition.
The approach is simplistic: if a recursive function, defined prior (not mutually recursive) is used in a guard it should be fine and the condition can guard the current recursive unfolding.
The existing comment describes macros as "formulas of the form
`(forall X (= (f X) T[X]))` ... where `T[X]` does not contain `X`". This is
incorrect; of course the macros' definitions are allowed to be in terms of
the macros' arguments. The comment should say "...does not contain `f`" because
macros can't be recursive.
This commit overhauls the proof format (in development) for the new core.
NOTE: this functionality is work in progress with a long way to go.
It is shielded by the sat.euf option, which is off by default and in pre-release state.
It is too early to fuzz or use it. It is pushed into master to shed light on road-map for certifying inferences of sat.euf.
It retires the ad-hoc extension of DRUP used by the SAT solver.
Instead it relies on SMT with ad-hoc extensions for proof terms.
It adds the following commands (consumed by proof_cmds.cpp):
- assume - for input clauses
- learn - when a clause is learned (or redundant clause is added)
- del - when a clause is deleted.
The commands take a list of expressions of type Bool and the
last argument can optionally be of type Proof.
When the last argument is of type Proof it is provided as a hint
to justify the learned clause.
Proof hints can be checked using a self-contained proof
checker. The sat/smt/euf_proof_checker.h class provides
a plugin dispatcher for checkers.
It is instantiated with a checker for arithmetic lemmas,
so far for Farkas proofs.
Use example:
```
(set-option :sat.euf true)
(set-option :tactic.default_tactic smt)
(set-option :sat.smt.proof f.proof)
(declare-const x Int)
(declare-const y Int)
(declare-const z Int)
(declare-const u Int)
(assert (< x y))
(assert (< y z))
(assert (< z x))
(check-sat)
```
Run z3 on a file with above content.
Then run z3 on f.proof
```
(verified-smt)
(verified-smt)
(verified-smt)
(verified-farkas)
(verified-smt)
```
a recent opened and closed bug report was due to an error of taking bit-wise or between two bit-vectors of different size. The error message was not understood by the user. Adding a little extra generic information to see if it helps.
- add option smt.bv.reduce_size.
- it allows to apply incremental pre-processing of bit-vectors by identifying ranges that are known to be constant.
This rewrite is beneficial, for instance, when bit-vectors are constrained to have many high-level bits set to 0.
This update allows the python bindings for user-propagator to handle functions that are declared to be registered with the user propagator plugin. It fixes a bug in UserPropagateBase.add to allow registering terms dynamically during search.
It also fixes a bug in theory_user_propagate as scopes were not fully pushed when the solver gets the callbacks for new equalities and new disequalities.
It also adds equality and disequality interfaces to the sat/smt solver version (which isn't being exercised in earnest yet)
remove option for uzers (users who are in reality fuzzers) to toggle flat option. The legacy arithmetic solver bakes in assumptions about flat format so it isn't helpful to expose this to fuzzers, I mean uzers.
remaining perf bug is dealing with very large bit-widths. mod 2^n should be computed natively based on n instead of 2^n because we pre-populate an array with all values up to n. Suppose n is 10000, the array has size 10000.
tactic/lia2card shows a huge slowdown because the same replace function is called on thousands of assertions. Each time the cache gets reset with thousands of entries - they are all the same.
So don't reset the cache just because... Instead reset the cache if m_refs grows large.
add API to define forward reference to recursively defined datatype.
The forward reference should be used only when passed to constructor declarations that are used in a datatype definition (Z3_mk_datatypes). The call to Z3_mk_datatypes ensures that the forward reference can be resolved with respect to constructors.
* Fixed problem with registering bitvector functions
* Added rewriting distinct with bitvectors to false if bit-size is too low
* Removed debug output
* Incorporated Nikolaj's comments
* Simplifications
