Per-variable extra constraints are now expressed as additional (var in R')
memberships passed to solve_and, so the var_extra parameter is dropped from
solve/solve_and and the internal decide_dnf. Tests updated to route extra
constraints through solve_and.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 57b9b87e-950a-49ea-bbb3-ed585646a5a9
Extract the guard_set class (element-value set over derivative cofactor
guards) from the anonymous namespace in seq_monadic.cpp into a
self-contained guard_set.h / guard_set.cpp pair.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 57b9b87e-950a-49ea-bbb3-ed585646a5a9
## Summary
Pin constant elements extracted while parsing sequence terms so they
remain alive throughout monadic decomposition.
Without these references, character ASTs could be reclaimed while still
stored as raw pointers in decomposition atoms, causing assertion
failures, fast-fail exits, and access violations during derivative
construction.
The fix eliminates 26 previously observed process failures in the regex
benchmark corpus. No stack-depth bailout is needed for these failures.
Light-weight Antimirov cofactors for seq_monadic
================================================
Overview
--------
The seq_monadic solver explores symbolic regex derivatives when
computing live
states and product transitions. Its original transition representation
used
Brzozowski cofactors.
This change adds a light-weight Antimirov representation. It first
computes the
existing Brzozowski cofactors and then decomposes targets whose outer
shape is
s1 | ... | sn
or
(s1 | ... | sn) . tail
into separate transitions. Concatenations are maintained in
right-associative
form, so a distributable union occurs as the head of the concatenation.
Targets are reconstructed with mk_regex_concat to preserve
normalization.
After decomposition, transitions with the same target are merged by
disjoining
their guards. The existing path-aware cofactor traversal and range
predicates
are therefore retained.
Example
-------
For
r = .* a .{k}
the Brzozowski cofactors have the shape
[a, r | .{k}]
[^a, r]
The light-weight Antimirov transformation produces
[., r]
[a, .{k}]
This preserves the language while avoiding the deterministic subset
states
that grow exponentially on this family.
Modes
-----
seq_monadic exposes two transition modes:
light_antimirov default
brzozowski retained as an explicit option
The implementation is seq_rewriter::light_ant_derivative_cofactors.
Correctness
-----------
The seq_monadic unit tests run in both modes. They cover character and
generic
element sequences, multiple and repeated variables, variable
constraints,
bounded loops, conjunctions of memberships, and witness construction. A
focused test checks the cofactor transformation above. The complete
94-test
unit suite used during evaluation passed. The benchmark harness is not
registered as a normal unit test; after detaching it, all 93 registered
tests
pass.
Benchmark evaluation
--------------------
The final optimized comparison used all 1,545 SMT2 files under
C:\git\bench\inputs\regexes. Each file was run in a separate process
with a
15-second timeout. There were 1,513 cases where both modes completed
without a
process failure or timeout.
Brzozowski Light-Ant
paired solver time 94.98 s 63.26 s
median solver time 1.734 ms 0.710 ms
derivative calls 5.70 M 3.39 M
cofactors 13.33 M 6.99 M
live states 2.74 M 0.44 M
product states 652.8 K 642.8 K
Light-Ant reduced paired solver time by 33.4%, derivative calls by
40.5%,
cofactors by 47.5%, and live states by 84.0%. It was faster on 1,091
cases;
Brzozowski was faster on 421 cases.
Light-Ant changed 131 Brzozowski undef results to sat. There were no
reverse
verdict changes and no mismatches against known sat/unsat statuses. Both
modes
had two timeouts. Process failures decreased from 30 to 27.
By corpus, paired solver time improved by 39.1% on ClemensRegex and by
11.9% on
MargusRegex.
Alternatives considered
-----------------------
Direct use of full Antimirov derivatives was also evaluated. It
introduced
large performance outliers, particularly around intersections, and
produced
additional undef results and timeouts. Disabling intersection-over-union
distribution improved some of these cases but remained slower and less
robust
than the light-weight transformation. The direct full-Ant mode was
therefore
removed from this change.
Copilot-Session: a2ce3573-4e15-4a4a-afb5-21e3cb04e4a2
## Summary
Adds `seq_monadic` (`src/ast/rewriter/seq_monadic.{h,cpp}`), a
self-contained,
rewriter-level decision procedure for regex membership of a term that is
a
concatenation of sequence variables and constant elements — e.g. `x·a·x
∈ R`,
including repeated and multiple variables. It uses a whole-language
*monadic
decomposition* plus automaton product-reachability; it is minterm-free
and does
**not** use Nielsen word-equation splitting or `seq_split`.
The component is **purely additive**: it is not wired into any solver
path, so
default behavior is unchanged. It ships with a unit test and an opt-in
benchmark
harness that is inert unless `Z3_SEQ_BENCH_DIR` is set.
## What it does
- `x·u ∈ R ⇔ ⋁_q ( x reaches q in A_R ∧ u ∈ q )` over the derivative
automaton; `reach(q)` is never materialized as a regex (avoids the
state-elimination blowup). A variable's constraint is decided by a lazy
product-reachability search over tuples of derivative states, with
transitions
= the product of `brz_derivative_cofactors` branches and
pairwise-conjoined
`seq::range_predicate` guards.
- **Generic in the element sort**: characters use the exact
`range_predicate`
algebra; any other element sort uses a candidate-basis over the element
values
the guards mention (sound and complete for the
`{true,false,=,<=,and,or,not}`
guard grammar the derivatives emit).
- **Concrete witnesses**: on sat it reconstructs a witness value (a
sequence of
concrete elements, not predicates) per variable from the accepting
product path.
- **Boolean combinations**: `solve_and` decides a conjunction of
memberships
jointly, so a variable shared across memberships is constrained
consistently.
This is the natural extension since `¬(t∈R) ≡ t∈~R`, `∨` = union of
DNFs, and
`∧` = product of DNFs.
Also de-duplicates the char-guard → `range_predicate` translator into a
single
public `seq::guard_to_range_predicate` in `seq_range_collapse` (it was
previously
duplicated there and in `seq_monadic`).
## Testing
- `tst_seq_monadic`: single / multiple / repeated variables, nested
complement,
bounded loops, per-variable constraints, a generic `(Seq Int)` section,
witness
verification (substitute the model back and re-decide membership), and
`solve_and` cases that are individually sat but jointly unsat.
- Full unit suite `test-z3 /a` passes (93/93).
## Evaluation (offline harness, not part of CI)
On a regex-membership benchmark corpus, restricted to files carrying a
genuine
`(set-info :status)`, the solver decides 318 and 316 are correct
(99.4%); the
only 2 disagreements are a length limitation (`|x|=2k`) that is outside
the
membership fragment.
## Known limitations / follow-ups
- Pathological deeply-nested, high-multiplicity regexes can overflow the
recursive derivative stack; a recursion-depth guard to degrade to
`unknown` is
a natural follow-up.
- Out-of-fragment constraints (word equations, length / Parikh) are not
handled,
by design — this decides regex membership only.
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Nikolaj Bjorner <nikolaj@cs.stanford.edu>
Copilot-Session: 916db256-43c6-4067-b6f4-fa8d2cf2f37f
