2 commits

Author	SHA1	Message	Date
Margus Veanes	683cb4ec03	seq_monadic: add light Antimirov cofactor mode (#10323) ... Light-weight Antimirov cofactors for seq_monadic ================================================ Overview -------- The seq_monadic solver explores symbolic regex derivatives when computing live states and product transitions. Its original transition representation used Brzozowski cofactors. This change adds a light-weight Antimirov representation. It first computes the existing Brzozowski cofactors and then decomposes targets whose outer shape is s1 | ... | sn or (s1 | ... | sn) . tail into separate transitions. Concatenations are maintained in right-associative form, so a distributable union occurs as the head of the concatenation. Targets are reconstructed with mk_regex_concat to preserve normalization. After decomposition, transitions with the same target are merged by disjoining their guards. The existing path-aware cofactor traversal and range predicates are therefore retained. Example ------- For r = .* a .{k} the Brzozowski cofactors have the shape [a, r | .{k}] [^a, r] The light-weight Antimirov transformation produces [., r] [a, .{k}] This preserves the language while avoiding the deterministic subset states that grow exponentially on this family. Modes ----- seq_monadic exposes two transition modes: light_antimirov default brzozowski retained as an explicit option The implementation is seq_rewriter::light_ant_derivative_cofactors. Correctness ----------- The seq_monadic unit tests run in both modes. They cover character and generic element sequences, multiple and repeated variables, variable constraints, bounded loops, conjunctions of memberships, and witness construction. A focused test checks the cofactor transformation above. The complete 94-test unit suite used during evaluation passed. The benchmark harness is not registered as a normal unit test; after detaching it, all 93 registered tests pass. Benchmark evaluation -------------------- The final optimized comparison used all 1,545 SMT2 files under C:\git\bench\inputs\regexes. Each file was run in a separate process with a 15-second timeout. There were 1,513 cases where both modes completed without a process failure or timeout. Brzozowski Light-Ant paired solver time 94.98 s 63.26 s median solver time 1.734 ms 0.710 ms derivative calls 5.70 M 3.39 M cofactors 13.33 M 6.99 M live states 2.74 M 0.44 M product states 652.8 K 642.8 K Light-Ant reduced paired solver time by 33.4%, derivative calls by 40.5%, cofactors by 47.5%, and live states by 84.0%. It was faster on 1,091 cases; Brzozowski was faster on 421 cases. Light-Ant changed 131 Brzozowski undef results to sat. There were no reverse verdict changes and no mismatches against known sat/unsat statuses. Both modes had two timeouts. Process failures decreased from 30 to 27. By corpus, paired solver time improved by 39.1% on ClemensRegex and by 11.9% on MargusRegex. Alternatives considered ----------------------- Direct use of full Antimirov derivatives was also evaluated. It introduced large performance outliers, particularly around intersections, and produced additional undef results and timeouts. Disabling intersection-over-union distribution improved some of these cases but remained slower and less robust than the light-weight transformation. The direct full-Ant mode was therefore removed from this change. Copilot-Session: a2ce3573-4e15-4a4a-afb5-21e3cb04e4a2	2026-07-31 10:34:34 -07:00
Margus Veanes	0972dd2141	seq_monadic: self-contained monadic-decomposition regex membership solver (generic elements, witnesses, Boolean combinations) (#10296) ... ## Summary Adds `seq_monadic` (`src/ast/rewriter/seq_monadic.{h,cpp}`), a self-contained, rewriter-level decision procedure for regex membership of a term that is a concatenation of sequence variables and constant elements — e.g. `x·a·x ∈ R`, including repeated and multiple variables. It uses a whole-language *monadic decomposition* plus automaton product-reachability; it is minterm-free and does **not** use Nielsen word-equation splitting or `seq_split`. The component is **purely additive**: it is not wired into any solver path, so default behavior is unchanged. It ships with a unit test and an opt-in benchmark harness that is inert unless `Z3_SEQ_BENCH_DIR` is set. ## What it does - `x·u ∈ R ⇔ ⋁_q ( x reaches q in A_R ∧ u ∈ q )` over the derivative automaton; `reach(q)` is never materialized as a regex (avoids the state-elimination blowup). A variable's constraint is decided by a lazy product-reachability search over tuples of derivative states, with transitions = the product of `brz_derivative_cofactors` branches and pairwise-conjoined `seq::range_predicate` guards. - **Generic in the element sort**: characters use the exact `range_predicate` algebra; any other element sort uses a candidate-basis over the element values the guards mention (sound and complete for the `{true,false,=,<=,and,or,not}` guard grammar the derivatives emit). - **Concrete witnesses**: on sat it reconstructs a witness value (a sequence of concrete elements, not predicates) per variable from the accepting product path. - **Boolean combinations**: `solve_and` decides a conjunction of memberships jointly, so a variable shared across memberships is constrained consistently. This is the natural extension since `¬(t∈R) ≡ t∈~R`, `∨` = union of DNFs, and `∧` = product of DNFs. Also de-duplicates the char-guard → `range_predicate` translator into a single public `seq::guard_to_range_predicate` in `seq_range_collapse` (it was previously duplicated there and in `seq_monadic`). ## Testing - `tst_seq_monadic`: single / multiple / repeated variables, nested complement, bounded loops, per-variable constraints, a generic `(Seq Int)` section, witness verification (substitute the model back and re-decide membership), and `solve_and` cases that are individually sat but jointly unsat. - Full unit suite `test-z3 /a` passes (93/93). ## Evaluation (offline harness, not part of CI) On a regex-membership benchmark corpus, restricted to files carrying a genuine `(set-info :status)`, the solver decides 318 and 316 are correct (99.4%); the only 2 disagreements are a length limitation (`|x|=2k`) that is outside the membership fragment. ## Known limitations / follow-ups - Pathological deeply-nested, high-multiplicity regexes can overflow the recursive derivative stack; a recursion-depth guard to degrade to `unknown` is a natural follow-up. - Out-of-fragment constraints (word equations, length / Parikh) are not handled, by design — this decides regex membership only. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Nikolaj Bjorner <nikolaj@cs.stanford.edu> Copilot-Session: 916db256-43c6-4067-b6f4-fa8d2cf2f37f	2026-07-29 15:16:54 -07:00