mirrors/z3
3
0
Fork 0
mirror of https://github.com/Z3Prover/z3 synced 2026-08-03 04:33:28 +00:00
Code Activity

seq_monadic: self-contained monadic-decomposition regex membership solver (generic elements, witnesses, Boolean combinations) (#10296)

Browse source 
## Summary

Adds `seq_monadic` (`src/ast/rewriter/seq_monadic.{h,cpp}`), a
self-contained,
rewriter-level decision procedure for regex membership of a term that is
a
concatenation of sequence variables and constant elements — e.g. `x·a·x
∈ R`,
including repeated and multiple variables. It uses a whole-language
*monadic
decomposition* plus automaton product-reachability; it is minterm-free
and does
**not** use Nielsen word-equation splitting or `seq_split`.

The component is **purely additive**: it is not wired into any solver
path, so
default behavior is unchanged. It ships with a unit test and an opt-in
benchmark
harness that is inert unless `Z3_SEQ_BENCH_DIR` is set.

## What it does

- `x·u ∈ R  ⇔  ⋁_q ( x reaches q in A_R  ∧  u ∈ q )` over the derivative
  automaton; `reach(q)` is never materialized as a regex (avoids the
state-elimination blowup). A variable's constraint is decided by a lazy
product-reachability search over tuples of derivative states, with
transitions
= the product of `brz_derivative_cofactors` branches and
pairwise-conjoined
  `seq::range_predicate` guards.
- **Generic in the element sort**: characters use the exact
`range_predicate`
algebra; any other element sort uses a candidate-basis over the element
values
the guards mention (sound and complete for the
`{true,false,=,<=,and,or,not}`
  guard grammar the derivatives emit).
- **Concrete witnesses**: on sat it reconstructs a witness value (a
sequence of
concrete elements, not predicates) per variable from the accepting
product path.
- **Boolean combinations**: `solve_and` decides a conjunction of
memberships
jointly, so a variable shared across memberships is constrained
consistently.
This is the natural extension since `¬(t∈R) ≡ t∈~R`, `∨` = union of
DNFs, and
  `∧` = product of DNFs.

Also de-duplicates the char-guard → `range_predicate` translator into a
single
public `seq::guard_to_range_predicate` in `seq_range_collapse` (it was
previously
duplicated there and in `seq_monadic`).

## Testing

- `tst_seq_monadic`: single / multiple / repeated variables, nested
complement,
bounded loops, per-variable constraints, a generic `(Seq Int)` section,
witness
  verification (substitute the model back and re-decide membership), and
  `solve_and` cases that are individually sat but jointly unsat.
- Full unit suite `test-z3 /a` passes (93/93).

## Evaluation (offline harness, not part of CI)

On a regex-membership benchmark corpus, restricted to files carrying a
genuine
`(set-info :status)`, the solver decides 318 and 316 are correct
(99.4%); the
only 2 disagreements are a length limitation (`|x|=2k`) that is outside
the
membership fragment.

## Known limitations / follow-ups

- Pathological deeply-nested, high-multiplicity regexes can overflow the
recursive derivative stack; a recursion-depth guard to degrade to
`unknown` is
  a natural follow-up.
- Out-of-fragment constraints (word equations, length / Parikh) are not
handled,
  by design — this decides regex membership only.

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Nikolaj Bjorner <nikolaj@cs.stanford.edu>
Copilot-Session: 916db256-43c6-4067-b6f4-fa8d2cf2f37f
This commit is contained in:
Margus Veanes 2026-07-29 15:16:54 -07:00 • committed by GitHub
parent 3c685d368b
commit 0972dd2141
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
8 changed files with 1118 additions and 36 deletions
Download patch file Download diff file Expand all files Collapse all files

1
src/ast/rewriter/CMakeLists.txt
View file

@ -43,6 +43,7 @@ z3_add_component(rewriter
seq_subset.cpp
seq_split.cpp
seq_derive.cpp
seq_monadic.cpp
seq_range_collapse.cpp
seq_range_predicate.cpp
seq_rewriter.cpp

609
src/ast/rewriter/seq_monadic.cpp Normal file
View file

@ -0,0 +1,609 @@
/*++
Copyright (c) 2026 Microsoft Corporation
Module Name:
seq_monadic.cpp
Abstract:
Whole-language monadic decomposition for regex membership. See seq_monadic.h.
Automaton-based (product-reachability); reach(q) is never materialized as a regex.
Generic in the element sort. The decomposition, liveness and product-reachability
are element-agnostic; only the *guard algebra* over the derivative cofactor guards
depends on the element sort. For the character sort it is the exact, compact
seq::range_predicate; for any other element sort it is a candidate-basis over the
element values mentioned by the guards (sound and complete for the
{true,false,=,<=,and,or,not} grammar the derivatives emit). The same guard algebra
yields the concrete element used to build a witness sequence.
Author:
Nikolaj Bjorner / Margus Veanes 2026
--*/
#include "ast/rewriter/seq_monadic.h"
#include "ast/rewriter/seq_range_collapse.h"
#include "ast/arith_decl_plugin.h"
#include "ast/bv_decl_plugin.h"
#include <set>
#include <vector>
#include <map>
#include <tuple>
#include <functional>
#include <algorithm>
namespace {
// A conjunction of derivative cofactor guards over the element variable v0 = (:var 0),
// interpreted as the set of element values satisfying it. Two representations by
// element sort:
// * character sort: the exact, compact seq::range_predicate.
// * any other sort: the guard predicate kept symbolically and decided by a candidate
// basis -- the element values mentioned in the guards, plus one fresh value. This
// is sound and complete for the {true,false,=,<=,and,or,not} grammar the derivatives
// emit (over a general element sort only equalities appear).
class guard_set {
ast_manager& m;
seq_util& u;
sort* m_sort;
expr* m_v0;
bool m_is_char;
bool m_ok = true; // false: an unsupported guard was conjoined
seq::range_predicate m_rp; // char representation
expr_ref m_guard; // generic representation (conjunction over v0)
// ---- generic path: candidate basis ----
// element values compared to v0 by the equalities in `g`.
void collect_consts(expr* g, ptr_vector<expr>& out) const {
expr* a = nullptr, * b = nullptr;
if (m.is_and(g) || m.is_or(g)) {
for (expr* arg : *to_app(g)) collect_consts(arg, out);
return;
}
if (m.is_not(g, a)) { collect_consts(a, out); return; }
if (m.is_eq(g, a, b)) {
if (a == m_v0 && b != m_v0) out.push_back(b);
else if (b == m_v0 && a != m_v0) out.push_back(a);
}
}
// evaluate `g` at v0 := cand ; l_undef on a construct outside the grammar.
lbool eval_at(expr* g, expr* cand) const {
expr* a = nullptr, * b = nullptr;
if (m.is_true(g)) return l_true;
if (m.is_false(g)) return l_false;
if (m.is_not(g, a)) {
lbool r = eval_at(a, cand);
return r == l_undef ? l_undef : (r == l_true ? l_false : l_true);
}
if (m.is_and(g)) {
lbool r = l_true;
for (expr* arg : *to_app(g)) {
lbool e = eval_at(arg, cand);
if (e == l_false) return l_false;
if (e == l_undef) r = l_undef;
}
return r;
}
if (m.is_or(g)) {
lbool r = l_false;
for (expr* arg : *to_app(g)) {
lbool e = eval_at(arg, cand);
if (e == l_true) return l_true;
if (e == l_undef) r = l_undef;
}
return r;
}
if (m.is_eq(g, a, b)) {
expr* other = (a == m_v0) ? b : (b == m_v0 ? a : nullptr);
if (!other) return l_undef;
if (other == m_v0) return l_true;
return (cand == other) ? l_true : l_false; // canonical values: identity == equality
}
return l_undef;
}
// a value of m_sort distinct from every element of `consts`, if one can be built.
bool mk_fresh(ptr_vector<expr> const& consts, expr_ref& out) const {
if (m.is_bool(m_sort)) {
bool hasT = false, hasF = false;
for (expr* c : consts) { if (m.is_true(c)) hasT = true; else if (m.is_false(c)) hasF = true; }
if (!hasT) { out = m.mk_true(); return true; }
if (!hasF) { out = m.mk_false(); return true; }
return false;
}
arith_util a(m);
if (a.is_int_real(m_sort)) {
rational mx(0); bool any = false;
for (expr* c : consts) {
rational v;
if (a.is_numeral(c, v)) { if (!any || v > mx) mx = v; any = true; }
}
out = a.mk_numeral(any ? mx + rational(1) : rational(0), a.is_int(m_sort));
return true;
}
bv_util bv(m);
if (bv.is_bv_sort(m_sort)) {
unsigned sz = bv.get_bv_size(m_sort);
for (unsigned k = 0; k <= consts.size(); ++k) {
rational kv(k);
bool clash = false;
for (expr* c : consts) {
rational v; unsigned bsz = 0;
if (bv.is_numeral(c, v, bsz) && v == kv) { clash = true; break; }
}
if (!clash) { out = bv.mk_numeral(kv, sz); return true; }
}
return false;
}
return false;
}
lbool generic_eval(expr_ref* witness) const {
ptr_vector<expr> consts;
collect_consts(m_guard, consts);
bool saw_undef = false;
for (expr* c : consts) {
lbool r = eval_at(m_guard, c);
if (r == l_true) { if (witness) *witness = expr_ref(c, m); return l_true; }
if (r == l_undef) saw_undef = true;
}
expr_ref fresh(m);
if (mk_fresh(consts, fresh)) {
lbool r = eval_at(m_guard, fresh);
if (r == l_true) { if (witness) *witness = fresh; return l_true; }
if (r == l_undef) saw_undef = true;
}
else
saw_undef = true; // the "distinct from all mentioned values" region is untested
return saw_undef ? l_undef : l_false;
}
public:
guard_set(ast_manager& _m, seq_util& _u, sort* elem_sort, expr* v0)
: m(_m), u(_u), m_sort(elem_sort), m_v0(v0),
m_is_char(_u.is_char(elem_sort)),
m_rp(_u.max_char()), m_guard(_m) {
if (m_is_char) m_rp = seq::range_predicate::top(u.max_char());
else m_guard = m.mk_true();
}
bool ok() const { return m_ok; }
// AND in a cofactor guard g (a Boolean over v0).
void conjoin(expr* g) {
if (!m_ok) return;
if (m_is_char) {
seq::range_predicate s(u.max_char());
if (!seq::guard_to_range_predicate(u, m_v0, g, s)) { m_ok = false; return; }
m_rp = m_rp & s;
}
else
m_guard = m.mk_and(m_guard, g);
}
// l_false = empty, l_true = non-empty (sets *witness if non-null to a concrete
// element of the set), l_undef = unknown / unsupported guard.
lbool eval(expr_ref* witness) const {
if (!m_ok) return l_undef;
if (m_is_char) {
if (m_rp.is_empty()) return l_false;
if (witness) *witness = expr_ref(u.mk_char(m_rp[0].first), m);
return l_true;
}
return generic_eval(witness);
}
};
}
expr_ref seq_monadic::der_elem(expr* r, expr* elem) {
expr_ref d = m_rw.mk_derivative(elem, r); // mk_derivative(element, regex)
// Normalize: for a general element sort the derivative by a non-matching constant can
// leave a ground guard (e.g. (= 1 2)) unfolded; simplifying collapses such dead
// branches to re.empty so nullability/emptiness stay decidable.
expr_ref d2(m);
m_thrw(d, d2);
return d2;
}
void seq_monadic::live_states(expr* R, ptr_vector<expr>& out, bool& ok) {
ok = true;
obj_map<expr, unsigned> id;
expr_ref_vector states(m);
vector<svector<unsigned>> succ;
bool_vector maybe_null;
auto intern = [&](expr* s) -> unsigned {
unsigned k;
if (id.find(s, k)) return k;
k = states.size();
id.insert(s, k);
states.push_back(s);
succ.push_back(svector<unsigned>());
expr_ref nb = m_rw.is_nullable(s);
maybe_null.push_back(!m.is_false(nb)); // unknown nullability => keep (conservative)
return k;
};
intern(R);
const unsigned STATE_CAP = 1u << 12;
for (unsigned i = 0; i < states.size(); ++i) {
if (states.size() > STATE_CAP || !m.inc()) { ok = false; return; }
expr_ref_pair_vector cof(m);
m_rw.brz_derivative_cofactors(states.get(i), cof);
for (auto const& [g, t] : cof) {
if (re().is_empty(t)) continue;
unsigned k = intern(t); // MUST precede succ[i] indexing: intern may
succ[i].push_back(k); // grow (realloc) succ, invalidating succ[i]&
}
}
unsigned n = states.size();
bool_vector live;
live.resize(n, false);
for (unsigned i = 0; i < n; ++i)
live[i] = maybe_null[i];
for (bool ch = true; ch; ) {
ch = false;
for (unsigned i = 0; i < n; ++i)
if (!live[i])
for (unsigned j : succ[i])
if (live[j]) { live[i] = true; ch = true; break; }
}
for (unsigned i = 0; i < n; ++i)
if (live[i]) { out.push_back(states.get(i)); m_pin.push_back(states.get(i)); }
}
lbool seq_monadic::product_nonempty(svector<component> const& comps, expr_ref* witness_word) {
unsigned n = comps.size();
if (n == 0) {
if (witness_word)
*witness_word = expr_ref(u().str.mk_empty(m_seq_sort), m);
return l_true;
}
expr_ref var0(m.mk_var(0, m_elem_sort), m); // the element variable the guards range over
svector<expr*> start;
for (auto const& c : comps)
start.push_back(c.state);
auto id_key = [&](svector<expr*> const& st) {
std::vector<unsigned> k;
k.reserve(st.size());
for (expr* e : st) k.push_back(e->get_id());
return k;
};
typedef std::vector<unsigned> key;
bool undecided = false;
auto is_accept = [&](svector<expr*> const& st) -> bool {
for (unsigned i = 0; i < n; ++i) {
if (comps[i].target) {
if (st[i] != comps[i].target) return false;
}
else {
expr_ref nb = m_rw.is_nullable(st[i]);
if (m.is_true(nb)) continue;
if (m.is_false(nb)) return false;
undecided = true; return false;
}
}
return true;
};
std::set<key> visited;
std::vector<svector<expr*>> work;
// tree of first-discovery edges for witness reconstruction (only built when a
// witness is requested): child-key -> (parent-key, element read on the edge).
std::map<key, std::pair<key, expr*>> parent;
key start_key = id_key(start);
auto reconstruct = [&](key end_key) -> expr_ref {
ptr_vector<expr> elems; // collected in accept..start order
key k = end_key;
while (k != start_key) {
auto it = parent.find(k);
if (it == parent.end()) break; // safety (should not happen)
elems.push_back(it->second.second);
k = it->second.first;
}
expr_ref_vector es(m); // start..accept order
for (unsigned idx = elems.size(); idx-- > 0; )
es.push_back(u().str.mk_unit(elems[idx]));
if (es.empty())
return expr_ref(u().str.mk_empty(m_seq_sort), m);
return expr_ref(u().str.mk_concat(es.size(), es.data(), m_seq_sort), m);
};
work.push_back(start);
visited.insert(start_key);
while (!work.empty()) {
if (m_budget == 0) { m_giveup = true; return l_undef; }
--m_budget;
if (!m.inc())
return l_undef;
svector<expr*> st = work.back();
work.pop_back();
if (is_accept(st)) {
if (witness_word)
*witness_word = reconstruct(id_key(st));
return l_true;
}
if (undecided)
return l_undef;
// per-component cofactor branches (target, guard); pin both, they outlive `cof`.
std::vector<std::vector<std::pair<expr*, expr*>>> branches(n);
for (unsigned i = 0; i < n; ++i) {
expr_ref_pair_vector cof(m);
m_rw.brz_derivative_cofactors(st[i], cof);
for (auto const& [g, t] : cof) {
if (re().is_empty(t)) continue;
m_pin.push_back(t);
m_pin.push_back(g);
branches[i].push_back(std::make_pair((expr*) t, (expr*) g));
}
}
// joint transitions = cartesian product of the branches with the guards
// conjoined; prune as soon as the accumulated guard is empty, bail on unknown.
svector<expr*> cur;
cur.resize(n);
key st_key = id_key(st);
bool bail = false;
std::function<void(unsigned, guard_set const&)> rec =
[&](unsigned i, guard_set const& acc) {
if (bail) return;
if (i == n) {
key ck = id_key(cur);
if (visited.find(ck) == visited.end()) {
visited.insert(ck);
if (witness_word) {
expr_ref e(m);
if (acc.eval(&e) == l_true) {
m_pin.push_back(e);
parent[ck] = std::make_pair(st_key, e.get());
}
}
work.push_back(cur);
}
return;
}
for (auto const& pr : branches[i]) {
guard_set nacc = acc;
nacc.conjoin(pr.second);
lbool ne = nacc.eval(nullptr);
if (ne == l_undef) { bail = true; return; } // non-range / unknown guard
if (ne == l_false) continue; // empty joint guard: prune
cur[i] = pr.first;
rec(i + 1, nacc);
if (bail) return;
}
};
guard_set top(m, u(), m_elem_sort, var0);
rec(0, top);
if (bail)
return l_undef;
}
return l_false;
}
bool seq_monadic::parse_term(expr* t, svector<atom>& atoms, expr*& the_var) {
if (u().str.is_concat(t)) {
app* a = to_app(t);
for (unsigned i = 0; i < a->get_num_args(); ++i)
if (!parse_term(a->get_arg(i), atoms, the_var))
return false;
return true;
}
if (u().str.is_empty(t))
return true; // epsilon: contributes nothing
zstring s;
if (u().str.is_string(t, s)) {
for (unsigned i = 0; i < s.length(); ++i)
atoms.push_back(atom{ false, nullptr, u().str.mk_char(s, i) });
return true;
}
if (u().str.is_unit(t)) { // seq.unit of a constant element
expr* elem = to_app(t)->get_arg(0);
if (m.is_value(elem)) {
atoms.push_back(atom{ false, nullptr, elem });
return true;
}
return false; // symbolic (non-constant) unit: unsupported
}
// uninterpreted 0-ary constant of sequence sort => a sequence variable
if (is_app(t) && to_app(t)->get_num_args() == 0 &&
to_app(t)->get_family_id() == null_family_id) {
the_var = t; // mark that at least one variable occurs
atoms.push_back(atom{ true, t, nullptr });
return true;
}
return false;
}
void seq_monadic::decompose(svector<atom> const& atoms, unsigned i, expr* R,
vector<disjunct>& out, bool& ok) {
if (!ok)
return;
if (m_giveup) { ok = false; return; }
m_pin.push_back(R);
if (i == atoms.size()) {
expr_ref nb = m_rw.is_nullable(R);
if (m.is_true(nb))
out.push_back(disjunct()); // empty conjunction = true
else if (!m.is_false(nb))
ok = false; // undecidable nullability => bail
return;
}
atom const& a = atoms[i];
if (!a.is_var) {
expr_ref d = der_elem(R, a.elem);
decompose(atoms, i + 1, d, out, ok);
return;
}
if (i + 1 == atoms.size()) { // last atom: membership component a.var in R
disjunct D;
D.push_back(component{ a.var, R, nullptr });
out.push_back(D);
return;
}
// a variable with a non-empty rest: split over the live states q of R (midpoints)
ptr_vector<expr> Q;
live_states(R, Q, ok);
if (!ok)
return;
const unsigned DISJUNCT_CAP = 1u << 13;
for (expr* q : Q) {
vector<disjunct> sub;
decompose(atoms, i + 1, q, sub, ok);
if (!ok)
return;
for (disjunct const& sd : sub) {
if (out.size() > DISJUNCT_CAP || m_budget == 0) { m_giveup = true; ok = false; return; }
--m_budget;
disjunct D(sd);
D.push_back(component{ a.var, R, q }); // reach component: a.var drives R -> q
out.push_back(D);
}
}
simplify_dnf(out);
}
void seq_monadic::simplify_dnf(vector<disjunct>& dnf) {
std::set<std::vector<std::tuple<unsigned, unsigned, unsigned>>> seen;
vector<disjunct> result;
for (disjunct const& D : dnf) {
bool dead = false;
for (auto const& c : D)
if (re().is_empty(c.state)) { dead = true; break; }
if (dead)
continue;
std::vector<std::tuple<unsigned, unsigned, unsigned>> sig;
sig.reserve(D.size());
for (auto const& c : D)
sig.push_back(std::make_tuple(c.var->get_id(), c.state->get_id(),
c.target ? c.target->get_id() : UINT_MAX));
std::sort(sig.begin(), sig.end());
if (seen.insert(sig).second)
result.push_back(D);
}
dnf.swap(result);
}
lbool seq_monadic::solve(expr* term, expr* R) {
obj_map<expr, expr*> none;
return solve(term, R, none, nullptr);
}
lbool seq_monadic::solve(expr* term, expr* R, obj_map<expr, expr*> const& var_extra) {
return solve(term, R, var_extra, nullptr);
}
bool seq_monadic::build_membership_dnf(expr* term, expr* R, vector<disjunct>& dnf) {
if (!u().is_re(R, m_seq_sort))
return false;
if (!u().is_seq(m_seq_sort, m_elem_sort))
return false;
svector<atom> atoms;
expr* the_var = nullptr;
if (!parse_term(term, atoms, the_var))
return false;
if (!the_var)
return false; // no variable: ground membership, not our case
m_pin.push_back(R);
bool ok = true;
decompose(atoms, 0, R, dnf, ok);
return ok;
}
lbool seq_monadic::decide_dnf(vector<disjunct> const& dnf, obj_map<expr, expr*> const& var_extra,
obj_map<expr, expr*>* model) {
bool any_undef = false;
for (disjunct const& D : dnf) {
// group components by variable, add the extra per-variable constraints
obj_map<expr, unsigned> idx;
vector<svector<component>> groups;
ptr_vector<expr> group_var;
auto bucket = [&](expr* v) -> unsigned {
unsigned gi;
if (idx.find(v, gi)) return gi;
gi = groups.size(); idx.insert(v, gi);
groups.push_back(svector<component>());
group_var.push_back(v);
return gi;
};
for (auto const& c : D)
groups[bucket(c.var)].push_back(c);
for (auto const& kv : var_extra)
groups[bucket(kv.m_key)].push_back(component{ kv.m_key, kv.m_value, nullptr });
bool has_empty = false, has_undef = false;
obj_map<expr, expr*> local; // var -> witness for this disjunct
for (unsigned gi = 0; gi < groups.size(); ++gi) {
expr_ref w(m);
lbool ne = product_nonempty(groups[gi], model ? &w : nullptr);
if (ne == l_false) { has_empty = true; break; } // this variable has no value
if (ne == l_undef) { has_undef = true; continue; }
if (model) { m_pin.push_back(w); local.insert(group_var[gi], w.get()); }
}
if (has_empty) continue;
if (has_undef) { any_undef = true; continue; }
if (model)
for (auto const& kv : local)
model->insert(kv.m_key, kv.m_value);
return l_true; // all variables satisfiable => sat
}
return any_undef ? l_undef : l_false;
}
lbool seq_monadic::solve(expr* term, expr* R, obj_map<expr, expr*> const& var_extra,
obj_map<expr, expr*>* model) {
m_pin.reset();
m_budget = 200000; // global work budget: bail fast on DNF explosion
m_giveup = false;
vector<disjunct> dnf;
if (!build_membership_dnf(term, R, dnf))
return l_undef;
return decide_dnf(dnf, var_extra, model);
}
lbool seq_monadic::solve_and(vector<std::pair<expr*, expr*>> const& mems,
obj_map<expr, expr*> const& var_extra, obj_map<expr, expr*>* model) {
if (mems.empty())
return l_undef;
m_pin.reset();
m_budget = 200000;
m_giveup = false;
// Multiply the per-membership DNFs: combined = { d ++ e : d in combined, e in dnf_i }.
// A variable shared by several memberships thus gets several components in the same
// disjunct, which decide_dnf/product_nonempty intersect -- enforcing one consistent
// value across all memberships (the joint solve the harness could not do per-term).
vector<disjunct> combined;
combined.push_back(disjunct()); // { true }
const unsigned DNF_CAP = 1u << 14;
for (auto const& tr : mems) {
vector<disjunct> dnf_i;
if (!build_membership_dnf(tr.first, tr.second, dnf_i))
return l_undef;
vector<disjunct> next;
for (disjunct const& d : combined) {
for (disjunct const& e : dnf_i) {
if (next.size() > DNF_CAP || m_budget == 0) { m_giveup = true; return l_undef; }
--m_budget;
disjunct D(d);
for (auto const& c : e)
D.push_back(c);
next.push_back(D);
}
}
combined.swap(next);
simplify_dnf(combined);
if (combined.empty())
return l_false; // no viable disjunct left => unsat
}
return decide_dnf(combined, var_extra, model);
}

148
src/ast/rewriter/seq_monadic.h Normal file
View file

@ -0,0 +1,148 @@
/*++
Copyright (c) 2026 Microsoft Corporation
Module Name:
seq_monadic.h
Abstract:
Whole-language monadic decomposition for regex membership of a term that is a
concatenation of sequence variables and constant elements, e.g. x.a.x in R.
Generic in the element sort: characters are one instance, but the procedure works
for any sequence element sort (the guard algebra falls back from the exact character
range_predicate to a candidate-basis over the element values mentioned by the
derivatives).
Self-contained decision procedure: NO Nielsen splitting (seq_split), NO minterms,
and NO materialization of reach(q) as a regex. It relies only on the symbolic
Brzozowski derivative (brz_derivative_cofactors as a transition regex) and on
automaton product-reachability for emptiness.
Method. For a term x.u in R and the whole-language split, x drives the derivative
automaton of R from R to some live state q, and the rest u must be accepted from q:
x.u in R <=> OR_{q live} ( x reaches q in A_R /\ u in q ).
Decomposing u recursively (a leading constant is consumed by a derivative, a leading
variable splits again, the last variable is a plain membership) yields a DNF whose
disjuncts are conjunctions of per-variable *components*:
- reach component <var, state0, q> : the variable's value drives the
derivative automaton from state0 to q
- membership component<var, state0, null> : the variable's value is in L(state0)
reach(q) is therefore NEVER built as a regex (which state-elimination would blow up
super-polynomially for lattice-shaped automata). Instead the constraints on a
variable are decided directly by a lazy product-reachability search over tuples of
component states: a product state accepts iff every reach component is at its target
and every membership component is nullable; transitions are the product of the
components' cofactor branches with pairwise-conjoined range guards (minterm-free).
This stays in the product-of-state-counts regime, never the path-enumeration (k!)
regime of regex state-elimination.
Supports single / multiple / repeated variables, and per-variable extra constraints
(base membership + length-regex) via `var_extra`.
Author:
Nikolaj Bjorner / Margus Veanes 2026
--*/
#pragma once
#include "ast/rewriter/seq_rewriter.h"
#include "ast/rewriter/seq_range_predicate.h"
#include "ast/rewriter/th_rewriter.h"
#include "util/lbool.h"
#include "util/obj_hashtable.h"
#include <utility>
class seq_monadic {
ast_manager& m;
seq_rewriter& m_rw;
th_rewriter m_thrw; // normalizes constant-element derivatives (folds
// ground guards so dead states become re.empty)
sort* m_seq_sort = nullptr; // sequence sort of the regex under analysis
sort* m_elem_sort = nullptr; // element sort of that sequence sort
expr_ref_vector m_pin; // pins derivative states / witnesses referenced later
unsigned m_budget = 0; // global work budget (decompose disjuncts + product pops)
bool m_giveup = false; // set when the budget is exhausted
seq_util& u() const { return m_rw.u(); }
seq_util::rex& re() const { return m_rw.u().re; }
// A term atom: a sequence variable or a constant element (a value of the element sort).
struct atom { bool is_var; expr* var; expr* elem; };
// A component of one variable's constraint. As the variable's value w is read,
// the current state is derived from `state`; the component accepts when
// target ? (current == target) -- reach component (w drives A from state to target)
// : nullable(current) -- membership component (w in L(state))
struct component { expr* var; expr* state; expr* target; };
typedef svector<component> disjunct; // a conjunction of components (a DNF disjunct)
// Brzozowski derivative of regex `r` by the concrete element `elem`.
expr_ref der_elem(expr* r, expr* elem);
// Live reachable derivative states of R (BFS over cofactor targets + liveness
// least-fixpoint). These are the split states q. Sets `ok` false on a cap overrun.
void live_states(expr* R, ptr_vector<expr>& out, bool& ok);
// Product-reachability emptiness of a conjunction of components (all on one
// variable). l_false = empty (unsat), l_true = non-empty (sat), l_undef = gave up
// (cap overrun, non-range guard, or undecidable nullability).
// On l_true, if `witness_word` is non-null it is set to a concrete sequence term
// (over the element sort) whose value drives every component to acceptance
// simultaneously -- i.e. a witness value for the variable the components constrain.
lbool product_nonempty(svector<component> const& comps, expr_ref* witness_word = nullptr);
// Flatten a str.++ term into atoms; false on an unsupported shape (non-constant unit).
bool parse_term(expr* term, svector<atom>& atoms, expr*& the_var);
// Monadic decomposition: append to `out` the DNF disjuncts for atoms[i..] in R,
// threading the current derivative state R. `ok` false on give-up.
void decompose(svector<atom> const& atoms, unsigned i, expr* R,
vector<disjunct>& out, bool& ok);
// Drop disjuncts with a syntactically-empty component and dedup identical disjuncts.
void simplify_dnf(vector<disjunct>& dnf);
// Build the DNF over primitive per-variable components for one membership term in R.
// Sets m_seq_sort/m_elem_sort; false on an unsupported shape or give-up.
bool build_membership_dnf(expr* term, expr* R, vector<disjunct>& dnf);
// Decide a DNF (over primitive components): sat iff some disjunct has every variable
// group non-empty. On l_true, fills `model` (var -> witness) if non-null.
lbool decide_dnf(vector<disjunct> const& dnf, obj_map<expr, expr*> const& var_extra,
obj_map<expr, expr*>* model);
public:
seq_monadic(seq_rewriter& rw) : m(rw.m()), m_rw(rw), m_thrw(rw.m()), m_pin(rw.m()) {}
// Decide (str.in_re term R) for a term that is a concatenation of string variables
// (possibly repeated / several distinct) and constant characters.
// l_true = sat, l_false = unsat, l_undef = unsupported shape / gave up.
lbool solve(expr* term, expr* R);
// As above, with extra per-variable constraints (e.g. a base membership intersected
// with a length-regex): `var_extra` maps a variable to a regex it must also satisfy.
lbool solve(expr* term, expr* R, obj_map<expr, expr*> const& var_extra);
// As above; on l_true, if `model` is non-null it is populated with var -> witness,
// where each witness is a concrete sequence term (over the element sort) giving one
// satisfying assignment. Witness terms are pinned by the solver and remain valid
// until the next call to solve().
lbool solve(expr* term, expr* R, obj_map<expr, expr*> const& var_extra,
obj_map<expr, expr*>* model);
// Decide a CONJUNCTION of memberships AND_i (term_i in R_i) jointly: a variable
// shared across memberships is constrained consistently (the DNFs are multiplied and
// each variable's constraints intersected). This is the natural extension of single-
// membership solving to a Boolean combination of memberships (a disjunction is the
// union of DNFs; a negated membership ~(t in R) is just t in complement(R)).
// var_extra / model as above. l_true = sat, l_false = unsat, l_undef = gave up.
lbool solve_and(vector<std::pair<expr*, expr*>> const& mems,
obj_map<expr, expr*> const& var_extra, obj_map<expr, expr*>* model = nullptr);
};

70
src/ast/rewriter/seq_range_collapse.cpp
View file

@ -21,66 +21,66 @@ Authors:
namespace seq {
// Cofactor path condition `pred` (a Boolean over x = (:var 0)) -> the canonical
// range_predicate (union of ranges) of the characters satisfying it. Returns
// false on a construct outside {true,false,and,or,not,=,char.<=} over x.
static bool pred_to_rp(ast_manager &m, seq_util &sq, expr *x, expr *pred,
seq::range_predicate &out) {
unsigned maxc = sq.max_char();
expr *a = nullptr, *b = nullptr;
// Cofactor guard `guard` (a Boolean over the character variable v0 = (:var 0)) ->
// the canonical range_predicate (union of ranges) of the characters satisfying it.
// Returns false on a construct outside {true,false,and,or,not,=,char.<=} over v0.
bool guard_to_range_predicate(seq_util& u, expr* v0, expr* guard, range_predicate& out) {
ast_manager& m = u.get_manager();
unsigned maxc = u.max_char();
expr* a = nullptr, * b = nullptr;
unsigned c = 0;
if (m.is_true(pred)) {
out = seq::range_predicate::top(maxc);
if (m.is_true(guard)) {
out = range_predicate::top(maxc);
return true;
}
if (m.is_false(pred)) {
out = seq::range_predicate::empty(maxc);
if (m.is_false(guard)) {
out = range_predicate::empty(maxc);
return true;
}
if (m.is_eq(pred, a, b)) {
if (a == x && sq.is_const_char(b, c)) {
out = seq::range_predicate::singleton(c, maxc);
if (m.is_eq(guard, a, b)) {
if (a == v0 && u.is_const_char(b, c)) {
out = range_predicate::singleton(c, maxc);
return true;
}
if (b == x && sq.is_const_char(a, c)) {
out = seq::range_predicate::singleton(c, maxc);
if (b == v0 && u.is_const_char(a, c)) {
out = range_predicate::singleton(c, maxc);
return true;
}
return false;
}
if (sq.is_char_le(pred, a, b)) {
if (b == x && sq.is_const_char(a, c)) {
out = seq::range_predicate::range(c, maxc, maxc);
if (u.is_char_le(guard, a, b)) {
if (b == v0 && u.is_const_char(a, c)) {
out = range_predicate::range(c, maxc, maxc);
return true;
}
if (a == x && sq.is_const_char(b, c)) {
out = seq::range_predicate::range(0, c, maxc);
if (a == v0 && u.is_const_char(b, c)) {
out = range_predicate::range(0, c, maxc);
return true;
}
return false;
}
if (m.is_not(pred, a)) {
seq::range_predicate s(maxc);
if (!pred_to_rp(m, sq, x, a, s))
if (m.is_not(guard, a)) {
range_predicate s(maxc);
if (!guard_to_range_predicate(u, v0, a, s))
return false;
out = ~s;
return true;
}
if (m.is_and(pred)) {
out = seq::range_predicate::top(maxc);
for (expr *arg : *to_app(pred)) {
seq::range_predicate s(maxc);
if (!pred_to_rp(m, sq, x, arg, s))
if (m.is_and(guard)) {
out = range_predicate::top(maxc);
for (expr *arg : *to_app(guard)) {
range_predicate s(maxc);
if (!guard_to_range_predicate(u, v0, arg, s))
return false;
out = out & s;
}
return true;
}
if (m.is_or(pred)) {
out = seq::range_predicate::empty(maxc);
for (expr *arg : *to_app(pred)) {
seq::range_predicate s(maxc);
if (!pred_to_rp(m, sq, x, arg, s))
if (m.is_or(guard)) {
out = range_predicate::empty(maxc);
for (expr *arg : *to_app(guard)) {
range_predicate s(maxc);
if (!guard_to_range_predicate(u, v0, arg, s))
return false;
out = out | s;
}
@ -183,7 +183,7 @@ namespace seq {
auto body = q->get_expr();
sort *char_sort = q->get_decl_sort(0);
expr_ref var(m.mk_var(0, char_sort), m);
if (u.get_char_plugin().get_family_id() == char_sort->get_family_id() && pred_to_rp(m, u, var, body, out))
if (u.get_char_plugin().get_family_id() == char_sort->get_family_id() && guard_to_range_predicate(u, var, body, out))
return true;
}

8
src/ast/rewriter/seq_range_collapse.h
View file

@ -31,6 +31,14 @@ Authors:
namespace seq {
/**
* Convert a Boolean guard over the single character variable v0 = (:var 0) -- a
* derivative cofactor path condition -- into the range_predicate of the characters
* satisfying it. Recognizes {true, false, =, char.<=, and, or, not} over v0 and
* concrete characters; returns false (out untouched) on anything else.
*/
bool guard_to_range_predicate(seq_util& u, expr* v0, expr* guard, range_predicate& out);
/**
* If r is a boolean combination of character-class regex primitives
* over the unsigned character domain [0, max_char], compute the