From 074f5e7ea606564f782e78812d1e5f6a7ee45627 Mon Sep 17 00:00:00 2001 From: Krystine Sherwin <93062060+KrystalDelusion@users.noreply.github.com> Date: Tue, 5 Aug 2025 09:53:56 +1200 Subject: [PATCH 01/27] Docs: Initial outline of minimizing designs How to guide for using bugpoint, minimizing yosys scripts, and minimizing verilog code. AKA how to MVCE. --- docs/source/using_yosys/bugpoint.rst | 210 +++++++++++++++++++++++++++ docs/source/using_yosys/index.rst | 1 + 2 files changed, 211 insertions(+) create mode 100644 docs/source/using_yosys/bugpoint.rst diff --git a/docs/source/using_yosys/bugpoint.rst b/docs/source/using_yosys/bugpoint.rst new file mode 100644 index 000000000..8c685c35c --- /dev/null +++ b/docs/source/using_yosys/bugpoint.rst @@ -0,0 +1,210 @@ +Minimizing failing (or bugged) designs +====================================== + +- how to guide +- assumes knowledge and familiarity with Yosys +- something is wrong with your design OR something is wrong with Yosys + + + how to work out which + +- *read* the error message +- is it a Yosys error? (starts with ERROR:) + + + does it give you a line number from your design + +- is it a runtime error, e.g. SEGFAULT +- are you using the latest release of Yosys + + + has your problem already been fixed + +- is your input design valid? + + + if you're using Verilog, try load it with `iverilog`_ or `verilator`_ + +.. _iverilog: https://steveicarus.github.io/iverilog/ +.. _verilator: https://www.veripool.org/verilator/ + +- make sure to back up your code (design source and yosys script(s)) before + making any modifications + + + even if the code itself isn't important, this can help avoid "losing" the + error while trying to debug it + + +.. _minimize your RTLIL: + +Minimizing RTLIL designs with bugpoint +-------------------------------------- + +- `bugpoint`, only usable on platforms where Yosys can spawn executables + + + unavailable on emscripten and wasm + + can test by running e.g. ``yosys -qqp '!echo test'`` + + * the ``-qq`` prevents Yosys from outputting non-error messages to the + console, so this will either display the text ``test``, or an error + message about ``Shell`` being unavailable + * be careful about using ``!`` in bash as it will perform a history + substitution if not escaped with single quotes (double quotes will not + escape it) + * ``!`` does not need to be escaped in *Yosys* scripts or when called within + the interactive Yosys shell, *only* when called on the command line with + ``-p`` + +- single command (``yosys -p "" design.il``) +- *or* multiple commands in a separate script file + + + script shouldn't load the design + + ``yosys -s design.il`` + + `minimize your script`_ to reduce the time needed by `bugpoint` + +- doesn't require design to be in RTLIL format + + + can e.g. ``read_verilog ; prep -top ;`` before `bugpoint` + + this method may be more useful if you are trying to find a bug in your + design rather than Yosys + + but, `bugpoint` itself calls the command/script with an RTLIL dump, so if it + isn't reproducible from RTLIL then `bugpoint` won't work + +- follow `bugpoint` instructions + + + output design after `bugpoint` with `write_rtlil` + + use ``-grep ""`` to only accept a minimized design that crashes + with the ```` in the log file + + ``-modules``, ``-ports``, ``-cells``, and ``-processes`` will enable those + parts of the design to be removed + + * use the ``bugpoint_keep`` attribute on objects you don't want to be + removed, usually because you already know they are related to the failure + * ``(* bugpoint_keep *)`` in Verilog, ``attribute \bugpoint_keep 1`` in + RTLIL, or ``setattr -set bugpoint_keep 1 [selection]`` from script + + + ``-runner ""`` can allow running ``yosys`` wrapped by another + command + + can also use `setenv` before `bugpoint` to set environment variables for + the spawned processes (e.g. ``setenv UBSAN_OPTIONS halt_on_error=1``) + +.. note:: + + Using `setenv` in this way **does not affect the current process**. Only + child processes will respect the assigned ``halt_on_error``. + + +.. _minimize your script: + +Minimizing scripts +------------------ + +- reminder to back up original code before modifying it +- if you're using command line, convert it to a script +- if you're using one of the :doc:`/using_yosys/synthesis/synth`, replace it + with its contents +- remove everything *after* the error occurs +- can use `log` command to print messages to help locate the failure point +- `echo` can also help (``echo on``) +- try ``write_rtlil ; design -reset; read_rtlil ;`` before + the failure point + + + ideally you now have a single command that is producing an error and can + `minimize your RTLIL`_ with the ```` output + + if not, try to move the write/reset/read earlier in the script until you can + reproduce the error + + if you have no idea where exactly you should put the reset, the best way is + to use a "binary search" type approach, reducing the possible options by + half after each attempt + + * for example, your script has 16 commands in it before failing on the 17th + * if resetting immediately before the 17th doesn't reproduce the error, try + between the 8th and 9th (8 is half of the total 16) + * if that produces the error then you can remove everything before the + `read_rtlil` and try reset again in the middle of what's left, making sure + to use a different name for the output file so that you don't overwrite + what you've already got + * if the error isn't produced then you need to go earlier still, so in this + case you would do between the 4th and 5th (4 is half of the previous 8) + * repeat this until you can't reduce the remaining commands any further + +.. TODO:: is it possible to dump scratchpad? + + is there anything else in the yosys/design state that doesn't get included in + `write_rtlil`? + +- you can also try to remove or comment out commands prior to the failing + command; just because the first and last commands are needed doesn't mean that + every command between them is + + +Minimizing Verilog designs +-------------------------- + +- manual process +- made easier if the error message is able to identify the source line or name + of the object +- reminder to back up original code before modifying it +- if a specific module is causing the problem, try to set that as the top + module, you can then remove + + + if the problem is parameter specific you may be able to change the default + parameters so that they match the problematic configuration + +- as with `minimize your script`_, if you have no idea what is or is not + relevant, try to follow a "binary search" type approach where you remove (or + comment out) roughly half of what's left at a time +- focusing on one type of object at a time simplifies the process, removing as + many as you can until the error disappears if any of the remaining objects are + removed +- periodically check if anything is totally disconnected (ports, wires, etc), if + it is then it can be removed too +- start by removing cells (instances of modules) + + + if a module has no more instances, remove it entirely + +- then processes +- try to remove or reduce assignments and operations + + + are there any wires/registers which get read but never written? + + * try removing the signal declaration and replacing references to it with + ``'0`` or ``'x`` + * try this with constants too + + + can you replace strings with numeric values? + + are you able to simplify any operations? like replacing ``a & '0`` with + ``'0`` + + if you have enable or reset logic, does the error still happen without that? + + can you reduce an ``if .. else`` to a single case? + +- if you're planning to share the minimized code: + + + make sure there is no sensitive or proprietary data in the design + + instead of a long string of numbers and letters that had some meaning (or + were randomly or sequentially generated), can you give it a single character + name like ``a`` or ``x`` + + please try to keep things in English, using the letters a-z and numbers 0-9 + (unless the error is arising because of the names used) + + +Creating an issue on GitHub +--------------------------- + +- "Reproduction Steps" is ideally a single code-block (starting and ending with + triple backquotes), containing the minimized yosys script file, which includes + the minimized design as a "here document" followed by the sequence of commands + which reproduce the error + +.. TODO:: https://tldp.org/LDP/abs/html/here-docs.html + + Actually fill out :doc:`/using_yosys/more_scripting/load_design` with here + docs info and then link to it from here + +.. code-block:: markdown + + ``` + read_rtlil < Date: Tue, 5 Aug 2025 09:53:56 +1200 Subject: [PATCH 02/27] docs: Outline loading a design page Talk about input files coming from command line, the `read` command, and features provided by `RTLIL::Frontend` (making note that `read_slang` is a subclass but `ghdl` isn't). --- .../more_scripting/load_design.rst | 119 +++++++++++++++--- 1 file changed, 101 insertions(+), 18 deletions(-) diff --git a/docs/source/using_yosys/more_scripting/load_design.rst b/docs/source/using_yosys/more_scripting/load_design.rst index bbc55a36b..964e9a7df 100644 --- a/docs/source/using_yosys/more_scripting/load_design.rst +++ b/docs/source/using_yosys/more_scripting/load_design.rst @@ -1,11 +1,88 @@ Loading a design -~~~~~~~~~~~~~~~~ +---------------- -keyword: Frontends +.. _input files: + +Input files on the command line +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- guesses frontend based on file extension + + + ``.v`` -> ``read -vlog2k`` + + ``.sv`` -> ``read -sv`` + + ``.vhd`` and ``.vhdl`` -> ``read -vhdl`` + + ``.blif`` and ``.eblif`` -> `read_blif` + + ``.json`` -> `read_json` + + ``.il`` -> `read_rtlil` (direct textual representation of Yosys internal + state) + +- command line also supports + + + ``.ys`` -> `script` + + ``.tcl`` -> `tcl` + + ``-`` -> reads stdin and treats it as a script + +The `read` command +~~~~~~~~~~~~~~~~~~ + +- standard method of loading designs +- also for defining macros and include directories +- uses `verific` command if available + + + ``-verific`` and ``-noverific`` options to enforce with/without Verific + + check ``help read`` for more about the options available and the filetypes + supported + +- fallback to `read_verilog` + +.. note:: + + The Verific frontend for Yosys, which provides the :cmd:ref:`verific` + command, requires Yosys to be built with Verific. For full functionality, + custom modifications to the Verific source code from YosysHQ are required, + but limited useability can be achieved with some stock Verific builds. Check + :doc:`/yosys_internals/extending_yosys/build_verific` for more. + +.. _Frontend: + +Yosys frontends +~~~~~~~~~~~~~~~ + +- typically start with ``read_`` +- built-in support for heredocs + + + in-line code with ``< Date: Tue, 5 Aug 2025 09:53:57 +1200 Subject: [PATCH 03/27] Docs: Bugpoint fixups from JF Also dropping the `autosectionlabel_maxdepth = 1` so that I can actually use the auto section labels. Adds warning on bash substitution on scripting intro page when talking about `yosys -p`. --- docs/source/conf.py | 1 - .../getting_started/scripting_intro.rst | 9 ++- docs/source/using_yosys/bugpoint.rst | 79 ++++++++++++------- .../more_scripting/load_design.rst | 18 ++++- 4 files changed, 74 insertions(+), 33 deletions(-) diff --git a/docs/source/conf.py b/docs/source/conf.py index 05dcb7d5f..e587b8d31 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -64,7 +64,6 @@ if os.getenv("READTHEDOCS"): # Ensure that autosectionlabel will produce unique names autosectionlabel_prefix_document = True -autosectionlabel_maxdepth = 1 # include todos for previews extensions.append('sphinx.ext.todo') diff --git a/docs/source/getting_started/scripting_intro.rst b/docs/source/getting_started/scripting_intro.rst index 01954c661..6a6e4ff51 100644 --- a/docs/source/getting_started/scripting_intro.rst +++ b/docs/source/getting_started/scripting_intro.rst @@ -26,7 +26,7 @@ of the comment is a semicolon ``;`` or a new line. .. code-block:: :caption: Using the ``-p`` option - $ yosys -p "read_verilog fifo.v; :this is a comment; prep" + $ yosys -p 'read_verilog fifo.v; :this is a comment; prep' .. warning:: @@ -42,6 +42,13 @@ will be raised by Yosys. `exec` provides a much more flexible way of executing commands, allowing the output to be logged and more control over when to generate errors. +.. warning:: + + Take care when using the ``yosys -p`` option. Some shells such as bash will + perform substitution options inside of a double quoted string, such as ``!`` + for history substitution and ``$`` for variable substitution; single quotes + should be used instead to pass the string to Yosys without substitution. + The synthesis starter script ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/docs/source/using_yosys/bugpoint.rst b/docs/source/using_yosys/bugpoint.rst index 8c685c35c..9b402858e 100644 --- a/docs/source/using_yosys/bugpoint.rst +++ b/docs/source/using_yosys/bugpoint.rst @@ -44,14 +44,10 @@ Minimizing RTLIL designs with bugpoint * the ``-qq`` prevents Yosys from outputting non-error messages to the console, so this will either display the text ``test``, or an error message about ``Shell`` being unavailable - * be careful about using ``!`` in bash as it will perform a history - substitution if not escaped with single quotes (double quotes will not - escape it) - * ``!`` does not need to be escaped in *Yosys* scripts or when called within - the interactive Yosys shell, *only* when called on the command line with - ``-p`` + * check :ref:`getting_started/scripting_intro:script parsing` for more about + the ``-p`` option and common pitfalls -- single command (``yosys -p "" design.il``) +- single command (``yosys -p '' design.il``) - *or* multiple commands in a separate script file + script shouldn't load the design @@ -68,26 +64,30 @@ Minimizing RTLIL designs with bugpoint - follow `bugpoint` instructions - + output design after `bugpoint` with `write_rtlil` - + use ``-grep ""`` to only accept a minimized design that crashes - with the ```` in the log file - + ``-modules``, ``-ports``, ``-cells``, and ``-processes`` will enable those - parts of the design to be removed + + output design after `bugpoint` with `write_rtlil` + + use ``-grep ""`` to only accept a minimized design that crashes + with the ```` in the log file + + ``-modules``, ``-ports``, ``-cells``, and ``-processes`` will enable those + parts of the design to be removed - * use the ``bugpoint_keep`` attribute on objects you don't want to be - removed, usually because you already know they are related to the failure - * ``(* bugpoint_keep *)`` in Verilog, ``attribute \bugpoint_keep 1`` in - RTLIL, or ``setattr -set bugpoint_keep 1 [selection]`` from script + * use the ``bugpoint_keep`` attribute on objects you don't want to be + removed, usually because you already know they are related to the failure + * ``(* bugpoint_keep *)`` in Verilog, ``attribute \bugpoint_keep 1`` in + RTLIL, or ``setattr -set bugpoint_keep 1 [selection]`` from script - + ``-runner ""`` can allow running ``yosys`` wrapped by another - command - + can also use `setenv` before `bugpoint` to set environment variables for - the spawned processes (e.g. ``setenv UBSAN_OPTIONS halt_on_error=1``) + + ``-runner ""`` can allow running ``yosys`` wrapped by another + command + + can also use `setenv` before `bugpoint` to set environment variables for + the spawned processes (e.g. ``setenv UBSAN_OPTIONS halt_on_error=1``) .. note:: - Using `setenv` in this way **does not affect the current process**. Only - child processes will respect the assigned ``halt_on_error``. + Using `setenv` in this way may not affect the current process as some + environment variables are only read on start up. For instance the + ``UBSAN_OPTIONS halt_on_error`` here only affects child processes, as does + the :doc:`Yosys environment variable` ``ABC``. While + others such as ``YOSYS_NOVERIFIC`` and ``HOME`` are evaluated each time they + are used. .. _minimize your script: @@ -187,15 +187,34 @@ Minimizing Verilog designs Creating an issue on GitHub --------------------------- -- "Reproduction Steps" is ideally a single code-block (starting and ending with - triple backquotes), containing the minimized yosys script file, which includes - the minimized design as a "here document" followed by the sequence of commands - which reproduce the error +- "Reproduction Steps" is ideally a code-block (starting and ending with triple + backquotes) containing the minimized design (Verilog or RTLIL), followed by a + code-block containing the minimized yosys script OR a command line call to + yosys with code-formatting (starting and ending with single backquotes) -.. TODO:: https://tldp.org/LDP/abs/html/here-docs.html +.. code-block:: markdown - Actually fill out :doc:`/using_yosys/more_scripting/load_design` with here - docs info and then link to it from here + min.v + ```verilog + // minimized Verilog design + ``` + + min.ys + ``` + read_verilog min.v + # minimum sequence of commands to reproduce error + ``` + + OR + + `yosys -p ': minimum sequence of commands;' min.v` + + +- alternatively can provide a single code-block which includes the minimized + design as a "here document" followed by the sequence of commands which + reproduce the error + + + see :doc:`/using_yosys/more_scripting/load_design` for more on heredocs. .. code-block:: markdown @@ -203,7 +222,7 @@ Creating an issue on GitHub read_rtlil <`` (or use + `hierarchy`) -- fallback to `read_verilog` +- fallback to `read_verilog` with ``-defer`` option + + + does not compile design until `hierarchy` command as discussed in + :doc:`/getting_started/example_synth` + + more similar to `verific` behaviour + +- ``read -define`` et al mapped to `verific` or `verilog_defines` +- similarly, ``read -incdir`` et al mapped to `verific` or `verilog_defaults` .. note:: @@ -129,3 +138,10 @@ Externally maintained plugins - both plugins above are included in `OSS CAD Suite`_ .. _OSS CAD Suite: https://github.com/YosysHQ/oss-cad-suite-build + +- `Synlig`_, which uses `Surelog`_ to provide SystemVerilog support + + + also implemented as a '`Frontend`_' + +.. _Synlig: https://github.com/chipsalliance/synlig +.. _Surelog: https://github.com/chipsalliance/Surelog From db3dc45bc68f97b244d0732260dbcd31094b1fc2 Mon Sep 17 00:00:00 2001 From: Krystine Sherwin <93062060+KrystalDelusion@users.noreply.github.com> Date: Tue, 5 Aug 2025 09:53:57 +1200 Subject: [PATCH 04/27] Docs: Tidying Fix error on duplicated heading. Drop `cmd_ref`_ link (everything already uses :doc:`cmd_ref`). --- docs/source/cmd_ref.rst | 2 -- .../using_yosys/more_scripting/interactive_investigation.rst | 4 ++-- 2 files changed, 2 insertions(+), 4 deletions(-) diff --git a/docs/source/cmd_ref.rst b/docs/source/cmd_ref.rst index acf2d1d41..d2d59ba54 100644 --- a/docs/source/cmd_ref.rst +++ b/docs/source/cmd_ref.rst @@ -1,5 +1,3 @@ -.. _cmd_ref: - ================================================================================ Command line reference ================================================================================ diff --git a/docs/source/using_yosys/more_scripting/interactive_investigation.rst b/docs/source/using_yosys/more_scripting/interactive_investigation.rst index e9c9bc9ac..ca4c76ee7 100644 --- a/docs/source/using_yosys/more_scripting/interactive_investigation.rst +++ b/docs/source/using_yosys/more_scripting/interactive_investigation.rst @@ -311,8 +311,8 @@ cells, as the net-names are usually suppressed in the circuit diagram if they are auto-generated. Note that the output is in the RTLIL representation, described in :doc:`/yosys_internals/formats/rtlil_rep`. -Interactive Design Investigation -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Design Investigation +~~~~~~~~~~~~~~~~~~~~ Yosys can also be used to investigate designs (or netlists created from other tools). From 6e1cc9c0cd278f30c1593e5d4a021de8ab578a53 Mon Sep 17 00:00:00 2001 From: Krystine Sherwin <93062060+KrystalDelusion@users.noreply.github.com> Date: Tue, 5 Aug 2025 09:53:57 +1200 Subject: [PATCH 05/27] docs: Some extra bugpoint bullets --- docs/source/using_yosys/bugpoint.rst | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/docs/source/using_yosys/bugpoint.rst b/docs/source/using_yosys/bugpoint.rst index 9b402858e..d292c2d32 100644 --- a/docs/source/using_yosys/bugpoint.rst +++ b/docs/source/using_yosys/bugpoint.rst @@ -67,8 +67,11 @@ Minimizing RTLIL designs with bugpoint + output design after `bugpoint` with `write_rtlil` + use ``-grep ""`` to only accept a minimized design that crashes with the ```` in the log file + + * only checks log file, will not match runtime errors + + ``-modules``, ``-ports``, ``-cells``, and ``-processes`` will enable those - parts of the design to be removed + parts of the design to be removed (default is allow removing all) * use the ``bugpoint_keep`` attribute on objects you don't want to be removed, usually because you already know they are related to the failure @@ -89,6 +92,23 @@ Minimizing RTLIL designs with bugpoint others such as ``YOSYS_NOVERIFIC`` and ``HOME`` are evaluated each time they are used. +- check minimized design still fails, especially if not using `write_rtlil` + + + e.g. if you ran :ref:`bugpoint_script` below, then calling ``yosys -s + min.v`` should still fail in the same way + + `write_rtlil` is more reliable since `bugpoint` will have run that exact + code through the failing script; other ``write_*`` commands convert from the + RTLIL and then back again during the ``read_*`` which can result in + differences which mean the design no longer fails + +.. code-block:: yoscrypt + :caption: example `bugpoint` minimizer + :name: bugpoint_script + + read_verilog design.v + bugpoint -script + write_verilog min.v + .. _minimize your script: From aefe3443aac9fbeb5549502893d12afdebc9f9a3 Mon Sep 17 00:00:00 2001 From: Krystine Sherwin <93062060+KrystalDelusion@users.noreply.github.com> Date: Tue, 5 Aug 2025 09:53:57 +1200 Subject: [PATCH 06/27] docs: Minimizing synth with -run bullets --- docs/source/using_yosys/bugpoint.rst | 35 ++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/docs/source/using_yosys/bugpoint.rst b/docs/source/using_yosys/bugpoint.rst index d292c2d32..804787df8 100644 --- a/docs/source/using_yosys/bugpoint.rst +++ b/docs/source/using_yosys/bugpoint.rst @@ -119,9 +119,44 @@ Minimizing scripts - if you're using command line, convert it to a script - if you're using one of the :doc:`/using_yosys/synthesis/synth`, replace it with its contents + + + can also do this piece-wise with the ``-run`` option + + e.g. replacing ``synth -top -lut`` with :ref:`replace_synth` + + the options ``-top -lut`` can be provided to each `synth` step, or + to just the step(s) where it is relevant, as done here + +.. code-block:: yoscrypt + :caption: example replacement script for `synth` command + :name: replace_synth + + synth -top -run :coarse + synth -lut -run coarse:fine + synth -lut -run fine:check + synth -run check: + - remove everything *after* the error occurs - can use `log` command to print messages to help locate the failure point - `echo` can also help (``echo on``) + + + if you used a ``-run`` option like in :ref:`replace_synth` above, you can + now replace the failing step with its contents and repeat the above if + needed + + checking the log should tell you the last command that ran which can make + this easier + + say we ran :ref:`replace_synth` and were able to remove the ``synth -run + check:`` and still got our error, then we check the log and we see the last + thing before the error was `7.2. Executing MEMORY_MAP pass (converting + memories to logic and flip-flops).` + + we can then update our script to the following: + +.. code-block:: yoscrypt + :caption: example replacement script for `synth` when `memory_map` is failing + + synth -top -run :fine + opt -fast -full + memory_map + + - try ``write_rtlil ; design -reset; read_rtlil ;`` before the failure point From 8ec3e3a1025943b45772644b0a2387e1c3a1f238 Mon Sep 17 00:00:00 2001 From: Krystine Sherwin <93062060+KrystalDelusion@users.noreply.github.com> Date: Tue, 5 Aug 2025 09:53:58 +1200 Subject: [PATCH 07/27] docs: Bullets for identifying issues Add a note on fuzzers, with a polite suggestion that if you're fuzzing you should put in the work of identifying the underlying issue so that you (and we) are confident you're not raising multiple issues for the same bug. --- docs/source/using_yosys/bugpoint.rst | 113 +++++++++++++++++++++------ 1 file changed, 89 insertions(+), 24 deletions(-) diff --git a/docs/source/using_yosys/bugpoint.rst b/docs/source/using_yosys/bugpoint.rst index 804787df8..abc6094a6 100644 --- a/docs/source/using_yosys/bugpoint.rst +++ b/docs/source/using_yosys/bugpoint.rst @@ -36,6 +36,11 @@ Minimizing failing (or bugged) designs Minimizing RTLIL designs with bugpoint -------------------------------------- +- what is `bugpoint` + +Can I use bugpoint? +~~~~~~~~~~~~~~~~~~~ + - `bugpoint`, only usable on platforms where Yosys can spawn executables + unavailable on emscripten and wasm @@ -62,26 +67,29 @@ Minimizing RTLIL designs with bugpoint + but, `bugpoint` itself calls the command/script with an RTLIL dump, so if it isn't reproducible from RTLIL then `bugpoint` won't work + +How do I use bugpoint? +~~~~~~~~~~~~~~~~~~~~~~ + - follow `bugpoint` instructions +- output design after `bugpoint` with `write_rtlil` +- use ``-grep ""`` to only accept a minimized design that crashes +- with the ```` in the log file - + output design after `bugpoint` with `write_rtlil` - + use ``-grep ""`` to only accept a minimized design that crashes - with the ```` in the log file + + only checks log file, will not match runtime errors - * only checks log file, will not match runtime errors +- ``-modules``, ``-ports``, ``-cells``, and ``-processes`` will enable those +- parts of the design to be removed (default is allow removing all) - + ``-modules``, ``-ports``, ``-cells``, and ``-processes`` will enable those - parts of the design to be removed (default is allow removing all) + + use the ``bugpoint_keep`` attribute on objects you don't want to be + removed, usually because you already know they are related to the failure + + ``(* bugpoint_keep *)`` in Verilog, ``attribute \bugpoint_keep 1`` in + RTLIL, or ``setattr -set bugpoint_keep 1 [selection]`` from script - * use the ``bugpoint_keep`` attribute on objects you don't want to be - removed, usually because you already know they are related to the failure - * ``(* bugpoint_keep *)`` in Verilog, ``attribute \bugpoint_keep 1`` in - RTLIL, or ``setattr -set bugpoint_keep 1 [selection]`` from script - - + ``-runner ""`` can allow running ``yosys`` wrapped by another - command - + can also use `setenv` before `bugpoint` to set environment variables for - the spawned processes (e.g. ``setenv UBSAN_OPTIONS halt_on_error=1``) +- ``-runner ""`` can allow running ``yosys`` wrapped by another +- command +- can also use `setenv` before `bugpoint` to set environment variables for +- the spawned processes (e.g. ``setenv UBSAN_OPTIONS halt_on_error=1``) .. note:: @@ -92,14 +100,13 @@ Minimizing RTLIL designs with bugpoint others such as ``YOSYS_NOVERIFIC`` and ``HOME`` are evaluated each time they are used. -- check minimized design still fails, especially if not using `write_rtlil` - + e.g. if you ran :ref:`bugpoint_script` below, then calling ``yosys -s - min.v`` should still fail in the same way - + `write_rtlil` is more reliable since `bugpoint` will have run that exact - code through the failing script; other ``write_*`` commands convert from the - RTLIL and then back again during the ``read_*`` which can result in - differences which mean the design no longer fails +What do I do with the minimized design? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- check minimized design still fails, especially if not using `write_rtlil` +- e.g. if you ran :ref:`bugpoint_script` below, then calling ``yosys -s + min.v`` should still fail in the same way .. code-block:: yoscrypt :caption: example `bugpoint` minimizer @@ -109,6 +116,12 @@ Minimizing RTLIL designs with bugpoint bugpoint -script write_verilog min.v +- `write_rtlil` is more reliable since `bugpoint` will have run that exact + code through the failing script; other ``write_*`` commands convert from the + RTLIL and then back again during the ``read_*`` which can result in + differences which mean the design no longer fails +- check out :ref:`using_yosys/bugpoint:identifying issues` for more on what to + do next .. _minimize your script: @@ -145,8 +158,8 @@ Minimizing scripts this easier + say we ran :ref:`replace_synth` and were able to remove the ``synth -run check:`` and still got our error, then we check the log and we see the last - thing before the error was `7.2. Executing MEMORY_MAP pass (converting - memories to logic and flip-flops).` + thing before the error was ``7.2. Executing MEMORY_MAP pass (converting + memories to logic and flip-flops).`` + we can then update our script to the following: .. code-block:: yoscrypt @@ -239,6 +252,58 @@ Minimizing Verilog designs (unless the error is arising because of the names used) +Identifying issues +------------------ + +- does the failing command indicate limited support, or does it mention some + other command that needs to be run first? +- if you're able to, try to match the minimized design back to its original + context + + + could you achieve the same thing a different way? + + and if so, does this other method have the same issue? + +- try to change the design in small ways and see what happens + + + `bugpoint` can reduce and simplify a design, but it doesn't *change* much + + what happens if you change operators, for example a left shift (or `$shl`) + to a right shift (or `$shr`)? + + is the issue tied to specific parameters, widths, or values? + +- if you're familiar with C/C++ you might try to have a look at the source + code of the command that's failing + + + even if you can't fix the problem yourself, it can be very helpful for + anyone else investigating if you're able to identify where exactly the + issue is + + if you're using a fuzzer to find issues in Yosys, you should be prepared to + do this step + +.. warning:: + + In the event that you are unable to identify the root cause of a fuzzer + generated issue, **do not** open more than one issue at a time. You have no + way of being able to tell if multiple fuzzer generated issues are simply + different cases of the same problem, and opening multiple issues for the same + problem means more time is spent on triaging and diagnosing bug reports and + less on fixing the problem. If you are found to be doing this, your issues + may be closed without further investigation. + +- search `the existing issues`_ and see if someone has already made a bug report + + + this is where changing the design and finding the limits of what causes the + failure really comes in handy + + if you're more familiar with how the problem can arise, you may be able to + find a related issue more easily + + if an issue already exists for one case of the problem but you've found + other cases, you can comment on the issue and help get it solved + +.. _the existing issues: https://github.com/YosysHQ/yosys/issues + +- if there are no existing or related issues already, the check out the steps + for :ref:`using_yosys/bugpoint:creating an issue on github` + + Creating an issue on GitHub --------------------------- From 385d58562d171f86884247a1ae1971666362df82 Mon Sep 17 00:00:00 2001 From: Krystine Sherwin <93062060+KrystalDelusion@users.noreply.github.com> Date: Tue, 5 Aug 2025 09:53:58 +1200 Subject: [PATCH 08/27] Docs: Move verilog.rst to using_yosys Was previously in yosys_internals which is more developer focused, rather than user focused. --- docs/source/using_yosys/index.rst | 1 + docs/source/{yosys_internals => using_yosys}/verilog.rst | 2 -- docs/source/yosys_internals/index.rst | 1 - 3 files changed, 1 insertion(+), 3 deletions(-) rename docs/source/{yosys_internals => using_yosys}/verilog.rst (99%) diff --git a/docs/source/using_yosys/index.rst b/docs/source/using_yosys/index.rst index 0ea91fae7..b243d431e 100644 --- a/docs/source/using_yosys/index.rst +++ b/docs/source/using_yosys/index.rst @@ -16,3 +16,4 @@ ways Yosys can interact with designs for a deeper investigation. synthesis/index more_scripting/index bugpoint + verilog diff --git a/docs/source/yosys_internals/verilog.rst b/docs/source/using_yosys/verilog.rst similarity index 99% rename from docs/source/yosys_internals/verilog.rst rename to docs/source/using_yosys/verilog.rst index d67553aa9..1701458f7 100644 --- a/docs/source/yosys_internals/verilog.rst +++ b/docs/source/using_yosys/verilog.rst @@ -1,8 +1,6 @@ Notes on Verilog support in Yosys ================================= -.. TODO:: how much of this is specific to the read_verilog and should be in :doc:`/yosys_internals/flow/verilog_frontend`? - Unsupported Verilog-2005 Features --------------------------------- diff --git a/docs/source/yosys_internals/index.rst b/docs/source/yosys_internals/index.rst index 3dd4224fa..483cc2bf8 100644 --- a/docs/source/yosys_internals/index.rst +++ b/docs/source/yosys_internals/index.rst @@ -38,5 +38,4 @@ as reference to implement a similar system in any language. formats/index extending_yosys/index techmap - verilog hashing From be999219a2972acb6d1931c6c589d8dd5f02383a Mon Sep 17 00:00:00 2001 From: Krystine Sherwin <93062060+KrystalDelusion@users.noreply.github.com> Date: Tue, 5 Aug 2025 09:53:58 +1200 Subject: [PATCH 09/27] docs: User-defined failures in bugpoint Also some other tidy up and clarifications. --- docs/source/using_yosys/bugpoint.rst | 47 +++++++++++++++++++--------- 1 file changed, 33 insertions(+), 14 deletions(-) diff --git a/docs/source/using_yosys/bugpoint.rst b/docs/source/using_yosys/bugpoint.rst index abc6094a6..a74693fa3 100644 --- a/docs/source/using_yosys/bugpoint.rst +++ b/docs/source/using_yosys/bugpoint.rst @@ -36,12 +36,16 @@ Minimizing failing (or bugged) designs Minimizing RTLIL designs with bugpoint -------------------------------------- -- what is `bugpoint` +Yosys provides the `bugpoint` command for reducing a failing design to the +smallest portion of that design which still results in failure. While initially +developed for Yosys crashes, `bugpoint` can also be used for designs that lead +to non-fatal errors, or even failures in other tools that use the output of a +Yosys script. Can I use bugpoint? ~~~~~~~~~~~~~~~~~~~ -- `bugpoint`, only usable on platforms where Yosys can spawn executables +- only usable on platforms where Yosys can spawn executables + unavailable on emscripten and wasm + can test by running e.g. ``yosys -qqp '!echo test'`` @@ -67,6 +71,18 @@ Can I use bugpoint? + but, `bugpoint` itself calls the command/script with an RTLIL dump, so if it isn't reproducible from RTLIL then `bugpoint` won't work +- works with user-defined failures in scripts + + + e.g. `select` command with ``-assert-*`` option + + or `equiv_opt` + + can even call another tool with `exec` + + * useful for when Yosys is outputting an invalid design + * use the ``-expect-*`` options to ensure the script correctly returns the + failure state to `bugpoint` + * can call shell scripts with e.g. ``exec -expect-return 1 -- bash + `` + How do I use bugpoint? ~~~~~~~~~~~~~~~~~~~~~~ @@ -74,31 +90,34 @@ How do I use bugpoint? - follow `bugpoint` instructions - output design after `bugpoint` with `write_rtlil` - use ``-grep ""`` to only accept a minimized design that crashes -- with the ```` in the log file + with the ```` in the log file + only checks log file, will not match runtime errors + + can be particularly important for scripts with multiple commands to avoid + unrelated failures + + call e.g. ``yosys -qqp '' design.il`` or ``yosys -qqs + design.il`` to print only the error message(s) and use that (or a portion of + that) as the ```` to search for - ``-modules``, ``-ports``, ``-cells``, and ``-processes`` will enable those -- parts of the design to be removed (default is allow removing all) + parts of the design to be removed (default is to allow removing all) + use the ``bugpoint_keep`` attribute on objects you don't want to be removed, usually because you already know they are related to the failure + ``(* bugpoint_keep *)`` in Verilog, ``attribute \bugpoint_keep 1`` in RTLIL, or ``setattr -set bugpoint_keep 1 [selection]`` from script -- ``-runner ""`` can allow running ``yosys`` wrapped by another -- command +- ``-runner ""`` can allow running ``yosys`` wrapped by another command - can also use `setenv` before `bugpoint` to set environment variables for -- the spawned processes (e.g. ``setenv UBSAN_OPTIONS halt_on_error=1``) + the spawned processes (e.g. ``setenv UBSAN_OPTIONS halt_on_error=1``) .. note:: - Using `setenv` in this way may not affect the current process as some - environment variables are only read on start up. For instance the - ``UBSAN_OPTIONS halt_on_error`` here only affects child processes, as does - the :doc:`Yosys environment variable` ``ABC``. While - others such as ``YOSYS_NOVERIFIC`` and ``HOME`` are evaluated each time they - are used. + Using `setenv` in this way may or may not affect the current process. For + instance the ``UBSAN_OPTIONS halt_on_error`` here only affects child + processes, as does the :doc:`Yosys environment variable` + ``ABC`` because they are only read on start-up. While others, such as + ``YOSYS_NOVERIFIC`` and ``HOME``, are evaluated each time they are used. What do I do with the minimized design? @@ -300,7 +319,7 @@ Identifying issues .. _the existing issues: https://github.com/YosysHQ/yosys/issues -- if there are no existing or related issues already, the check out the steps +- if there are no existing or related issues already, then check out the steps for :ref:`using_yosys/bugpoint:creating an issue on github` From 47c89a61df5c85f60ff738e0340554e583597570 Mon Sep 17 00:00:00 2001 From: Krystine Sherwin <93062060+KrystalDelusion@users.noreply.github.com> Date: Tue, 5 Aug 2025 09:53:58 +1200 Subject: [PATCH 10/27] Docs: What is bugpoint in paragraphs --- docs/source/using_yosys/bugpoint.rst | 82 +++++++++++++++++----------- 1 file changed, 51 insertions(+), 31 deletions(-) diff --git a/docs/source/using_yosys/bugpoint.rst b/docs/source/using_yosys/bugpoint.rst index a74693fa3..ffefcb144 100644 --- a/docs/source/using_yosys/bugpoint.rst +++ b/docs/source/using_yosys/bugpoint.rst @@ -45,43 +45,64 @@ Yosys script. Can I use bugpoint? ~~~~~~~~~~~~~~~~~~~ -- only usable on platforms where Yosys can spawn executables +The first thing to be aware of is that `bugpoint` is not available in every +build of Yosys. Because the command works by invoking external processes, it +requires that Yosys can spawn executables. Notably this means `bugpoint` is not +able to be used in WebAssembly builds such as that available via YoWASP. The +easiest way to check your build of Yosys is by running ``yosys -qq -p '!echo +test'``. If this echoes ``test`` in the console, then `bugpoint` will work as +expected. If instead if it displays the text ``ERROR: Shell is not available.`` +then `bugpoint` will not work either. - + unavailable on emscripten and wasm - + can test by running e.g. ``yosys -qqp '!echo test'`` +.. note:: - * the ``-qq`` prevents Yosys from outputting non-error messages to the - console, so this will either display the text ``test``, or an error - message about ``Shell`` being unavailable - * check :ref:`getting_started/scripting_intro:script parsing` for more about - the ``-p`` option and common pitfalls + The console command ``yosys -qq -p '!echo test'`` uses the ``-qq`` flag to + prevent Yosys from outputting non-error messages to the console. The ``-p`` + option executes ``!echo test`` as a Yosys command, attempting to pass ``echo + test`` to the shell for execution. For more about the ``-p`` option and + common pitfalls, check out :ref:`getting_started/scripting_intro:script + parsing` in the :doc:`/getting_started/index` section. -- single command (``yosys -p '' design.il``) -- *or* multiple commands in a separate script file +.. TODO:: Add ``YOSYS_DISABLE_SPAWN`` check in ``bugpoint.cc``. - + script shouldn't load the design - + ``yosys -s design.il`` - + `minimize your script`_ to reduce the time needed by `bugpoint` + At least in the help text, so that ``yosys -h bugpoint`` will correctly + indicate if the command will work instead of this roundabout method. -- doesn't require design to be in RTLIL format +Next you need to separate loading the design from the failure point; you should +be aiming to reproduce the failure by running ``yosys -s -s +``. If the failure occurs while loading the design, such as during +`read_verilog` you will instead have to minimize the input design yourself. +Check out the instructions for :ref:`using_yosys/bugpoint:minimizing verilog +designs` below. - + can e.g. ``read_verilog ; prep -top ;`` before `bugpoint` - + this method may be more useful if you are trying to find a bug in your - design rather than Yosys - + but, `bugpoint` itself calls the command/script with an RTLIL dump, so if it - isn't reproducible from RTLIL then `bugpoint` won't work +The commands in ```` only need to be run once, while those in +```` will be run on each iteration of `bugpoint`. If you haven't +already, following the instructions for :ref:`using_yosys/bugpoint:minimizing +scripts` will also help with identifying exactly which commands are needed to +produce the failure and which can be safely moved to the loading script. -- works with user-defined failures in scripts +.. note:: - + e.g. `select` command with ``-assert-*`` option - + or `equiv_opt` - + can even call another tool with `exec` - - * useful for when Yosys is outputting an invalid design - * use the ``-expect-*`` options to ensure the script correctly returns the - failure state to `bugpoint` - * can call shell scripts with e.g. ``exec -expect-return 1 -- bash - `` + You should also be able to run the two scripts separately, calling first + ``yosys -s -p 'write_rtlil design.il'`` and then ``yosys -s + design.il``. If this doesn't work then it may mean that the + failure isn't reproducible from RTLIL and `bugpoint` won't work either. + +When we talk about failure points here, it doesn't just mean crashes or errors +in Yosys. The ```` script can also be a user-defined failure such +as the `select` command with one of the ``-assert-*`` options; an example where +this might be useful is when a pass is supposed to remove a certain kind of +cell, but there is some edge case where the cell is not removed. Another +use-case would be minimizing a design which fails with the `equiv_opt` command, +suggesting that the optimization in question alters the circuit in some way. + +It is even possible to use `bugpoint` with failures *external* to Yosys, by +making use of the `exec` command in ````. This is especially useful +when Yosys is outputting an invalid design, or when some other tool is +incompatible with the design. Be sure to use the ``exec -expect-*`` options so +that the pass/fail can be detected correctly. Multiple calls to `exec` can be +made, or even entire shell scripts (e.g. ``exec -expect-return 1 -- bash +``). How do I use bugpoint? @@ -142,6 +163,7 @@ What do I do with the minimized design? - check out :ref:`using_yosys/bugpoint:identifying issues` for more on what to do next + .. _minimize your script: Minimizing scripts @@ -188,7 +210,6 @@ Minimizing scripts opt -fast -full memory_map - - try ``write_rtlil ; design -reset; read_rtlil ;`` before the failure point @@ -348,7 +369,6 @@ Creating an issue on GitHub `yosys -p ': minimum sequence of commands;' min.v` - - alternatively can provide a single code-block which includes the minimized design as a "here document" followed by the sequence of commands which reproduce the error From 2c534c88282e6d04760b0a09a7b76b697d6530f4 Mon Sep 17 00:00:00 2001 From: Krystine Sherwin <93062060+KrystalDelusion@users.noreply.github.com> Date: Tue, 5 Aug 2025 09:53:59 +1200 Subject: [PATCH 11/27] Docs: How to use bugpoint paragraphs --- docs/source/using_yosys/bugpoint.rst | 98 ++++++++++++++++++++++------ 1 file changed, 79 insertions(+), 19 deletions(-) diff --git a/docs/source/using_yosys/bugpoint.rst b/docs/source/using_yosys/bugpoint.rst index ffefcb144..7d38124bf 100644 --- a/docs/source/using_yosys/bugpoint.rst +++ b/docs/source/using_yosys/bugpoint.rst @@ -104,33 +104,75 @@ that the pass/fail can be detected correctly. Multiple calls to `exec` can be made, or even entire shell scripts (e.g. ``exec -expect-return 1 -- bash ``). +Our final failure we can use with `bugpoint` is one returned by a wrapper +process, such as ``valgrind`` or ``timeout``. In this case you will be calling +something like `` yosys -s design.il``. Here, Yosys is +run under a wrapper process which checks for some failure state, like a memory +leak or excessive runtime. Note however that unlike the `exec` command, there +is currently no way to check the return status or messages from the wrapper +process; only a binary pass/fail. + How do I use bugpoint? ~~~~~~~~~~~~~~~~~~~~~~ -- follow `bugpoint` instructions -- output design after `bugpoint` with `write_rtlil` -- use ``-grep ""`` to only accept a minimized design that crashes - with the ```` in the log file +At this point you should have: - + only checks log file, will not match runtime errors - + can be particularly important for scripts with multiple commands to avoid - unrelated failures - + call e.g. ``yosys -qqp '' design.il`` or ``yosys -qqs - design.il`` to print only the error message(s) and use that (or a portion of - that) as the ```` to search for +1. either an RTLIL file containing the design to minimize (referred to here as + ``design.il``), or a Yosys script, ````, which loads it; and +2. a Yosys script, ````, which produces the failure and returns a + non-zero return status. -- ``-modules``, ``-ports``, ``-cells``, and ``-processes`` will enable those - parts of the design to be removed (default is to allow removing all) +Now call ``yosys -qq -s design.il`` and take note of the error(s) +that get printed. A template script, ````, is provided here which +you can use. Make sure to configure it with the correct filenames and use only +one of the methods to load the design. Fill in the ``-grep`` option with the +error message printed just before. If you are using a wrapper process for your +failure state, add the ``-runner ""`` option to the `bugpoint` call. +For more about the options available, check ``help bugpoint`` or +:doc:`/cmd/bugpoint`. - + use the ``bugpoint_keep`` attribute on objects you don't want to be - removed, usually because you already know they are related to the failure - + ``(* bugpoint_keep *)`` in Verilog, ``attribute \bugpoint_keep 1`` in - RTLIL, or ``setattr -set bugpoint_keep 1 [selection]`` from script +.. code-block:: yoscrypt + :caption: ```` template script -- ``-runner ""`` can allow running ``yosys`` wrapped by another command -- can also use `setenv` before `bugpoint` to set environment variables for - the spawned processes (e.g. ``setenv UBSAN_OPTIONS halt_on_error=1``) + # Load design + read_rtlil design.il + ## OR + script + + # Call bugpoint with failure + bugpoint -script -grep "" + + # Save minimized design + write_rtlil min.il + +.. note:: + + Using ``-grep ""`` with `bugpoint` is optional, but helps to ensure + that the minimized design is reproducing the right error, especially when + ```` contains more than one command. Unfortunately this does not + work with runtime errors such as a ``SEGFAULT`` as it is only able to match + strings from the log file. + +.. TODO:: Consider checking ``run_command`` return value for runtime errors. + + Currently ``BugpointPass::run_yosys`` returns ``run_command(yosys_cmdline) == + 0``, so it shouldn't be too hard to add an option for it. Could also be + used with the ``-runner`` option, which might give it a bit more flexibility. + +By default, `bugpoint` is able to remove any part of the design. In order to +keep certain parts, for instance because you already know they are related to +the failure, you can use the ``bugpoint_keep`` attribute. This can be done with +``(* bugpoint_keep *)`` in Verilog, ``attribute \bugpoint_keep 1`` in RTLIL, or +``setattr -set bugpoint_keep 1 [selection]`` from a Yosys script. It is also +possible to limit `bugpoint` to only removing certain *kinds* of objects, such +as only removing entire modules or cells (instances of modules). For more about +the options available, check ``help bugpoint`` or :doc:`/cmd/bugpoint`. + +In some situations, it may also be helpful to use `setenv` before `bugpoint` to +set environment variables for the spawned processes. An example of this is +``setenv UBSAN_OPTIONS halt_on_error=1`` for where you are trying to raise an +error on undefined behaviour but only want the child process to halt on error. .. note:: @@ -140,6 +182,24 @@ How do I use bugpoint? ``ABC`` because they are only read on start-up. While others, such as ``YOSYS_NOVERIFIC`` and ``HOME``, are evaluated each time they are used. +Once you have finished configuration, you can now run ``yosys ``. +The first thing `bugpoint` will do is test the input design fails. If it +doesn't, make sure you are using the right ``yosys`` executable; unless the +``-yosys`` option is provided, it will use whatever the shell defaults to. If +you are using the ``-runner`` option, try replacing the `bugpoint` command with +``write_rtlil test.il`` and then on a new line, ``! yosys -s + test.il`` to check it works as expected and returns a non-zero +status. + +Depending on the size of your design, and the length of your ````, +`bugpoint` may take some time; remember, it will run ``yosys -s `` +on each iteration of the design. The bigger the design, the more iterations. +The longer the ````, the longer each iteration will take. As the +design shrinks and `bugpoint` converges, each iteration should take less and +less time. Once all simplifications are exhausted and there are no more objects +that can be removed, the script will continue and the minimized design can be +saved. + What do I do with the minimized design? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From 9fa1f76cf2b364a825d98a0b9ef672f059965a94 Mon Sep 17 00:00:00 2001 From: Krystine Sherwin <93062060+KrystalDelusion@users.noreply.github.com> Date: Tue, 5 Aug 2025 09:53:59 +1200 Subject: [PATCH 12/27] bugpoint.rst: Why context matters (bullets) --- docs/source/using_yosys/bugpoint.rst | 50 ++++++++++++++++++++++++++++ 1 file changed, 50 insertions(+) diff --git a/docs/source/using_yosys/bugpoint.rst b/docs/source/using_yosys/bugpoint.rst index 7d38124bf..54559de32 100644 --- a/docs/source/using_yosys/bugpoint.rst +++ b/docs/source/using_yosys/bugpoint.rst @@ -230,6 +230,10 @@ Minimizing scripts ------------------ - reminder to back up original code before modifying it + + + sometimes over-minimizing scripts can hide underlying issues, so maintaining + the original context is important + - if you're using command line, convert it to a script - if you're using one of the :doc:`/using_yosys/synthesis/synth`, replace it with its contents @@ -370,6 +374,18 @@ Identifying issues to a right shift (or `$shr`)? + is the issue tied to specific parameters, widths, or values? +- if the failing command was part of a larger script, such as one of the + :doc:`/using_yosys/synthesis/synth`, you could try to follow the design + through the script + + + sometimes when a command is raising an error, you're seeing a symptom rather + than the underlying issue + + an earlier command may be putting the design in an invalid state which isn't + picked up until the error is raised + + check out :ref:`using_yosys/bugpoint:Why context matters` + + if you're using a fuzzer to find issues in Yosys, you should be prepared to + do this step + - if you're familiar with C/C++ you might try to have a look at the source code of the command that's failing @@ -404,6 +420,40 @@ Identifying issues for :ref:`using_yosys/bugpoint:creating an issue on github` +Why context matters +------------------- + +- if you did `minimize your script`_, and removed commands prior to the failure + to get a smaller design, try to work backwards and find which commands may + have contributed to the design failing +- especially important when the bug is happening inside of a ``synth_*`` script +- example (#4590) + + + say you did all the minimization and found that the error occurs when a call + to ``techmap -map +/xilinx/cells_map.v`` with ``MIN_MUX_INPUTS`` defined + parses a `$_MUX16_` with all inputs set to ``1'x`` + + step through the original script, calling `stat` after each step to find + when the `$_MUX16_` is added + + find that the `$_MUX16_` is introduced by a call to `muxcover`, but all the + inputs are defined, so calling `techmap` now works as expected + + * and from running `bugpoint` with the failing techmap you know that the + cell with index ``2297`` will fail, so you can now call ``select + top/*$2297`` to limit to just that cell, and optionally call ``design + -save pre_bug`` or ``write_rtlil -selected pre_bug.il`` to save this state + + + next you step through the remaining commands and call `dump` after each to + find when the inputs are disconnected + + find that ``opt -full`` has optimized away portions of the circuit, leading + to `opt_expr` setting the undriven mux inputs to ``x``, but failing to + remove the now unnecessary `$_MUX16_` + +- in this example, you might've stopped with the minimal reproducer, fixed the + bug in ``+/xilinx/cells_map.v``, and carried on +- but by following the failure back you've managed to identify a problem with + `opt_expr` that could be causing other problems either now or in the future + + Creating an issue on GitHub --------------------------- From e776f1dca20d406097e47b230e247a23b4d59ddf Mon Sep 17 00:00:00 2001 From: Krystine Sherwin <93062060+KrystalDelusion@users.noreply.github.com> Date: Tue, 5 Aug 2025 09:53:59 +1200 Subject: [PATCH 13/27] bugpoint.rst: More paragraphs What do I do with the minimized design and (the first half of) Minimizing scripts --- docs/source/using_yosys/bugpoint.rst | 107 ++++++++++++++++++--------- 1 file changed, 74 insertions(+), 33 deletions(-) diff --git a/docs/source/using_yosys/bugpoint.rst b/docs/source/using_yosys/bugpoint.rst index 54559de32..80694fd2d 100644 --- a/docs/source/using_yosys/bugpoint.rst +++ b/docs/source/using_yosys/bugpoint.rst @@ -204,9 +204,10 @@ saved. What do I do with the minimized design? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- check minimized design still fails, especially if not using `write_rtlil` -- e.g. if you ran :ref:`bugpoint_script` below, then calling ``yosys -s - min.v`` should still fail in the same way +First off, check the minimized design still fails. This is especially important +if you're not using `write_rtlil` to output the minimized design. For example, +if you ran :ref:`bugpoint_script` below, then calling ``yosys -s +min.v`` should still fail in the same way. .. code-block:: yoscrypt :caption: example `bugpoint` minimizer @@ -216,12 +217,21 @@ What do I do with the minimized design? bugpoint -script write_verilog min.v -- `write_rtlil` is more reliable since `bugpoint` will have run that exact - code through the failing script; other ``write_*`` commands convert from the - RTLIL and then back again during the ``read_*`` which can result in - differences which mean the design no longer fails -- check out :ref:`using_yosys/bugpoint:identifying issues` for more on what to - do next +The `write_rtlil` command is generally more reliable, since `bugpoint` will have +run that exact code through the failing script. Other ``write_*`` commands +convert from the RTLIL and then back again during the ``read_*`` which can +result in differences which mean the design no longer fails. + +.. note:: + + Simply calling Yosys with the output of ``write_*``, as in ``yosys -s + min.v``, does not guarantee that the corresponding ``read_*`` + will be used. For more about this, refer to + :doc:`/using_yosys/more_scripting/load_design`, or load the design explicitly + with ``yosys -p 'read_verilog min.v' -s ``. + +Once you've verified the failure still happens, check out +:ref:`using_yosys/bugpoint:identifying issues` for more on what to do next. .. _minimize your script: @@ -229,19 +239,43 @@ What do I do with the minimized design? Minimizing scripts ------------------ -- reminder to back up original code before modifying it +If you're using a command line prompt, such as ``yosys -p 'synth_xilinx' -o +design.json design.v``, consider converting it to a script. It's generally much +easier to iterate over changes to a script in a file rather than one on the +command line, as well as being better for sharing with others. - + sometimes over-minimizing scripts can hide underlying issues, so maintaining - the original context is important +.. code-block:: yoscrypt + :caption: example script, ``script.ys``, for prompt ``yosys -p 'synth_xilinx' -o design.json design.v`` -- if you're using command line, convert it to a script -- if you're using one of the :doc:`/using_yosys/synthesis/synth`, replace it - with its contents + read_verilog design.v + synth_xilinx + write_json design.json - + can also do this piece-wise with the ``-run`` option - + e.g. replacing ``synth -top -lut`` with :ref:`replace_synth` - + the options ``-top -lut`` can be provided to each `synth` step, or - to just the step(s) where it is relevant, as done here +Next up you want to remove everything *after* the error occurs. Using the +``-L`` flag can help here, allowing you to specify a file to log to, such as +``yosys -L out.log -s script.ys``. Most commands will print a header message +when they begin; something like ``2.48. Executing HIERARCHY pass (managing +design hierarchy).`` The last header message will usually be the failing +command. There are some commands which don't print a header message, so you may +want to add ``echo on`` to the start of your script. The `echo` command echoes +each command executed, along with any arguments given to it. For the +`hierarchy` example above this might be ``yosys> hierarchy -check``. + +.. note:: + + It may also be helpful to use the `log` command to add messages which you can + then search for either in the terminal or the logfile. This can be quite + useful if your script contains script-passes, like the + :doc:`/using_yosys/synthesis/synth`, which call many sub-commands and you're + not sure exactly which script-pass is calling the failing command. + +If your final command calls sub-commands, replace it with its contents and +repeat the previous step. In the case of the +:doc:`/using_yosys/synthesis/synth`, as well as certain other script-passes, you +can use the ``-run`` option to simplify this. For example we can replace +``synth -top -lut`` with the :ref:`replace_synth`. The options ``-top + -lut`` can be provided to each `synth` step, or to just the step(s) where +it is relevant, as done here. .. code-block:: yoscrypt :caption: example replacement script for `synth` command @@ -252,20 +286,12 @@ Minimizing scripts synth -lut -run fine:check synth -run check: -- remove everything *after* the error occurs -- can use `log` command to print messages to help locate the failure point -- `echo` can also help (``echo on``) - - + if you used a ``-run`` option like in :ref:`replace_synth` above, you can - now replace the failing step with its contents and repeat the above if - needed - + checking the log should tell you the last command that ran which can make - this easier - + say we ran :ref:`replace_synth` and were able to remove the ``synth -run - check:`` and still got our error, then we check the log and we see the last - thing before the error was ``7.2. Executing MEMORY_MAP pass (converting - memories to logic and flip-flops).`` - + we can then update our script to the following: +Say we ran :ref:`replace_synth` and were able to remove the ``synth -run +check:`` and still got our error, then we check the log and we see the last +thing before the error was ``7.2. Executing MEMORY_MAP pass (converting memories +to logic and flip-flops)``. By checking the output of ``yosys -h synth`` (or the +`synth` help page) we can see that the `memory_map` pass is called in the +``fine`` step. We can then update our script to the following: .. code-block:: yoscrypt :caption: example replacement script for `synth` when `memory_map` is failing @@ -274,6 +300,21 @@ Minimizing scripts opt -fast -full memory_map +By giving `synth` the option ``-run :fine``, we are telling it to run from the +beginning of the script until the ``fine`` step, where we then give it the exact +commands to run. There are some cases where the commands given in the help +output are not an exact match for what is being run, but are instead a +simplification. If you find that replacing the script-pass with its contents +causes the error to disappear, or change, try calling the script-pass with +``echo on`` to see exactly what commands are being called and what options are +used. + +.. warning:: + + Before continuing further, *back up your code*. The following steps can + remove context and lead to over-minimizing scripts, hiding underlying issues. + Check out :ref:`using_yosys/bugpoint:Why context matters` to learn more. + - try ``write_rtlil ; design -reset; read_rtlil ;`` before the failure point From c75b07820ffbd5f317a88c94e7bae77d2e4e63f4 Mon Sep 17 00:00:00 2001 From: Krystine Sherwin <93062060+KrystalDelusion@users.noreply.github.com> Date: Tue, 5 Aug 2025 09:53:59 +1200 Subject: [PATCH 14/27] Docs: More bugpoint bullets More info for creating GitHub issues and the different sections. Discuss additional details that can be included as comments on the issue. Also mention Gists for large files (preferable to downloading a .txt). Add a warning about external plugins/tools. Also add a note to `load_design.rst` about `Frontend`s and `-f` command line option. --- docs/source/using_yosys/bugpoint.rst | 102 ++++++++++++++++-- .../more_scripting/load_design.rst | 4 + 2 files changed, 100 insertions(+), 6 deletions(-) diff --git a/docs/source/using_yosys/bugpoint.rst b/docs/source/using_yosys/bugpoint.rst index 80694fd2d..2316510cf 100644 --- a/docs/source/using_yosys/bugpoint.rst +++ b/docs/source/using_yosys/bugpoint.rst @@ -498,10 +498,27 @@ Why context matters Creating an issue on GitHub --------------------------- -- "Reproduction Steps" is ideally a code-block (starting and ending with triple - backquotes) containing the minimized design (Verilog or RTLIL), followed by a - code-block containing the minimized yosys script OR a command line call to - yosys with code-formatting (starting and ending with single backquotes) +- use the `bug report template`_ + +.. _bug report template: https://github.com/YosysHQ/yosys/issues/new?template=bug_report.yml + +- short title briefly describing the issue, e.g. + + techmap of wide mux with undefined inputs raises error during synth_xilinx + + + tells us what's happening ("raises error") + + gives the command affected (`techmap`) + + an overview of the input design ("wide mux with undefined inputs") + + and some context where it was found ("during `synth_xilinx`") + + +Reproduction Steps +~~~~~~~~~~~~~~~~~~ + +- ideally a code-block (starting and ending with triple backquotes) containing + the minimized design (Verilog or RTLIL), followed by a code-block containing + the minimized yosys script OR a command line call to yosys with + code-formatting (starting and ending with single backquotes) .. code-block:: markdown @@ -535,5 +552,78 @@ Creating an issue on GitHub # minimum sequence of commands ``` -- any environment variables or command line options should also be mentioned in - the "Reproduction Steps" +- any environment variables or command line options should also be mentioned +- if the problem occurs for a range of values/designs, what is that range +- if you're using an external tool, such as ``valgrind``, to detect the issue, + what version of that tool are you using and what options are you giving it + +.. warning:: + + Please try to avoid the use of any external plugins/tools in the reproduction + steps if they are not directly related to the issue being raised. This + includes frontend plugins such as GHDL or slang; use `write_rtlil` on the + minimized design instead. This also includes tools which provide a wrapper + around Yosys such as OpenLane; you should instead minimize your input and + reproduction steps to just the Yosys part. + +"Expected Behaviour" +~~~~~~~~~~~~~~~~~~~~ + +- if you have a similar design/script that doesn't give the error, include it + here as a reference +- if the bug is that an error *should* be raised but isn't, are there any other + commands with similar error messages + + +"Actual Behaviour" +~~~~~~~~~~~~~~~~~~ + +- any error messages go here +- any details relevant to the crash that were found with ``--trace`` or + ``--debug`` flags +- if you identified the point of failure in the source code, you could mention + it here, or as a comment below + + + if possible, use a permalink to the source on GitHub + + you can browse the source repository for a certain commit with the failure + and open the source file, select the relevant lines (click on the line + number for the first relevant line, then while holding shift click on the + line number for the last relevant line), click on the `...` that appears and + select "Copy permalink" + + should look something like + ``https://github.com/YosysHQ/yosys/blob//path/to/file#L139-L147`` + + clicking on "Preview" should reveal a code block containing the lines of + source specified, with a link to the source file at the given commit + + +Additional details +~~~~~~~~~~~~~~~~~~ + +- once you have created the issue, any additional details can be added as a + comment on that issue +- could include any additional context as to what you were doing when you first + encountered the bug +- was this issue discovered through the use of a fuzzer +- if you've minimized the script, consider including the `bugpoint` script you + used, or the original script, e.g. + +.. code-block:: markdown + + Minimized with + ``` + read_verilog design.v + # original sequence of commands prior to error + bugpoint -script -grep "" + write_rtlil min.il + ``` + + OR + + Minimized from + `yosys -p ': original sequence of commands to produce error;' design.v` + +- if you're able to, it may also help to share the original un-minimized design + + + if the design is too big for a comment, consider turning it into a `Gist`_ + +.. _Gist: https://gist.github.com/ diff --git a/docs/source/using_yosys/more_scripting/load_design.rst b/docs/source/using_yosys/more_scripting/load_design.rst index 1c597bdfc..7e417eff9 100644 --- a/docs/source/using_yosys/more_scripting/load_design.rst +++ b/docs/source/using_yosys/more_scripting/load_design.rst @@ -68,6 +68,10 @@ Yosys frontends + executed as multiple successive calls to the frontend +- compatible with ``-f`` command line option, e.g. ``yosys -f verilog + design.txt`` will use the `read_verilog` frontend with the input file + ``design.txt`` + - `verific` and `read` commands are technically not 'Frontends', but their behaviour is kept in sync From 113a6f6e525f3fc98bac64c8fd6b65ae08ff12dd Mon Sep 17 00:00:00 2001 From: Krystine Sherwin <93062060+KrystalDelusion@users.noreply.github.com> Date: Tue, 5 Aug 2025 09:54:00 +1200 Subject: [PATCH 15/27] bugpoint.rst: yosys -h bugpoint does work I just missed that it only gets included in the makefile if `DISABLE_SPAWN` is set, because I was looking for the C define `YOSYS_DISABLE_SPAWN`. --- docs/source/using_yosys/bugpoint.rst | 35 ++++++++++++---------------- 1 file changed, 15 insertions(+), 20 deletions(-) diff --git a/docs/source/using_yosys/bugpoint.rst b/docs/source/using_yosys/bugpoint.rst index 2316510cf..e7bbfa665 100644 --- a/docs/source/using_yosys/bugpoint.rst +++ b/docs/source/using_yosys/bugpoint.rst @@ -49,24 +49,10 @@ The first thing to be aware of is that `bugpoint` is not available in every build of Yosys. Because the command works by invoking external processes, it requires that Yosys can spawn executables. Notably this means `bugpoint` is not able to be used in WebAssembly builds such as that available via YoWASP. The -easiest way to check your build of Yosys is by running ``yosys -qq -p '!echo -test'``. If this echoes ``test`` in the console, then `bugpoint` will work as -expected. If instead if it displays the text ``ERROR: Shell is not available.`` -then `bugpoint` will not work either. - -.. note:: - - The console command ``yosys -qq -p '!echo test'`` uses the ``-qq`` flag to - prevent Yosys from outputting non-error messages to the console. The ``-p`` - option executes ``!echo test`` as a Yosys command, attempting to pass ``echo - test`` to the shell for execution. For more about the ``-p`` option and - common pitfalls, check out :ref:`getting_started/scripting_intro:script - parsing` in the :doc:`/getting_started/index` section. - -.. TODO:: Add ``YOSYS_DISABLE_SPAWN`` check in ``bugpoint.cc``. - - At least in the help text, so that ``yosys -h bugpoint`` will correctly - indicate if the command will work instead of this roundabout method. +easiest way to check your build of Yosys is by running ``yosys -h bugpoint``. If +Yosys displays the help text for `bugpoint` then it is available for use, but if +it instead prints "No such command or cell type: bugpoint", then `bugpoint` is +not available. Next you need to separate loading the design from the failure point; you should be aiming to reproduce the failure by running ``yosys -s -s @@ -315,6 +301,15 @@ used. remove context and lead to over-minimizing scripts, hiding underlying issues. Check out :ref:`using_yosys/bugpoint:Why context matters` to learn more. +When a problem is occurring many steps into a script, minimizing the design at +the start of the script isn't always enough to identify the cause of the issue. +Each extra step of the script can lead to larger sections of the input design +being needed for the specific problem to be preserved until it causes a crash. +So to find the smallest possible reproducer it can sometimes be helpful to +remove commands prior to the failure point. + + + - try ``write_rtlil ; design -reset; read_rtlil ;`` before the failure point @@ -588,8 +583,8 @@ Reproduction Steps + you can browse the source repository for a certain commit with the failure and open the source file, select the relevant lines (click on the line number for the first relevant line, then while holding shift click on the - line number for the last relevant line), click on the `...` that appears and - select "Copy permalink" + line number for the last relevant line), click on the ``...`` that appears + and select "Copy permalink" + should look something like ``https://github.com/YosysHQ/yosys/blob//path/to/file#L139-L147`` + clicking on "Preview" should reveal a code block containing the lines of From 20a573953cbb483aec74e4f66d0cabe3463679c3 Mon Sep 17 00:00:00 2001 From: Krystine Sherwin <93062060+KrystalDelusion@users.noreply.github.com> Date: Tue, 5 Aug 2025 09:54:00 +1200 Subject: [PATCH 16/27] bugpoint.rst: Minimizing scripts part 2: electric boogaloo --- docs/source/using_yosys/bugpoint.rst | 66 ++++++++++++++++++---------- 1 file changed, 44 insertions(+), 22 deletions(-) diff --git a/docs/source/using_yosys/bugpoint.rst b/docs/source/using_yosys/bugpoint.rst index e7bbfa665..ca54bc8f0 100644 --- a/docs/source/using_yosys/bugpoint.rst +++ b/docs/source/using_yosys/bugpoint.rst @@ -308,38 +308,60 @@ being needed for the specific problem to be preserved until it causes a crash. So to find the smallest possible reproducer it can sometimes be helpful to remove commands prior to the failure point. +The simplest way to do this is by writing out the design, resetting the current +state, and reading back the design: +.. code-block:: yoscrypt -- try ``write_rtlil ; design -reset; read_rtlil ;`` before - the failure point + write_rtlil ; design -reset; read_rtlil ; - + ideally you now have a single command that is producing an error and can - `minimize your RTLIL`_ with the ```` output - + if not, try to move the write/reset/read earlier in the script until you can - reproduce the error - + if you have no idea where exactly you should put the reset, the best way is - to use a "binary search" type approach, reducing the possible options by - half after each attempt +In most cases, this can be inserted immediately before the failing command while +still producing the error, allowing you to `minimize your RTLIL`_ with the +```` output. For our previous example with `memory_map`, if +:ref:`map_reset` still gives the same error, then we should now be able to call +``yosys design.il -p 'memory_map'`` to reproduce it. - * for example, your script has 16 commands in it before failing on the 17th - * if resetting immediately before the 17th doesn't reproduce the error, try - between the 8th and 9th (8 is half of the total 16) - * if that produces the error then you can remove everything before the - `read_rtlil` and try reset again in the middle of what's left, making sure - to use a different name for the output file so that you don't overwrite - what you've already got - * if the error isn't produced then you need to go earlier still, so in this - case you would do between the 4th and 5th (4 is half of the previous 8) - * repeat this until you can't reduce the remaining commands any further +.. code-block:: yoscrypt + :caption: resetting the design immediately before failure + :name: map_reset + + synth -top -run :fine + opt -fast -full + write_rtlil design.il; design -reset; read_rtlil design.il; + memory_map + +If that doesn't give the error (or doesn't give the same error), then you should +try to move the write/reset/read earlier in the script until it does. If you +have no idea where exactly you should put the reset, the best way is to use a +"binary search" type approach, reducing the possible options by half after each +attempt. + + As an example, your script has 16 commands in it before failing on the 17th. + If resetting immediately before the 17th doesn't reproduce the error, try + between the 8th and 9th (8 is half of the total 16). If that produces the + error then you can remove everything before the `read_rtlil` and try reset + again in the middle of what's left, making sure to use a different name for + the output file so that you don't overwrite what you've already got. If the + error isn't produced then you need to go earlier still, so in this case you + would do between the 4th and 5th (4 is half of the previous 8). Repeat this + until you can't reduce the remaining commands any further. .. TODO:: is it possible to dump scratchpad? is there anything else in the yosys/design state that doesn't get included in `write_rtlil`? -- you can also try to remove or comment out commands prior to the failing - command; just because the first and last commands are needed doesn't mean that - every command between them is +A more conservative, but more involved, method is to remove or comment out +commands prior to the failing command. Each command, or group of commands, can +be disabled one at a time while checking if the error still occurs, eventually +giving the smallest subset of commands needed to take the original input through +to the error. The difficulty with this method is that depending on your design, +some commands may be necessary even if they aren't needed to reproduce the +error. For example, if your design includes ``process`` blocks, many commands +will fail unless you run the `proc` command. While this approach can do a +better job of maintaining context, it is often easier to *recover* the context +after the design has been minimized for producing the error. For more on +recovering context, checkout :ref:`using_yosys/bugpoint:Why context matters`. Minimizing Verilog designs From f0b4f7012e689fcf98b991075e3cf67365504d5b Mon Sep 17 00:00:00 2001 From: Krystine Sherwin <93062060+KrystalDelusion@users.noreply.github.com> Date: Tue, 5 Aug 2025 09:54:00 +1200 Subject: [PATCH 17/27] bugpoint.rst: Extra notes Move `yosys -h bugpoint` failure into a code-block to break up text. Same for the `exec -expect-return` example. TODOs on #5068 being merged. --- docs/source/using_yosys/bugpoint.rst | 36 ++++++++++++++++++++-------- 1 file changed, 26 insertions(+), 10 deletions(-) diff --git a/docs/source/using_yosys/bugpoint.rst b/docs/source/using_yosys/bugpoint.rst index ca54bc8f0..6f58e3d5b 100644 --- a/docs/source/using_yosys/bugpoint.rst +++ b/docs/source/using_yosys/bugpoint.rst @@ -24,6 +24,13 @@ Minimizing failing (or bugged) designs .. _iverilog: https://steveicarus.github.io/iverilog/ .. _verilator: https://www.veripool.org/verilator/ +- are there any warnings before the error (either immediately before or in an + earlier command) that could be related? +- does calling `check` before the failure give any errors or warnings? +- did you call `hierarchy` before the failure? + + + can you call ``hierarchy -check``? + - make sure to back up your code (design source and yosys script(s)) before making any modifications @@ -50,9 +57,15 @@ build of Yosys. Because the command works by invoking external processes, it requires that Yosys can spawn executables. Notably this means `bugpoint` is not able to be used in WebAssembly builds such as that available via YoWASP. The easiest way to check your build of Yosys is by running ``yosys -h bugpoint``. If -Yosys displays the help text for `bugpoint` then it is available for use, but if -it instead prints "No such command or cell type: bugpoint", then `bugpoint` is -not available. +Yosys displays the help text for `bugpoint` then it is available for use. + +.. code-block:: console + :caption: `bugpoint` is unavailable + + $ yosys -h bugpoint + + -- Running command `help bugpoint' -- + No such command or cell type: bugpoint Next you need to separate loading the design from the failure point; you should be aiming to reproduce the failure by running ``yosys -s -s @@ -87,8 +100,11 @@ making use of the `exec` command in ````. This is especially useful when Yosys is outputting an invalid design, or when some other tool is incompatible with the design. Be sure to use the ``exec -expect-*`` options so that the pass/fail can be detected correctly. Multiple calls to `exec` can be -made, or even entire shell scripts (e.g. ``exec -expect-return 1 -- bash -``). +made, or even entire shell scripts: + +.. code-block:: yoscrypt + + exec -expect-return 1 --bash Our final failure we can use with `bugpoint` is one returned by a wrapper process, such as ``valgrind`` or ``timeout``. In this case you will be calling @@ -98,6 +114,8 @@ leak or excessive runtime. Note however that unlike the `exec` command, there is currently no way to check the return status or messages from the wrapper process; only a binary pass/fail. +.. TODO:: above note pending updated bugpoint #5068 + How do I use bugpoint? ~~~~~~~~~~~~~~~~~~~~~~ @@ -140,11 +158,7 @@ For more about the options available, check ``help bugpoint`` or work with runtime errors such as a ``SEGFAULT`` as it is only able to match strings from the log file. -.. TODO:: Consider checking ``run_command`` return value for runtime errors. - - Currently ``BugpointPass::run_yosys`` returns ``run_command(yosys_cmdline) == - 0``, so it shouldn't be too hard to add an option for it. Could also be - used with the ``-runner`` option, which might give it a bit more flexibility. +.. TODO:: above note pending updated bugpoint #5068 By default, `bugpoint` is able to remove any part of the design. In order to keep certain parts, for instance because you already know they are related to @@ -177,6 +191,8 @@ you are using the ``-runner`` option, try replacing the `bugpoint` command with test.il`` to check it works as expected and returns a non-zero status. +.. TODO:: note on ``!`` (link to :ref:`getting_started/scripting_intro:script parsing`) + Depending on the size of your design, and the length of your ````, `bugpoint` may take some time; remember, it will run ``yosys -s `` on each iteration of the design. The bigger the design, the more iterations. From 65b75049aa5da7683e576a2c7165f30c1560e885 Mon Sep 17 00:00:00 2001 From: Krystine Sherwin <93062060+KrystalDelusion@users.noreply.github.com> Date: Tue, 5 Aug 2025 09:54:00 +1200 Subject: [PATCH 18/27] docs: Shuffling bug reporting guidelines Move the "creating an issue" section from bugpoint.rst to "reporting bugs" in `contributing.rst`. Fix link to `CONTRIBUTING.md`. Update `CONTRIBUTING.md` to refer to the bugpoint guide instead of the stack overflow guide. --- CONTRIBUTING.md | 16 +-- docs/source/using_yosys/bugpoint.rst | 136 +----------------- .../extending_yosys/contributing.rst | 136 +++++++++++++++++- 3 files changed, 144 insertions(+), 144 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 6c4376cc4..74f9ab10d 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -19,14 +19,14 @@ much easier for someone to respond and help. ### Bug reports -Before you submit an issue, please have a search of the existing issues in case -one already exists. Making sure that you have a minimal, complete and -verifiable example (MVCE) is a great way to quickly check an existing issue -against a new one. Stack overflow has a guide on [how to create an -MVCE](https://stackoverflow.com/help/minimal-reproducible-example). The -[`bugpoint` -command](https://yosyshq.readthedocs.io/projects/yosys/en/latest/cmd/bugpoint.html) -in Yosys can be helpful for this process. +Before you submit an issue, please check out the [how-to guide for +`bugpoint`](https://yosys.readthedocs.io/en/latest/using_yosys/bugpoint.html). +This guide will take you through the process of using the [`bugpoint` +command](https://yosys.readthedocs.io/en/latest/cmd/bugpoint.html) in Yosys to +produce a [minimal, complete and verifiable +example](https://stackoverflow.com/help/minimal-reproducible-example) (MVCE). +Providing an MVCE with your bug report drastically increases the likelihood that +someone will be able to help resolve your issue. # Using pull requests diff --git a/docs/source/using_yosys/bugpoint.rst b/docs/source/using_yosys/bugpoint.rst index 6f58e3d5b..89c58a8f4 100644 --- a/docs/source/using_yosys/bugpoint.rst +++ b/docs/source/using_yosys/bugpoint.rst @@ -491,7 +491,7 @@ Identifying issues .. _the existing issues: https://github.com/YosysHQ/yosys/issues - if there are no existing or related issues already, then check out the steps - for :ref:`using_yosys/bugpoint:creating an issue on github` + for :ref:`yosys_internals/extending_yosys/contributing:reporting bugs` Why context matters @@ -526,137 +526,3 @@ Why context matters bug in ``+/xilinx/cells_map.v``, and carried on - but by following the failure back you've managed to identify a problem with `opt_expr` that could be causing other problems either now or in the future - - -Creating an issue on GitHub ---------------------------- - -- use the `bug report template`_ - -.. _bug report template: https://github.com/YosysHQ/yosys/issues/new?template=bug_report.yml - -- short title briefly describing the issue, e.g. - - techmap of wide mux with undefined inputs raises error during synth_xilinx - - + tells us what's happening ("raises error") - + gives the command affected (`techmap`) - + an overview of the input design ("wide mux with undefined inputs") - + and some context where it was found ("during `synth_xilinx`") - - -Reproduction Steps -~~~~~~~~~~~~~~~~~~ - -- ideally a code-block (starting and ending with triple backquotes) containing - the minimized design (Verilog or RTLIL), followed by a code-block containing - the minimized yosys script OR a command line call to yosys with - code-formatting (starting and ending with single backquotes) - -.. code-block:: markdown - - min.v - ```verilog - // minimized Verilog design - ``` - - min.ys - ``` - read_verilog min.v - # minimum sequence of commands to reproduce error - ``` - - OR - - `yosys -p ': minimum sequence of commands;' min.v` - -- alternatively can provide a single code-block which includes the minimized - design as a "here document" followed by the sequence of commands which - reproduce the error - - + see :doc:`/using_yosys/more_scripting/load_design` for more on heredocs. - -.. code-block:: markdown - - ``` - read_rtlil </path/to/file#L139-L147`` - + clicking on "Preview" should reveal a code block containing the lines of - source specified, with a link to the source file at the given commit - - -Additional details -~~~~~~~~~~~~~~~~~~ - -- once you have created the issue, any additional details can be added as a - comment on that issue -- could include any additional context as to what you were doing when you first - encountered the bug -- was this issue discovered through the use of a fuzzer -- if you've minimized the script, consider including the `bugpoint` script you - used, or the original script, e.g. - -.. code-block:: markdown - - Minimized with - ``` - read_verilog design.v - # original sequence of commands prior to error - bugpoint -script -grep "" - write_rtlil min.il - ``` - - OR - - Minimized from - `yosys -p ': original sequence of commands to produce error;' design.v` - -- if you're able to, it may also help to share the original un-minimized design - - + if the design is too big for a comment, consider turning it into a `Gist`_ - -.. _Gist: https://gist.github.com/ diff --git a/docs/source/yosys_internals/extending_yosys/contributing.rst b/docs/source/yosys_internals/extending_yosys/contributing.rst index 69258aa5f..70170fc48 100644 --- a/docs/source/yosys_internals/extending_yosys/contributing.rst +++ b/docs/source/yosys_internals/extending_yosys/contributing.rst @@ -7,7 +7,7 @@ Contributing to Yosys |CONTRIBUTING|_ file. .. |CONTRIBUTING| replace:: :file:`CONTRIBUTING.md` -.. _CONTRIBUTING: https://github.com/YosysHQ/yosys/CONTRIBUTING.md +.. _CONTRIBUTING: https://github.com/YosysHQ/yosys/blob/main/CONTRIBUTING.md Coding Style ------------ @@ -42,3 +42,137 @@ for implicit type casts, always use ``GetSize(foobar)`` instead of ``foobar.size()``. (``GetSize()`` is defined in :file:`kernel/yosys.h`) Use range-based for loops whenever applicable. + + +Reporting bugs +-------------- + +- use the `bug report template`_ + +.. _bug report template: https://github.com/YosysHQ/yosys/issues/new?template=bug_report.yml + +- short title briefly describing the issue, e.g. + + techmap of wide mux with undefined inputs raises error during synth_xilinx + + + tells us what's happening ("raises error") + + gives the command affected (`techmap`) + + an overview of the input design ("wide mux with undefined inputs") + + and some context where it was found ("during `synth_xilinx`") + + +Reproduction Steps +~~~~~~~~~~~~~~~~~~ + +- ideally a code-block (starting and ending with triple backquotes) containing + the minimized design (Verilog or RTLIL), followed by a code-block containing + the minimized yosys script OR a command line call to yosys with + code-formatting (starting and ending with single backquotes) + +.. code-block:: markdown + + min.v + ```verilog + // minimized Verilog design + ``` + + min.ys + ``` + read_verilog min.v + # minimum sequence of commands to reproduce error + ``` + + OR + + `yosys -p ': minimum sequence of commands;' min.v` + +- alternatively can provide a single code-block which includes the minimized + design as a "here document" followed by the sequence of commands which + reproduce the error + + + see :doc:`/using_yosys/more_scripting/load_design` for more on heredocs. + +.. code-block:: markdown + + ``` + read_rtlil </path/to/file#L139-L147`` + + clicking on "Preview" should reveal a code block containing the lines of + source specified, with a link to the source file at the given commit + + +Additional details +~~~~~~~~~~~~~~~~~~ + +- once you have created the issue, any additional details can be added as a + comment on that issue +- could include any additional context as to what you were doing when you first + encountered the bug +- was this issue discovered through the use of a fuzzer +- if you've minimized the script, consider including the `bugpoint` script you + used, or the original script, e.g. + +.. code-block:: markdown + + Minimized with + ``` + read_verilog design.v + # original sequence of commands prior to error + bugpoint -script -grep "" + write_rtlil min.il + ``` + + OR + + Minimized from + `yosys -p ': original sequence of commands to produce error;' design.v` + +- if you're able to, it may also help to share the original un-minimized design + + + if the design is too big for a comment, consider turning it into a `Gist`_ + +.. _Gist: https://gist.github.com/ From c47b533a3da6a6938ea735a19f6c2ba578f80a30 Mon Sep 17 00:00:00 2001 From: Krystine Sherwin <93062060+KrystalDelusion@users.noreply.github.com> Date: Tue, 5 Aug 2025 09:54:01 +1200 Subject: [PATCH 19/27] docs: Split bugpoint.rst into user/developer Minimizing scripts (and more generally identifying root cause) isn't necessary for regular bug reports. Rather, it can be useful for developers working on *fixing* bugs, and also for fuzzers to avoid spam. Minor adjustments to `bugpoint.rst`. Add note to `advanced_bugpoint.rst` about primitives when minimizing scripts. --- docs/source/using_yosys/bugpoint.rst | 225 +----------------- .../extending_yosys/advanced_bugpoint.rst | 208 ++++++++++++++++ .../yosys_internals/extending_yosys/index.rst | 1 + 3 files changed, 217 insertions(+), 217 deletions(-) create mode 100644 docs/source/yosys_internals/extending_yosys/advanced_bugpoint.rst diff --git a/docs/source/using_yosys/bugpoint.rst b/docs/source/using_yosys/bugpoint.rst index 89c58a8f4..82eab3cfa 100644 --- a/docs/source/using_yosys/bugpoint.rst +++ b/docs/source/using_yosys/bugpoint.rst @@ -38,8 +38,6 @@ Minimizing failing (or bugged) designs error while trying to debug it -.. _minimize your RTLIL: - Minimizing RTLIL designs with bugpoint -------------------------------------- @@ -74,12 +72,6 @@ be aiming to reproduce the failure by running ``yosys -s -s Check out the instructions for :ref:`using_yosys/bugpoint:minimizing verilog designs` below. -The commands in ```` only need to be run once, while those in -```` will be run on each iteration of `bugpoint`. If you haven't -already, following the instructions for :ref:`using_yosys/bugpoint:minimizing -scripts` will also help with identifying exactly which commands are needed to -produce the failure and which can be safely moved to the loading script. - .. note:: You should also be able to run the two scripts separately, calling first @@ -236,150 +228,6 @@ Once you've verified the failure still happens, check out :ref:`using_yosys/bugpoint:identifying issues` for more on what to do next. -.. _minimize your script: - -Minimizing scripts ------------------- - -If you're using a command line prompt, such as ``yosys -p 'synth_xilinx' -o -design.json design.v``, consider converting it to a script. It's generally much -easier to iterate over changes to a script in a file rather than one on the -command line, as well as being better for sharing with others. - -.. code-block:: yoscrypt - :caption: example script, ``script.ys``, for prompt ``yosys -p 'synth_xilinx' -o design.json design.v`` - - read_verilog design.v - synth_xilinx - write_json design.json - -Next up you want to remove everything *after* the error occurs. Using the -``-L`` flag can help here, allowing you to specify a file to log to, such as -``yosys -L out.log -s script.ys``. Most commands will print a header message -when they begin; something like ``2.48. Executing HIERARCHY pass (managing -design hierarchy).`` The last header message will usually be the failing -command. There are some commands which don't print a header message, so you may -want to add ``echo on`` to the start of your script. The `echo` command echoes -each command executed, along with any arguments given to it. For the -`hierarchy` example above this might be ``yosys> hierarchy -check``. - -.. note:: - - It may also be helpful to use the `log` command to add messages which you can - then search for either in the terminal or the logfile. This can be quite - useful if your script contains script-passes, like the - :doc:`/using_yosys/synthesis/synth`, which call many sub-commands and you're - not sure exactly which script-pass is calling the failing command. - -If your final command calls sub-commands, replace it with its contents and -repeat the previous step. In the case of the -:doc:`/using_yosys/synthesis/synth`, as well as certain other script-passes, you -can use the ``-run`` option to simplify this. For example we can replace -``synth -top -lut`` with the :ref:`replace_synth`. The options ``-top - -lut`` can be provided to each `synth` step, or to just the step(s) where -it is relevant, as done here. - -.. code-block:: yoscrypt - :caption: example replacement script for `synth` command - :name: replace_synth - - synth -top -run :coarse - synth -lut -run coarse:fine - synth -lut -run fine:check - synth -run check: - -Say we ran :ref:`replace_synth` and were able to remove the ``synth -run -check:`` and still got our error, then we check the log and we see the last -thing before the error was ``7.2. Executing MEMORY_MAP pass (converting memories -to logic and flip-flops)``. By checking the output of ``yosys -h synth`` (or the -`synth` help page) we can see that the `memory_map` pass is called in the -``fine`` step. We can then update our script to the following: - -.. code-block:: yoscrypt - :caption: example replacement script for `synth` when `memory_map` is failing - - synth -top -run :fine - opt -fast -full - memory_map - -By giving `synth` the option ``-run :fine``, we are telling it to run from the -beginning of the script until the ``fine`` step, where we then give it the exact -commands to run. There are some cases where the commands given in the help -output are not an exact match for what is being run, but are instead a -simplification. If you find that replacing the script-pass with its contents -causes the error to disappear, or change, try calling the script-pass with -``echo on`` to see exactly what commands are being called and what options are -used. - -.. warning:: - - Before continuing further, *back up your code*. The following steps can - remove context and lead to over-minimizing scripts, hiding underlying issues. - Check out :ref:`using_yosys/bugpoint:Why context matters` to learn more. - -When a problem is occurring many steps into a script, minimizing the design at -the start of the script isn't always enough to identify the cause of the issue. -Each extra step of the script can lead to larger sections of the input design -being needed for the specific problem to be preserved until it causes a crash. -So to find the smallest possible reproducer it can sometimes be helpful to -remove commands prior to the failure point. - -The simplest way to do this is by writing out the design, resetting the current -state, and reading back the design: - -.. code-block:: yoscrypt - - write_rtlil ; design -reset; read_rtlil ; - -In most cases, this can be inserted immediately before the failing command while -still producing the error, allowing you to `minimize your RTLIL`_ with the -```` output. For our previous example with `memory_map`, if -:ref:`map_reset` still gives the same error, then we should now be able to call -``yosys design.il -p 'memory_map'`` to reproduce it. - -.. code-block:: yoscrypt - :caption: resetting the design immediately before failure - :name: map_reset - - synth -top -run :fine - opt -fast -full - write_rtlil design.il; design -reset; read_rtlil design.il; - memory_map - -If that doesn't give the error (or doesn't give the same error), then you should -try to move the write/reset/read earlier in the script until it does. If you -have no idea where exactly you should put the reset, the best way is to use a -"binary search" type approach, reducing the possible options by half after each -attempt. - - As an example, your script has 16 commands in it before failing on the 17th. - If resetting immediately before the 17th doesn't reproduce the error, try - between the 8th and 9th (8 is half of the total 16). If that produces the - error then you can remove everything before the `read_rtlil` and try reset - again in the middle of what's left, making sure to use a different name for - the output file so that you don't overwrite what you've already got. If the - error isn't produced then you need to go earlier still, so in this case you - would do between the 4th and 5th (4 is half of the previous 8). Repeat this - until you can't reduce the remaining commands any further. - -.. TODO:: is it possible to dump scratchpad? - - is there anything else in the yosys/design state that doesn't get included in - `write_rtlil`? - -A more conservative, but more involved, method is to remove or comment out -commands prior to the failing command. Each command, or group of commands, can -be disabled one at a time while checking if the error still occurs, eventually -giving the smallest subset of commands needed to take the original input through -to the error. The difficulty with this method is that depending on your design, -some commands may be necessary even if they aren't needed to reproduce the -error. For example, if your design includes ``process`` blocks, many commands -will fail unless you run the `proc` command. While this approach can do a -better job of maintaining context, it is often easier to *recover* the context -after the design has been minimized for producing the error. For more on -recovering context, checkout :ref:`using_yosys/bugpoint:Why context matters`. - - Minimizing Verilog designs -------------------------- @@ -393,9 +241,9 @@ Minimizing Verilog designs + if the problem is parameter specific you may be able to change the default parameters so that they match the problematic configuration -- as with `minimize your script`_, if you have no idea what is or is not - relevant, try to follow a "binary search" type approach where you remove (or - comment out) roughly half of what's left at a time +- if you have no idea what is or is not relevant, try to follow a "binary + search" type approach where you remove (or comment out) roughly half of what's + left at a time - focusing on one type of object at a time simplifies the process, removing as many as you can until the error disappears if any of the remaining objects are removed @@ -448,36 +296,13 @@ Identifying issues to a right shift (or `$shr`)? + is the issue tied to specific parameters, widths, or values? -- if the failing command was part of a larger script, such as one of the - :doc:`/using_yosys/synthesis/synth`, you could try to follow the design - through the script - - + sometimes when a command is raising an error, you're seeing a symptom rather - than the underlying issue - + an earlier command may be putting the design in an invalid state which isn't - picked up until the error is raised - + check out :ref:`using_yosys/bugpoint:Why context matters` - + if you're using a fuzzer to find issues in Yosys, you should be prepared to - do this step - -- if you're familiar with C/C++ you might try to have a look at the source - code of the command that's failing - - + even if you can't fix the problem yourself, it can be very helpful for - anyone else investigating if you're able to identify where exactly the - issue is - + if you're using a fuzzer to find issues in Yosys, you should be prepared to - do this step - .. warning:: - In the event that you are unable to identify the root cause of a fuzzer - generated issue, **do not** open more than one issue at a time. You have no - way of being able to tell if multiple fuzzer generated issues are simply - different cases of the same problem, and opening multiple issues for the same - problem means more time is spent on triaging and diagnosing bug reports and - less on fixing the problem. If you are found to be doing this, your issues - may be closed without further investigation. + If you are using a fuzzer to find bugs, follow the instructions for + :doc:`/yosys_internals/extending_yosys/advanced_bugpoint`. **Do not** open + more than one fuzzer generated issue at a time if you can not identify the + root cause. If you are found to be doing this, your issues may be closed + without further investigation. - search `the existing issues`_ and see if someone has already made a bug report @@ -492,37 +317,3 @@ Identifying issues - if there are no existing or related issues already, then check out the steps for :ref:`yosys_internals/extending_yosys/contributing:reporting bugs` - - -Why context matters -------------------- - -- if you did `minimize your script`_, and removed commands prior to the failure - to get a smaller design, try to work backwards and find which commands may - have contributed to the design failing -- especially important when the bug is happening inside of a ``synth_*`` script -- example (#4590) - - + say you did all the minimization and found that the error occurs when a call - to ``techmap -map +/xilinx/cells_map.v`` with ``MIN_MUX_INPUTS`` defined - parses a `$_MUX16_` with all inputs set to ``1'x`` - + step through the original script, calling `stat` after each step to find - when the `$_MUX16_` is added - + find that the `$_MUX16_` is introduced by a call to `muxcover`, but all the - inputs are defined, so calling `techmap` now works as expected - - * and from running `bugpoint` with the failing techmap you know that the - cell with index ``2297`` will fail, so you can now call ``select - top/*$2297`` to limit to just that cell, and optionally call ``design - -save pre_bug`` or ``write_rtlil -selected pre_bug.il`` to save this state - - + next you step through the remaining commands and call `dump` after each to - find when the inputs are disconnected - + find that ``opt -full`` has optimized away portions of the circuit, leading - to `opt_expr` setting the undriven mux inputs to ``x``, but failing to - remove the now unnecessary `$_MUX16_` - -- in this example, you might've stopped with the minimal reproducer, fixed the - bug in ``+/xilinx/cells_map.v``, and carried on -- but by following the failure back you've managed to identify a problem with - `opt_expr` that could be causing other problems either now or in the future diff --git a/docs/source/yosys_internals/extending_yosys/advanced_bugpoint.rst b/docs/source/yosys_internals/extending_yosys/advanced_bugpoint.rst new file mode 100644 index 000000000..32d334581 --- /dev/null +++ b/docs/source/yosys_internals/extending_yosys/advanced_bugpoint.rst @@ -0,0 +1,208 @@ +Identifying the root cause of bugs +================================== + +This document references Yosys internals and is intended for people interested +in solving or investigating Yosys bugs. This also applies if you are using a +fuzzing tool; fuzzers have a tendency to find many variations of the same bug, +so identifying the root cause is important for avoiding issue spam. + +- if the failing command was part of a larger script, such as one of the + :doc:`/using_yosys/synthesis/synth`, you could try to follow the design + through the script + + + sometimes when a command is raising an error, you're seeing a symptom rather + than the underlying issue + + an earlier command may be putting the design in an invalid state which isn't + picked up until the error is raised + + check out :ref:`yosys_internals/extending_yosys/advanced_bugpoint:why context matters` + +- if you're familiar with C/C++ you might try to have a look at the source + code of the command that's failing + + + even if you can't fix the problem yourself, it can be very helpful for + anyone else investigating if you're able to identify where exactly the + issue is + +Minimizing scripts +------------------ + +.. TODO:: disclaimer this is intended as advanced usage, and generally not necessary for bug reports + +If you're using a command line prompt, such as ``yosys -p 'synth_xilinx' -o +design.json design.v``, consider converting it to a script. It's generally much +easier to iterate over changes to a script in a file rather than one on the +command line, as well as being better for sharing with others. + +.. code-block:: yoscrypt + :caption: example script, ``script.ys``, for prompt ``yosys -p 'synth_xilinx' -o design.json design.v`` + + read_verilog design.v + synth_xilinx + write_json design.json + +Next up you want to remove everything *after* the error occurs. Using the +``-L`` flag can help here, allowing you to specify a file to log to, such as +``yosys -L out.log -s script.ys``. Most commands will print a header message +when they begin; something like ``2.48. Executing HIERARCHY pass (managing +design hierarchy).`` The last header message will usually be the failing +command. There are some commands which don't print a header message, so you may +want to add ``echo on`` to the start of your script. The `echo` command echoes +each command executed, along with any arguments given to it. For the +`hierarchy` example above this might be ``yosys> hierarchy -check``. + +.. note:: + + It may also be helpful to use the `log` command to add messages which you can + then search for either in the terminal or the logfile. This can be quite + useful if your script contains script-passes, like the + :doc:`/using_yosys/synthesis/synth`, which call many sub-commands and you're + not sure exactly which script-pass is calling the failing command. + +If your final command calls sub-commands, replace it with its contents and +repeat the previous step. In the case of the +:doc:`/using_yosys/synthesis/synth`, as well as certain other script-passes, you +can use the ``-run`` option to simplify this. For example we can replace +``synth -top -lut`` with the :ref:`replace_synth`. The options ``-top + -lut`` can be provided to each `synth` step, or to just the step(s) where +it is relevant, as done here. + +.. code-block:: yoscrypt + :caption: example replacement script for `synth` command + :name: replace_synth + + synth -top -run :coarse + synth -lut -run coarse:fine + synth -lut -run fine:check + synth -run check: + +Say we ran :ref:`replace_synth` and were able to remove the ``synth -run +check:`` and still got our error, then we check the log and we see the last +thing before the error was ``7.2. Executing MEMORY_MAP pass (converting memories +to logic and flip-flops)``. By checking the output of ``yosys -h synth`` (or the +`synth` help page) we can see that the `memory_map` pass is called in the +``fine`` step. We can then update our script to the following: + +.. code-block:: yoscrypt + :caption: example replacement script for `synth` when `memory_map` is failing + + synth -top -run :fine + opt -fast -full + memory_map + +By giving `synth` the option ``-run :fine``, we are telling it to run from the +beginning of the script until the ``fine`` step, where we then give it the exact +commands to run. There are some cases where the commands given in the help +output are not an exact match for what is being run, but are instead a +simplification. If you find that replacing the script-pass with its contents +causes the error to disappear, or change, try calling the script-pass with +``echo on`` to see exactly what commands are being called and what options are +used. + +.. warning:: + + Before continuing further, *back up your code*. The following steps can + remove context and lead to over-minimizing scripts, hiding underlying issues. + Check out :ref:`yosys_internals/extending_yosys/advanced_bugpoint:Why + context matters` to learn more. + +When a problem is occurring many steps into a script, minimizing the design at +the start of the script isn't always enough to identify the cause of the issue. +Each extra step of the script can lead to larger sections of the input design +being needed for the specific problem to be preserved until it causes a crash. +So to find the smallest possible reproducer it can sometimes be helpful to +remove commands prior to the failure point. + +The simplest way to do this is by writing out the design, resetting the current +state, and reading back the design: + +.. code-block:: yoscrypt + + write_rtlil ; design -reset; read_rtlil ; + +In most cases, this can be inserted immediately before the failing command while +still producing the error, allowing you to :ref:`minimize your +RTLIL` with the +```` output. For our previous example with `memory_map`, if +:ref:`map_reset` still gives the same error, then we should now be able to call +``yosys design.il -p 'memory_map'`` to reproduce it. + +.. code-block:: yoscrypt + :caption: resetting the design immediately before failure + :name: map_reset + + synth -top -run :fine + opt -fast -full + write_rtlil design.il; design -reset; read_rtlil design.il; + memory_map + +If that doesn't give the error (or doesn't give the same error), then you should +try to move the write/reset/read earlier in the script until it does. If you +have no idea where exactly you should put the reset, the best way is to use a +"binary search" type approach, reducing the possible options by half after each +attempt. + +.. note:: + + By default, `write_rtlil` doesn't include platform specific IP blocks and + other primitive cell models which are typically loaded with a ``read_verilog + -lib`` command at the start of the synthesis script. You may have to + duplicate these commands *after* the call to ``design -reset``. It is also + possible to write out *everything* with ``select =*; write_rtlil -selected + ``. + +As an example, your script has 16 commands in it before failing on the 17th. If +resetting immediately before the 17th doesn't reproduce the error, try between +the 8th and 9th (8 is half of the total 16). If that produces the error then +you can remove everything before the `read_rtlil` and try reset again in the +middle of what's left, making sure to use a different name for the output file +so that you don't overwrite what you've already got. If the error isn't +produced then you need to go earlier still, so in this case you would do between +the 4th and 5th (4 is half of the previous 8). Repeat this until you can't +reduce the remaining commands any further. + +A more conservative, but more involved, method is to remove or comment out +commands prior to the failing command. Each command, or group of commands, can +be disabled one at a time while checking if the error still occurs, eventually +giving the smallest subset of commands needed to take the original input through +to the error. The difficulty with this method is that depending on your design, +some commands may be necessary even if they aren't needed to reproduce the +error. For example, if your design includes ``process`` blocks, many commands +will fail unless you run the `proc` command. While this approach can do a +better job of maintaining context, it is often easier to *recover* the context +after the design has been minimized for producing the error. For more on +recovering context, checkout +:ref:`yosys_internals/extending_yosys/advanced_bugpoint:Why context matters`. + + +Why context matters +------------------- + +- if you did minimized your script, and removed commands prior to the failure + to get a smaller design, try to work backwards and find which commands may + have contributed to the design failing +- especially important when the bug is happening inside of a ``synth_*`` script +- example (#4590) + + + say you did all the minimization and found that the error occurs when a call + to ``techmap -map +/xilinx/cells_map.v`` with ``MIN_MUX_INPUTS`` defined + parses a `$_MUX16_` with all inputs set to ``1'x`` + + step through the original script, calling `stat` after each step to find + when the `$_MUX16_` is added + + find that the `$_MUX16_` is introduced by a call to `muxcover`, but all the + inputs are defined, so calling `techmap` now works as expected + + * and from running `bugpoint` with the failing techmap you know that the + cell with index ``2297`` will fail, so you can now call ``select + top/*$2297`` to limit to just that cell, and optionally call ``design + -save pre_bug`` or ``write_rtlil -selected pre_bug.il`` to save this state + + + next you step through the remaining commands and call `dump` after each to + find when the inputs are disconnected + + find that ``opt -full`` has optimized away portions of the circuit, leading + to `opt_expr` setting the undriven mux inputs to ``x``, but failing to + remove the now unnecessary `$_MUX16_` + +- in this example, you might've stopped with the minimal reproducer, fixed the + bug in ``+/xilinx/cells_map.v``, and carried on +- but by following the failure back you've managed to identify a problem with + `opt_expr` that could be causing other problems either now or in the future diff --git a/docs/source/yosys_internals/extending_yosys/index.rst b/docs/source/yosys_internals/extending_yosys/index.rst index 4ee21517b..72843ecd6 100644 --- a/docs/source/yosys_internals/extending_yosys/index.rst +++ b/docs/source/yosys_internals/extending_yosys/index.rst @@ -11,6 +11,7 @@ of interest for developers looking to customise Yosys builds. extensions build_verific functional_ir + advanced_bugpoint contributing test_suites From aa6c6fd2832bfb59acf84e7822aea919b2ac2440 Mon Sep 17 00:00:00 2001 From: Krystine Sherwin <93062060+KrystalDelusion@users.noreply.github.com> Date: Tue, 5 Aug 2025 09:54:01 +1200 Subject: [PATCH 20/27] bugpoint.rst: Some paragraphs on verilog --- docs/source/using_yosys/bugpoint.rst | 57 ++++++++++++++++++---------- 1 file changed, 38 insertions(+), 19 deletions(-) diff --git a/docs/source/using_yosys/bugpoint.rst b/docs/source/using_yosys/bugpoint.rst index 82eab3cfa..a2d0b901e 100644 --- a/docs/source/using_yosys/bugpoint.rst +++ b/docs/source/using_yosys/bugpoint.rst @@ -231,29 +231,47 @@ Once you've verified the failure still happens, check out Minimizing Verilog designs -------------------------- -- manual process -- made easier if the error message is able to identify the source line or name - of the object -- reminder to back up original code before modifying it -- if a specific module is causing the problem, try to set that as the top - module, you can then remove +Unlike RTLIL designs where we can use `bugpoint`, minimizing Verilog designs is +a much more manual, iterative process. Be sure to check any errors or warnings +for messages that might identify source lines or object names that might be +causing the failure, and back up your source code before modifying it. At any +point in the process, you can check for anything that is unused or totally +disconnected (ports, wires, etc) and remove them too. - + if the problem is parameter specific you may be able to change the default - parameters so that they match the problematic configuration +.. note:: -- if you have no idea what is or is not relevant, try to follow a "binary - search" type approach where you remove (or comment out) roughly half of what's - left at a time -- focusing on one type of object at a time simplifies the process, removing as - many as you can until the error disappears if any of the remaining objects are - removed -- periodically check if anything is totally disconnected (ports, wires, etc), if - it is then it can be removed too -- start by removing cells (instances of modules) + If a specific module is causing the problem, try to set that as the top + module instead. Any parameters should have their default values changed to + match the failing usage. - + if a module has no more instances, remove it entirely +As a rule of thumb, try to split things roughly in half at each step; similar to +a "binary search". If you have 10 cells (instances of modules) in your top +module, and have no idea what is causing the issue, split them into two groups +of 5 cells. For each group of cells, try remove them and see if the failure +still happens. If the error still occurs with the first group removed, but +disappears when the second group is removed, then the first group can be safely +removed. If a module has no more instances, remove it entirely. Repeat this +for each remaining group of cells until each group only has 1 cell in it and no +more cells can be removed without making the error disappear. You can also +repeat this for each module still in your design. + +After minimizing the number of cells, do the same for the process blocks in your +top module. And again for any generate blocks and combinational blocks. +Remember to check for any ports or signals which are no longer used and remove +those too. Any signals which are written but never read can also be removed. + +.. note:: + + Depending on where the design is failing, there are some commands which may + help in identifying unused objects in the design. `hierarchy` will identify + which modules are used and which are not, but check for `$paramod` modules + before removing unused ones. ``debug clean`` will list all unused wires in + each module, as well as unused cells which were automatically generated + (giving the line number of the source that generated them). Adding the + ``-purge`` flag will also include named wires that would normally be ignored + by `clean`. Though when there are large numbers of unused wires it is often + easier to just delete sections of the code and see what happens. -- then processes - try to remove or reduce assignments and operations + are there any wires/registers which get read but never written? @@ -267,6 +285,7 @@ Minimizing Verilog designs ``'0`` + if you have enable or reset logic, does the error still happen without that? + can you reduce an ``if .. else`` to a single case? + + can you remove states from a ``case`` block? - if you're planning to share the minimized code: From 6b7756b67a3226cc84d7473add7167d771b3ad30 Mon Sep 17 00:00:00 2001 From: Krystine Sherwin <93062060+KrystalDelusion@users.noreply.github.com> Date: Tue, 5 Aug 2025 09:54:01 +1200 Subject: [PATCH 21/27] bugpoint.rst: Finish paragraphs Update text to assume bugpoint PR changes. --- docs/source/using_yosys/bugpoint.rst | 221 ++++++++++++++++----------- 1 file changed, 131 insertions(+), 90 deletions(-) diff --git a/docs/source/using_yosys/bugpoint.rst b/docs/source/using_yosys/bugpoint.rst index a2d0b901e..38a978695 100644 --- a/docs/source/using_yosys/bugpoint.rst +++ b/docs/source/using_yosys/bugpoint.rst @@ -1,41 +1,62 @@ Minimizing failing (or bugged) designs ====================================== -- how to guide -- assumes knowledge and familiarity with Yosys -- something is wrong with your design OR something is wrong with Yosys +.. TODO:: pending merge of https://github.com/YosysHQ/yosys/pull/5068 - + how to work out which +This document is a how-to guide for reducing problematic designs to the bare +minimum needed for reproducing the issue. This is a Yosys specific alternative +to the Stack Overflow article: `How to create a Minimal, Reproducible Example`_, +and is intended to help when there's something wrong with your design, or with +Yosys itself. -- *read* the error message -- is it a Yosys error? (starts with ERROR:) +.. _How to create a Minimal, Reproducible Example: https://stackoverflow.com/help/minimal-reproducible-example - + does it give you a line number from your design +.. note:: -- is it a runtime error, e.g. SEGFAULT -- are you using the latest release of Yosys + This guide assumes a moderate degree of familiarity with Yosys and requires + some amount of problem solving ability. - + has your problem already been fixed -- is your input design valid? +Before you start +---------------- - + if you're using Verilog, try load it with `iverilog`_ or `verilator`_ +The first (and often overlooked) step, is to check for and *read* any error +messages or warnings. Passing the ``-q`` flag when running Yosys will make it +so that only warnings and error messages are written to the console. Don't just +read the last message either, there may be warnings that indicate a problem +before it happens. While some things may only be regarded as warnings, such as +multiple drivers for the same signal or logic loops, these can cause problems in +some synthesis flows but not others. + +A Yosys error (one that starts with ``ERROR:``) may give you a line number from +your design, or the name of the object causing issues. If so, you may already +have enough information to resolve the problem, or at least understand why it's +happening. + +.. note:: + + If you're not already, try using the latest version from the `Yosys GitHub`_. + You may find that your issue has already been fixed! And even if it isn't, + testing with two different versions is a good way to ensure reproducibility. + +.. _Yosys GitHub: https://github.com/YosysHQ/yosys + +Another thing to be aware of is that Yosys generally doesn't perform rigorous +checking of input designs to ensure they are valid. This is especially true for +the `read_verilog` frontend. It is instead recommended that you try load it +with `iverilog`_ or `verilator`_ first, as an invalid design can often lead to +unexpected issues. .. _iverilog: https://steveicarus.github.io/iverilog/ .. _verilator: https://www.veripool.org/verilator/ -- are there any warnings before the error (either immediately before or in an - earlier command) that could be related? -- does calling `check` before the failure give any errors or warnings? -- did you call `hierarchy` before the failure? - - + can you call ``hierarchy -check``? - -- make sure to back up your code (design source and yosys script(s)) before - making any modifications - - + even if the code itself isn't important, this can help avoid "losing" the - error while trying to debug it +If you're using a custom synthesis script, try take a bit of time to figure out +which command is failing. Calling ``echo on`` at the start of your script will +`echo` each command executed; the last echo before the error should then be +where the error has come from. Check the help message for the failing command; +does it indicate limited support, or mention some other command that needs to be +run first? You can also try to call `check` and/or ``hierarchy -check`` before +the failure to see if they report and errors or warnings. Minimizing RTLIL designs with bugpoint @@ -47,6 +68,12 @@ developed for Yosys crashes, `bugpoint` can also be used for designs that lead to non-fatal errors, or even failures in other tools that use the output of a Yosys script. +.. note:: + + Make sure to back up your code (design source and yosys script(s)) before + making any modifications. Even if the code itself isn't important, this can + help avoid "losing" the error while trying to debug it. + Can I use bugpoint? ~~~~~~~~~~~~~~~~~~~ @@ -102,11 +129,7 @@ Our final failure we can use with `bugpoint` is one returned by a wrapper process, such as ``valgrind`` or ``timeout``. In this case you will be calling something like `` yosys -s design.il``. Here, Yosys is run under a wrapper process which checks for some failure state, like a memory -leak or excessive runtime. Note however that unlike the `exec` command, there -is currently no way to check the return status or messages from the wrapper -process; only a binary pass/fail. - -.. TODO:: above note pending updated bugpoint #5068 +leak or excessive runtime. How do I use bugpoint? @@ -125,8 +148,6 @@ you can use. Make sure to configure it with the correct filenames and use only one of the methods to load the design. Fill in the ``-grep`` option with the error message printed just before. If you are using a wrapper process for your failure state, add the ``-runner ""`` option to the `bugpoint` call. -For more about the options available, check ``help bugpoint`` or -:doc:`/cmd/bugpoint`. .. code-block:: yoscrypt :caption: ```` template script @@ -142,15 +163,17 @@ For more about the options available, check ``help bugpoint`` or # Save minimized design write_rtlil min.il +The ``-grep`` option is used to search the log file generated by the Yosys under +test. If the error message is generated by something else, such as a wrapper +process or compiler sanitizer, then you should instead use ``-err_grep``. For +an OS error, like a SEGFAULT, you can also use ``-expect-return`` to check the +error code returned. + .. note:: - Using ``-grep ""`` with `bugpoint` is optional, but helps to ensure - that the minimized design is reproducing the right error, especially when - ```` contains more than one command. Unfortunately this does not - work with runtime errors such as a ``SEGFAULT`` as it is only able to match - strings from the log file. - -.. TODO:: above note pending updated bugpoint #5068 + Checking the error message or return status with is optional, but helps to + ensure that the minimized design is reproducing the right error, especially + when ```` contains more than one command. By default, `bugpoint` is able to remove any part of the design. In order to keep certain parts, for instance because you already know they are related to @@ -177,13 +200,16 @@ error on undefined behaviour but only want the child process to halt on error. Once you have finished configuration, you can now run ``yosys ``. The first thing `bugpoint` will do is test the input design fails. If it doesn't, make sure you are using the right ``yosys`` executable; unless the -``-yosys`` option is provided, it will use whatever the shell defaults to. If -you are using the ``-runner`` option, try replacing the `bugpoint` command with -``write_rtlil test.il`` and then on a new line, ``! yosys -s - test.il`` to check it works as expected and returns a non-zero -status. +``-yosys`` option is provided, it will use whatever the shell defaults to, *not* +the current ``yosys``. If you are using the ``-runner`` option, try replacing +the `bugpoint` command with ``write_rtlil test.il`` and then on a new line, +``! yosys -s test.il`` to check it works as expected and +returns a non-zero status. -.. TODO:: note on ``!`` (link to :ref:`getting_started/scripting_intro:script parsing`) +.. seealso:: + + For more on script parsing and the use of ``!``, check out + :ref:`getting_started/scripting_intro:script parsing`. Depending on the size of your design, and the length of your ````, `bugpoint` may take some time; remember, it will run ``yosys -s `` @@ -231,12 +257,19 @@ Once you've verified the failure still happens, check out Minimizing Verilog designs -------------------------- +.. seealso:: + + This section is not specific to Yosys, so feel free to use another guide such + as Stack Overflow's `How to create a Minimal, Reproducible Example`_. + Unlike RTLIL designs where we can use `bugpoint`, minimizing Verilog designs is a much more manual, iterative process. Be sure to check any errors or warnings for messages that might identify source lines or object names that might be causing the failure, and back up your source code before modifying it. At any point in the process, you can check for anything that is unused or totally -disconnected (ports, wires, etc) and remove them too. +disconnected (ports, wires, etc) and remove them too. If you have multiple +source files, try to reduce them down to a single file; either by removing files +or combining them. .. note:: @@ -264,7 +297,7 @@ those too. Any signals which are written but never read can also be removed. Depending on where the design is failing, there are some commands which may help in identifying unused objects in the design. `hierarchy` will identify - which modules are used and which are not, but check for `$paramod` modules + which modules are used and which are not, but check for ``$paramod`` modules before removing unused ones. ``debug clean`` will list all unused wires in each module, as well as unused cells which were automatically generated (giving the line number of the source that generated them). Adding the @@ -272,48 +305,70 @@ those too. Any signals which are written but never read can also be removed. by `clean`. Though when there are large numbers of unused wires it is often easier to just delete sections of the code and see what happens. -- try to remove or reduce assignments and operations +Next, try to remove or reduce assignments (``a = b``) and operations (``a + +b``). A good place to start is by checking for any wires/registers which are +read but never written. Try removing the signal declaration and replacing +references to it with ``'0`` or ``'x``. Do this with any constants too. Try to +replace strings with numeric values, and wide signals with smaller ones, then +see if the error persists. - + are there any wires/registers which get read but never written? +Check if there are any operations that you can simplify, like replacing ``a & +'0`` with ``'0``. If you have enable or reset logic, try removing it and see if +the error still occurs. Try reducing ``if .. else`` and ``case`` blocks to a +single case. Even if that doesn't work, you may still be able to remove some +paths; start with cases that appear to be unreachable and go from there. - * try removing the signal declaration and replacing references to it with - ``'0`` or ``'x`` - * try this with constants too +If you're planning to share the minimized code, remember to make sure there is +no sensitive or proprietary data in the design. Maybe rename that +``ibex_prefetch_buffer`` module to ``buf``, and ``very_important_signal_name`` +could just as easily be ``sig``. The point here isn't to make names as small as +possible, but rather to remove the context that is no longer necessary. Calling +something ``multiplier_output_value`` doesn't mean as much if you no longer have +the multiplier being referred to; but if the name does still make sense then +it's fine to leave it as-is. - + can you replace strings with numeric values? - + are you able to simplify any operations? like replacing ``a & '0`` with - ``'0`` - + if you have enable or reset logic, does the error still happen without that? - + can you reduce an ``if .. else`` to a single case? - + can you remove states from a ``case`` block? +.. note:: -- if you're planning to share the minimized code: - - + make sure there is no sensitive or proprietary data in the design - + instead of a long string of numbers and letters that had some meaning (or - were randomly or sequentially generated), can you give it a single character - name like ``a`` or ``x`` - + please try to keep things in English, using the letters a-z and numbers 0-9 - (unless the error is arising because of the names used) + When sharing code on the `Yosys GitHub`_, please try to keep things in + English. Declarations and strings should stick to the letters a-z and + numbers 0-9, unless the error is arising because of the names/characters + used. Identifying issues ------------------ -- does the failing command indicate limited support, or does it mention some - other command that needs to be run first? -- if you're able to, try to match the minimized design back to its original - context +When identifying issues, it is quite useful to understand the conditions under +which the issue is occurring. While there are occasionally bugs that affect a +significant number of designs, Yosys changes are tested on a variety of designs +and operating systems which typically catch any such issues before they make it +into the main branch. So what is is it about your situation that makes it +unusual? - + could you achieve the same thing a different way? - + and if so, does this other method have the same issue? +.. note:: -- try to change the design in small ways and see what happens + If you have access to a different platform you could also check if your issue + is reproducible there. Some issues may be specific to the platform or build + of Yosys. - + `bugpoint` can reduce and simplify a design, but it doesn't *change* much - + what happens if you change operators, for example a left shift (or `$shl`) - to a right shift (or `$shr`)? - + is the issue tied to specific parameters, widths, or values? +Try to match the minimized design back to its original context. Could you +achieve the same thing a different way, and if so, does this other method have +the same issue? Try to change the design in small ways and see what happens; +while `bugpoint` can reduce and simplify a design, it doesn't *change* much. +What happens if you change operators, for example a left shift (or `$shl`) to a +right shift (or `$shr`)? Try to see if the issue is tied to specific +parameters, widths, or values. + +Search `the existing issues`_ and see if someone has already made a bug report. +This is where changing the design and finding the limits of what causes the +failure really comes in handy. If you're more familiar with how the problem can +arise, you may be able to find a related issue more easily. If an issue already +exists for one case of the problem but you've found other cases, you can comment +on the issue and help get it solved. If there are no existing or related issues +already, then check out the steps for +:ref:`yosys_internals/extending_yosys/contributing:reporting bugs`. + +.. _the existing issues: https://github.com/YosysHQ/yosys/issues .. warning:: @@ -322,17 +377,3 @@ Identifying issues more than one fuzzer generated issue at a time if you can not identify the root cause. If you are found to be doing this, your issues may be closed without further investigation. - -- search `the existing issues`_ and see if someone has already made a bug report - - + this is where changing the design and finding the limits of what causes the - failure really comes in handy - + if you're more familiar with how the problem can arise, you may be able to - find a related issue more easily - + if an issue already exists for one case of the problem but you've found - other cases, you can comment on the issue and help get it solved - -.. _the existing issues: https://github.com/YosysHQ/yosys/issues - -- if there are no existing or related issues already, then check out the steps - for :ref:`yosys_internals/extending_yosys/contributing:reporting bugs` From c994b59ac6a0a87df33fe5ace0a162571329d2e2 Mon Sep 17 00:00:00 2001 From: Krystine Sherwin <93062060+KrystalDelusion@users.noreply.github.com> Date: Tue, 5 Aug 2025 09:54:01 +1200 Subject: [PATCH 22/27] advanced_bugpoint.rst: Paragraphing --- .../extending_yosys/advanced_bugpoint.rst | 170 +++++++++++------- 1 file changed, 109 insertions(+), 61 deletions(-) diff --git a/docs/source/yosys_internals/extending_yosys/advanced_bugpoint.rst b/docs/source/yosys_internals/extending_yosys/advanced_bugpoint.rst index 32d334581..570155c61 100644 --- a/docs/source/yosys_internals/extending_yosys/advanced_bugpoint.rst +++ b/docs/source/yosys_internals/extending_yosys/advanced_bugpoint.rst @@ -6,27 +6,40 @@ in solving or investigating Yosys bugs. This also applies if you are using a fuzzing tool; fuzzers have a tendency to find many variations of the same bug, so identifying the root cause is important for avoiding issue spam. -- if the failing command was part of a larger script, such as one of the - :doc:`/using_yosys/synthesis/synth`, you could try to follow the design - through the script +If you're familiar with C/C++, you might try to have a look at the source code +of the command that's failing. Even if you can't fix the problem yourself, it +can be very helpful for anyone else investigating if you're able to identify +where the issue is arising. - + sometimes when a command is raising an error, you're seeing a symptom rather - than the underlying issue - + an earlier command may be putting the design in an invalid state which isn't - picked up until the error is raised - + check out :ref:`yosys_internals/extending_yosys/advanced_bugpoint:why context matters` -- if you're familiar with C/C++ you might try to have a look at the source - code of the command that's failing +Finding the failing command +--------------------------- + +Using the ``-L`` flag can help here, allowing you to specify a file to log to, +such as ``yosys -L out.log -s script.ys``. Most commands will print a header +message when they begin; something like ``2.48. Executing HIERARCHY pass +(managing design hierarchy).`` The last header message will usually be the +failing command. There are some commands which don't print a header message, so +you may want to add ``echo on`` to the start of your script. The `echo` command +echoes each command executed, along with any arguments given to it. For the +`hierarchy` example above this might be ``yosys> hierarchy -check``. + +.. note:: + + It may also be helpful to use the `log` command to add messages which you can + then search for either in the terminal or the logfile. This can be quite + useful if your script contains script-passes, like the + :doc:`/using_yosys/synthesis/synth`, which call many sub-commands and you're + not sure exactly which script-pass is calling the failing command. - + even if you can't fix the problem yourself, it can be very helpful for - anyone else investigating if you're able to identify where exactly the - issue is Minimizing scripts ------------------ -.. TODO:: disclaimer this is intended as advanced usage, and generally not necessary for bug reports +.. warning:: + + This section is intended as **advanced usage**, and generally not necessary + for normal bug reports. If you're using a command line prompt, such as ``yosys -p 'synth_xilinx' -o design.json design.v``, consider converting it to a script. It's generally much @@ -40,28 +53,10 @@ command line, as well as being better for sharing with others. synth_xilinx write_json design.json -Next up you want to remove everything *after* the error occurs. Using the -``-L`` flag can help here, allowing you to specify a file to log to, such as -``yosys -L out.log -s script.ys``. Most commands will print a header message -when they begin; something like ``2.48. Executing HIERARCHY pass (managing -design hierarchy).`` The last header message will usually be the failing -command. There are some commands which don't print a header message, so you may -want to add ``echo on`` to the start of your script. The `echo` command echoes -each command executed, along with any arguments given to it. For the -`hierarchy` example above this might be ``yosys> hierarchy -check``. - -.. note:: - - It may also be helpful to use the `log` command to add messages which you can - then search for either in the terminal or the logfile. This can be quite - useful if your script contains script-passes, like the - :doc:`/using_yosys/synthesis/synth`, which call many sub-commands and you're - not sure exactly which script-pass is calling the failing command. - -If your final command calls sub-commands, replace it with its contents and -repeat the previous step. In the case of the -:doc:`/using_yosys/synthesis/synth`, as well as certain other script-passes, you -can use the ``-run`` option to simplify this. For example we can replace +Next up you want to remove everything *after* the error occurs. If your final +command calls sub-commands, replace it with its contents first. In the case of +the :doc:`/using_yosys/synthesis/synth`, as well as certain other script-passes, +you can use the ``-run`` option to simplify this. For example we can replace ``synth -top -lut`` with the :ref:`replace_synth`. The options ``-top -lut`` can be provided to each `synth` step, or to just the step(s) where it is relevant, as done here. @@ -177,32 +172,85 @@ recovering context, checkout Why context matters ------------------- -- if you did minimized your script, and removed commands prior to the failure - to get a smaller design, try to work backwards and find which commands may - have contributed to the design failing -- especially important when the bug is happening inside of a ``synth_*`` script -- example (#4590) +Sometimes when a command is raising an error, you're seeing a symptom rather +than the underlying issue. It's possible that an earlier command may be putting +the design in an invalid state, which isn't picked up until the error is raised. +This is particularly true for the pre-packaged +:doc:`/using_yosys/synthesis/synth`, which rely on a combination of generic and +architecture specific passes. As new features are added to Yosys and more +designs are supported, the types of cells output by a pass can grow and change; +and sometimes this leads to a mismatch in what a pass is intended to handle. + +If you minimized your script, and removed commands prior to the failure to get a +smaller reproducer, try to work backwards and find which commands may have +contributed to the design failing. From the minimized design you should have +some understanding of the cell or cells which are producing the error; but where +did those cells come from? The name and/or type of the cell can often point you +in the right direction: + +.. code-block:: + + # internal cell types start with a $ + # lowercase for word-level, uppercase for bit-level + $and + $_AND_ + + # cell types with $__ are typically intermediate cells used in techmapping + $__MUL16X16 + + # cell types without a $ are either user-defined or architecture specific + my_module + SB_MAC16 + + # object names might give you the name of the pass that created them + $procdff$1204 + $memory\rom$rdmux[0][0][0]$a$1550 + + # or even the line number in the Yosys source + $auto$muxcover.cc:557:implement_best_cover$2152 + $auto$alumacc.cc:495:replace_alu$1209 + +Try running the unminimized script and search the log for the names of the +objects in your minimized design. In the case of cells you can also search for +the type of the cell. Remember that calling `stat` will list all the types of +cells currently used in the design, and `select -list =*` will list the names of +of all the current objects. You can add these commands to your script, or use +an interactive terminal to run each command individually. Adding them to the +script can be more repeatable, but if it takes a long time to run to the point +you're interested in then an interactive shell session can give you more +flexibility once you reach that point. You can also add a call to the `shell` +command at any point in a script to start an interactive session at a given +point; allowing you to script any preparation steps, then come back once it's +done. + +A worked example +~~~~~~~~~~~~~~~~ - + say you did all the minimization and found that the error occurs when a call - to ``techmap -map +/xilinx/cells_map.v`` with ``MIN_MUX_INPUTS`` defined - parses a `$_MUX16_` with all inputs set to ``1'x`` - + step through the original script, calling `stat` after each step to find - when the `$_MUX16_` is added - + find that the `$_MUX16_` is introduced by a call to `muxcover`, but all the - inputs are defined, so calling `techmap` now works as expected +Say you did all the minimization and found that an error in `synth_xilinx` +occurs when a call to ``techmap -map +/xilinx/cells_map.v`` with +``MIN_MUX_INPUTS`` defined parses a `$_MUX16_` with all inputs set to ``1'x``. +You could fix the bug in ``+/xilinx/cells_map.v``, but that might only solve +this one case while leaving other problems that haven't been found yet. So you +step through the original script, calling `stat` after each step to find when +the `$_MUX16_` is added. - * and from running `bugpoint` with the failing techmap you know that the - cell with index ``2297`` will fail, so you can now call ``select - top/*$2297`` to limit to just that cell, and optionally call ``design - -save pre_bug`` or ``write_rtlil -selected pre_bug.il`` to save this state +You find that the `$_MUX16_` is introduced by a call to `muxcover`, but all the +inputs are defined, so calling `techmap` now works as expected. From running +`bugpoint` with the failing techmap you know that the cell with index ``2297`` +will fail, so you call ``select top/*$2297`` to limit to just that cell. This +can then be saved with ``design -save pre_bug`` or ``write_rtlil -selected +pre_bug.il``, so that you don't have to re-run all the earlier steps to get back +here. - + next you step through the remaining commands and call `dump` after each to - find when the inputs are disconnected - + find that ``opt -full`` has optimized away portions of the circuit, leading - to `opt_expr` setting the undriven mux inputs to ``x``, but failing to - remove the now unnecessary `$_MUX16_` +Next you step through the remaining commands and call `dump` after each to find +when the inputs are disconnected. You find that ``opt -full`` has optimized +away portions of the circuit, leading to `opt_expr` setting the undriven mux +inputs to ``x``, but failing to remove the now unnecessary `$_MUX16_`. Now +you've identified a problem in `opt_expr` that affects all of the wide muxes, +and could happen in any synthesis flow, not just `synth_xilinx`. -- in this example, you might've stopped with the minimal reproducer, fixed the - bug in ``+/xilinx/cells_map.v``, and carried on -- but by following the failure back you've managed to identify a problem with - `opt_expr` that could be causing other problems either now or in the future +.. seealso:: + + This example is taken from `YosysHQ/yosys#4590 + `_ and can be reproduced with a + version of Yosys between 0.45 and 0.51. From 29d334186c8acc0329eb9b6cf06ac686d71e514e Mon Sep 17 00:00:00 2001 From: Krystine Sherwin <93062060+KrystalDelusion@users.noreply.github.com> Date: Tue, 5 Aug 2025 09:54:01 +1200 Subject: [PATCH 23/27] bugpoint.rst: How to creduce --- docs/source/using_yosys/bugpoint.rst | 87 +++++++++++++++++++++------- 1 file changed, 66 insertions(+), 21 deletions(-) diff --git a/docs/source/using_yosys/bugpoint.rst b/docs/source/using_yosys/bugpoint.rst index 38a978695..5c4bf3ae7 100644 --- a/docs/source/using_yosys/bugpoint.rst +++ b/docs/source/using_yosys/bugpoint.rst @@ -262,20 +262,74 @@ Minimizing Verilog designs This section is not specific to Yosys, so feel free to use another guide such as Stack Overflow's `How to create a Minimal, Reproducible Example`_. -Unlike RTLIL designs where we can use `bugpoint`, minimizing Verilog designs is -a much more manual, iterative process. Be sure to check any errors or warnings -for messages that might identify source lines or object names that might be -causing the failure, and back up your source code before modifying it. At any -point in the process, you can check for anything that is unused or totally -disconnected (ports, wires, etc) and remove them too. If you have multiple -source files, try to reduce them down to a single file; either by removing files -or combining them. +Be sure to check any errors or warnings for messages that might identify source +lines or object names that might be causing the failure, and back up your source +code before modifying it. If you have multiple source files, you should start +by reducing them down to a single file. If a specific file is failing to read, +try removing everything else and just focus on that one. If your source uses +the `include` directive, replace it with the contents of the file referenced. -.. note:: +Unlike RTLIL designs where we can use `bugpoint`, Yosys does not provide any +tools for minimizing Verilog designs. Instead, you should use an external tool +like `C-Reduce`_ (with the ``--not-c`` flag) or `sv-bugpoint`_. - If a specific module is causing the problem, try to set that as the top - module instead. Any parameters should have their default values changed to - match the failing usage. +.. _C-Reduce: https://github.com/csmith-project/creduce +.. _sv-bugpoint: https://github.com/antmicro/sv-bugpoint + +C-Reduce +~~~~~~~~ + +As a very brief overview for using C-Reduce, you want your failing source design +(``test.v``), and some shell script which checks for the error being +investigated (``test.sh``). Below is an :ref:`egtest` which uses `logger` and +the ``-expect error "" 1`` option to perform a similar role to +``bugpoint -grep``, along with ``verilator`` to lint the code and make sure it +is still valid. + +.. code-block:: bash + :caption: Example test.sh + :name: egtest + + #!/bin/bash + verilator --lint-only test.v &&/ + yosys -p 'logger -expect error "unsupported" 1; read_verilog test.v' + +.. code-block:: verilog + :caption: input test.v + + module top(input clk, a, b, c, output x, y, z); + always @(posedge clk) begin + if (a == 1'b1) + $stop; + end + assign x = a; + assign y = a ^ b; + assign z = c; + endmodule + +In this example ``read_verilog test.v`` is giving an error message that contains +the string "unsupported" because the ``$stop`` system task is only supported in +``initial`` blocks. By calling ``creduce ./test.sh test.v --not-c`` we can +minimize the design to just the failing code, while still being valid Verilog. + +.. code-block:: verilog + :caption: output test.v + + module a; + always begin $stop; + end endmodule + + +Doing it manually +~~~~~~~~~~~~~~~~~ + +If for some reason you are unable to use a tool to minimize your code, you can +still do it manually. But it can be a time consuming process and requires a lot +of iteration. At any point in the process, you can check for anything that is +unused or totally disconnected (ports, wires, etc) and remove them. If a +specific module is causing the problem, try to set that as the top module +instead. Any parameters should have their default values changed to match the +failing usage. As a rule of thumb, try to split things roughly in half at each step; similar to a "binary search". If you have 10 cells (instances of modules) in your top @@ -318,15 +372,6 @@ the error still occurs. Try reducing ``if .. else`` and ``case`` blocks to a single case. Even if that doesn't work, you may still be able to remove some paths; start with cases that appear to be unreachable and go from there. -If you're planning to share the minimized code, remember to make sure there is -no sensitive or proprietary data in the design. Maybe rename that -``ibex_prefetch_buffer`` module to ``buf``, and ``very_important_signal_name`` -could just as easily be ``sig``. The point here isn't to make names as small as -possible, but rather to remove the context that is no longer necessary. Calling -something ``multiplier_output_value`` doesn't mean as much if you no longer have -the multiplier being referred to; but if the name does still make sense then -it's fine to leave it as-is. - .. note:: When sharing code on the `Yosys GitHub`_, please try to keep things in From 4924670325f0768cb71f5072e300a874d1bef02e Mon Sep 17 00:00:00 2001 From: Krystine Sherwin <93062060+KrystalDelusion@users.noreply.github.com> Date: Tue, 5 Aug 2025 09:54:02 +1200 Subject: [PATCH 24/27] bugpoint.rst: How to sv-bugpoint --- docs/source/using_yosys/bugpoint.rst | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/docs/source/using_yosys/bugpoint.rst b/docs/source/using_yosys/bugpoint.rst index 5c4bf3ae7..41fc60cdb 100644 --- a/docs/source/using_yosys/bugpoint.rst +++ b/docs/source/using_yosys/bugpoint.rst @@ -287,7 +287,7 @@ the ``-expect error "" 1`` option to perform a similar role to is still valid. .. code-block:: bash - :caption: Example test.sh + :caption: Example test.sh for C-Reduce :name: egtest #!/bin/bash @@ -320,6 +320,25 @@ minimize the design to just the failing code, while still being valid Verilog. end endmodule +sv-bugpoint +~~~~~~~~~~~ + +sv-bugpoint works quite similarly to C-Reduce, except it requires an output +directory to be provided and the check script needs to accept the target file as +an input argument: ``sv-bugpoint outDir/ test.sh test.v`` + +.. code-block:: bash + :caption: Example test.sh for sv-bugpoint + + #!/bin/bash + verilator --lint-only $1 &&/ + yosys -p "logger -expect error \"unsupported\" 1; read_verilog $1" + +Notice that the commands for ``yosys -p`` are now in double quotes (``"``), and +the quotes around the error string are escaped (``\"``). This is necessary for +the ``$1`` argument subsitution to work correctly. + + Doing it manually ~~~~~~~~~~~~~~~~~ From 96b072aeb31ae6e476d419b1eb814febf75b31c6 Mon Sep 17 00:00:00 2001 From: Krystine Sherwin <93062060+KrystalDelusion@users.noreply.github.com> Date: Tue, 5 Aug 2025 09:54:02 +1200 Subject: [PATCH 25/27] advanced_bugpoint.rst: --dump-design Also fix missing double backtick. --- .../extending_yosys/advanced_bugpoint.rst | 25 +++++++++++++++---- 1 file changed, 20 insertions(+), 5 deletions(-) diff --git a/docs/source/yosys_internals/extending_yosys/advanced_bugpoint.rst b/docs/source/yosys_internals/extending_yosys/advanced_bugpoint.rst index 570155c61..22e4b1b7a 100644 --- a/docs/source/yosys_internals/extending_yosys/advanced_bugpoint.rst +++ b/docs/source/yosys_internals/extending_yosys/advanced_bugpoint.rst @@ -213,16 +213,31 @@ in the right direction: Try running the unminimized script and search the log for the names of the objects in your minimized design. In the case of cells you can also search for the type of the cell. Remember that calling `stat` will list all the types of -cells currently used in the design, and `select -list =*` will list the names of -of all the current objects. You can add these commands to your script, or use -an interactive terminal to run each command individually. Adding them to the -script can be more repeatable, but if it takes a long time to run to the point -you're interested in then an interactive shell session can give you more +cells currently used in the design, and ``select -list =*`` will list the names +of of all the current objects. You can add these commands to your script, or +use an interactive terminal to run each command individually. Adding them to +the script can be more repeatable, but if it takes a long time to run to the +point you're interested in then an interactive shell session can give you more flexibility once you reach that point. You can also add a call to the `shell` command at any point in a script to start an interactive session at a given point; allowing you to script any preparation steps, then come back once it's done. +The ``--dump-design`` option +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Yosys provides the ``--dump-design`` option (or ``-P`` for short) for dumping +the design at specific steps of the script based on the log header. If the last +step before an error is ``7.2. Executing MEMORY_MAP pass (converting memories to +logic and flip-flops)``, then calling Yosys with ``--dump-design 7.2:bad.il`` +will save the design *before* this command runs, in the file ``bad.il``. + +It is also possible to use this option multiple times, e.g. ``-P2:hierarchy.il +-P7 -P7.2:bad.il``, to get multiple dumps in the same run. This can make it +easier to follow the design through each step to find where certain cells or +connections are coming from. ``--dump-design ALL`` is also allowed, writing out +the design at each log header. + A worked example ~~~~~~~~~~~~~~~~ From bfe2418a67ea6f586c4b99a094c871f19c2a9871 Mon Sep 17 00:00:00 2001 From: Krystine Sherwin <93062060+KrystalDelusion@users.noreply.github.com> Date: Tue, 5 Aug 2025 09:54:02 +1200 Subject: [PATCH 26/27] bugpoint.rst: Expand note on checking errors --- docs/source/using_yosys/bugpoint.rst | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/docs/source/using_yosys/bugpoint.rst b/docs/source/using_yosys/bugpoint.rst index 41fc60cdb..ce84fd285 100644 --- a/docs/source/using_yosys/bugpoint.rst +++ b/docs/source/using_yosys/bugpoint.rst @@ -171,9 +171,12 @@ error code returned. .. note:: - Checking the error message or return status with is optional, but helps to - ensure that the minimized design is reproducing the right error, especially - when ```` contains more than one command. + Checking the error message or return status is optional, but highly + recommended. `bugpoint` can quite easily introduce bugs by creating + malformed designs that commands were not intended to handle. By having some + way to check the error, `bugpoint` can ensure that it is the *right* error + being reproduced. This is even more important when ```` contains + more than one command. By default, `bugpoint` is able to remove any part of the design. In order to keep certain parts, for instance because you already know they are related to From 8750ca42d38883030c703d31213cfcaeacf28b82 Mon Sep 17 00:00:00 2001 From: Krystine Sherwin <93062060+KrystalDelusion@users.noreply.github.com> Date: Tue, 5 Aug 2025 09:54:02 +1200 Subject: [PATCH 27/27] docs: Fix formatting --- docs/source/using_yosys/bugpoint.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/using_yosys/bugpoint.rst b/docs/source/using_yosys/bugpoint.rst index ce84fd285..60cabd879 100644 --- a/docs/source/using_yosys/bugpoint.rst +++ b/docs/source/using_yosys/bugpoint.rst @@ -270,7 +270,7 @@ lines or object names that might be causing the failure, and back up your source code before modifying it. If you have multiple source files, you should start by reducing them down to a single file. If a specific file is failing to read, try removing everything else and just focus on that one. If your source uses -the `include` directive, replace it with the contents of the file referenced. +the ``include`` directive, replace it with the contents of the file referenced. Unlike RTLIL designs where we can use `bugpoint`, Yosys does not provide any tools for minimizing Verilog designs. Instead, you should use an external tool