From 6e2695a206f55dd5ac9a9d2c89a23ec7a428cb3f Mon Sep 17 00:00:00 2001 From: Krystine Sherwin <93062060+KrystalDelusion@users.noreply.github.com> Date: Tue, 29 Apr 2025 12:18:11 +1200 Subject: [PATCH] docs: Split bugpoint.rst into user/developer Minimizing scripts (and more generally identifying root cause) isn't necessary for regular bug reports. Rather, it can be useful for developers working on *fixing* bugs, and also for fuzzers to avoid spam. Minor adjustments to `bugpoint.rst`. Add note to `advanced_bugpoint.rst` about primitives when minimizing scripts. --- docs/source/using_yosys/bugpoint.rst | 225 +----------------- .../extending_yosys/advanced_bugpoint.rst | 208 ++++++++++++++++ .../yosys_internals/extending_yosys/index.rst | 1 + 3 files changed, 217 insertions(+), 217 deletions(-) create mode 100644 docs/source/yosys_internals/extending_yosys/advanced_bugpoint.rst diff --git a/docs/source/using_yosys/bugpoint.rst b/docs/source/using_yosys/bugpoint.rst index 89c58a8f4..82eab3cfa 100644 --- a/docs/source/using_yosys/bugpoint.rst +++ b/docs/source/using_yosys/bugpoint.rst @@ -38,8 +38,6 @@ Minimizing failing (or bugged) designs error while trying to debug it -.. _minimize your RTLIL: - Minimizing RTLIL designs with bugpoint -------------------------------------- @@ -74,12 +72,6 @@ be aiming to reproduce the failure by running ``yosys -s -s Check out the instructions for :ref:`using_yosys/bugpoint:minimizing verilog designs` below. -The commands in ```` only need to be run once, while those in -```` will be run on each iteration of `bugpoint`. If you haven't -already, following the instructions for :ref:`using_yosys/bugpoint:minimizing -scripts` will also help with identifying exactly which commands are needed to -produce the failure and which can be safely moved to the loading script. - .. note:: You should also be able to run the two scripts separately, calling first @@ -236,150 +228,6 @@ Once you've verified the failure still happens, check out :ref:`using_yosys/bugpoint:identifying issues` for more on what to do next. -.. _minimize your script: - -Minimizing scripts ------------------- - -If you're using a command line prompt, such as ``yosys -p 'synth_xilinx' -o -design.json design.v``, consider converting it to a script. It's generally much -easier to iterate over changes to a script in a file rather than one on the -command line, as well as being better for sharing with others. - -.. code-block:: yoscrypt - :caption: example script, ``script.ys``, for prompt ``yosys -p 'synth_xilinx' -o design.json design.v`` - - read_verilog design.v - synth_xilinx - write_json design.json - -Next up you want to remove everything *after* the error occurs. Using the -``-L`` flag can help here, allowing you to specify a file to log to, such as -``yosys -L out.log -s script.ys``. Most commands will print a header message -when they begin; something like ``2.48. Executing HIERARCHY pass (managing -design hierarchy).`` The last header message will usually be the failing -command. There are some commands which don't print a header message, so you may -want to add ``echo on`` to the start of your script. The `echo` command echoes -each command executed, along with any arguments given to it. For the -`hierarchy` example above this might be ``yosys> hierarchy -check``. - -.. note:: - - It may also be helpful to use the `log` command to add messages which you can - then search for either in the terminal or the logfile. This can be quite - useful if your script contains script-passes, like the - :doc:`/using_yosys/synthesis/synth`, which call many sub-commands and you're - not sure exactly which script-pass is calling the failing command. - -If your final command calls sub-commands, replace it with its contents and -repeat the previous step. In the case of the -:doc:`/using_yosys/synthesis/synth`, as well as certain other script-passes, you -can use the ``-run`` option to simplify this. For example we can replace -``synth -top -lut`` with the :ref:`replace_synth`. The options ``-top - -lut`` can be provided to each `synth` step, or to just the step(s) where -it is relevant, as done here. - -.. code-block:: yoscrypt - :caption: example replacement script for `synth` command - :name: replace_synth - - synth -top -run :coarse - synth -lut -run coarse:fine - synth -lut -run fine:check - synth -run check: - -Say we ran :ref:`replace_synth` and were able to remove the ``synth -run -check:`` and still got our error, then we check the log and we see the last -thing before the error was ``7.2. Executing MEMORY_MAP pass (converting memories -to logic and flip-flops)``. By checking the output of ``yosys -h synth`` (or the -`synth` help page) we can see that the `memory_map` pass is called in the -``fine`` step. We can then update our script to the following: - -.. code-block:: yoscrypt - :caption: example replacement script for `synth` when `memory_map` is failing - - synth -top -run :fine - opt -fast -full - memory_map - -By giving `synth` the option ``-run :fine``, we are telling it to run from the -beginning of the script until the ``fine`` step, where we then give it the exact -commands to run. There are some cases where the commands given in the help -output are not an exact match for what is being run, but are instead a -simplification. If you find that replacing the script-pass with its contents -causes the error to disappear, or change, try calling the script-pass with -``echo on`` to see exactly what commands are being called and what options are -used. - -.. warning:: - - Before continuing further, *back up your code*. The following steps can - remove context and lead to over-minimizing scripts, hiding underlying issues. - Check out :ref:`using_yosys/bugpoint:Why context matters` to learn more. - -When a problem is occurring many steps into a script, minimizing the design at -the start of the script isn't always enough to identify the cause of the issue. -Each extra step of the script can lead to larger sections of the input design -being needed for the specific problem to be preserved until it causes a crash. -So to find the smallest possible reproducer it can sometimes be helpful to -remove commands prior to the failure point. - -The simplest way to do this is by writing out the design, resetting the current -state, and reading back the design: - -.. code-block:: yoscrypt - - write_rtlil ; design -reset; read_rtlil ; - -In most cases, this can be inserted immediately before the failing command while -still producing the error, allowing you to `minimize your RTLIL`_ with the -```` output. For our previous example with `memory_map`, if -:ref:`map_reset` still gives the same error, then we should now be able to call -``yosys design.il -p 'memory_map'`` to reproduce it. - -.. code-block:: yoscrypt - :caption: resetting the design immediately before failure - :name: map_reset - - synth -top -run :fine - opt -fast -full - write_rtlil design.il; design -reset; read_rtlil design.il; - memory_map - -If that doesn't give the error (or doesn't give the same error), then you should -try to move the write/reset/read earlier in the script until it does. If you -have no idea where exactly you should put the reset, the best way is to use a -"binary search" type approach, reducing the possible options by half after each -attempt. - - As an example, your script has 16 commands in it before failing on the 17th. - If resetting immediately before the 17th doesn't reproduce the error, try - between the 8th and 9th (8 is half of the total 16). If that produces the - error then you can remove everything before the `read_rtlil` and try reset - again in the middle of what's left, making sure to use a different name for - the output file so that you don't overwrite what you've already got. If the - error isn't produced then you need to go earlier still, so in this case you - would do between the 4th and 5th (4 is half of the previous 8). Repeat this - until you can't reduce the remaining commands any further. - -.. TODO:: is it possible to dump scratchpad? - - is there anything else in the yosys/design state that doesn't get included in - `write_rtlil`? - -A more conservative, but more involved, method is to remove or comment out -commands prior to the failing command. Each command, or group of commands, can -be disabled one at a time while checking if the error still occurs, eventually -giving the smallest subset of commands needed to take the original input through -to the error. The difficulty with this method is that depending on your design, -some commands may be necessary even if they aren't needed to reproduce the -error. For example, if your design includes ``process`` blocks, many commands -will fail unless you run the `proc` command. While this approach can do a -better job of maintaining context, it is often easier to *recover* the context -after the design has been minimized for producing the error. For more on -recovering context, checkout :ref:`using_yosys/bugpoint:Why context matters`. - - Minimizing Verilog designs -------------------------- @@ -393,9 +241,9 @@ Minimizing Verilog designs + if the problem is parameter specific you may be able to change the default parameters so that they match the problematic configuration -- as with `minimize your script`_, if you have no idea what is or is not - relevant, try to follow a "binary search" type approach where you remove (or - comment out) roughly half of what's left at a time +- if you have no idea what is or is not relevant, try to follow a "binary + search" type approach where you remove (or comment out) roughly half of what's + left at a time - focusing on one type of object at a time simplifies the process, removing as many as you can until the error disappears if any of the remaining objects are removed @@ -448,36 +296,13 @@ Identifying issues to a right shift (or `$shr`)? + is the issue tied to specific parameters, widths, or values? -- if the failing command was part of a larger script, such as one of the - :doc:`/using_yosys/synthesis/synth`, you could try to follow the design - through the script - - + sometimes when a command is raising an error, you're seeing a symptom rather - than the underlying issue - + an earlier command may be putting the design in an invalid state which isn't - picked up until the error is raised - + check out :ref:`using_yosys/bugpoint:Why context matters` - + if you're using a fuzzer to find issues in Yosys, you should be prepared to - do this step - -- if you're familiar with C/C++ you might try to have a look at the source - code of the command that's failing - - + even if you can't fix the problem yourself, it can be very helpful for - anyone else investigating if you're able to identify where exactly the - issue is - + if you're using a fuzzer to find issues in Yosys, you should be prepared to - do this step - .. warning:: - In the event that you are unable to identify the root cause of a fuzzer - generated issue, **do not** open more than one issue at a time. You have no - way of being able to tell if multiple fuzzer generated issues are simply - different cases of the same problem, and opening multiple issues for the same - problem means more time is spent on triaging and diagnosing bug reports and - less on fixing the problem. If you are found to be doing this, your issues - may be closed without further investigation. + If you are using a fuzzer to find bugs, follow the instructions for + :doc:`/yosys_internals/extending_yosys/advanced_bugpoint`. **Do not** open + more than one fuzzer generated issue at a time if you can not identify the + root cause. If you are found to be doing this, your issues may be closed + without further investigation. - search `the existing issues`_ and see if someone has already made a bug report @@ -492,37 +317,3 @@ Identifying issues - if there are no existing or related issues already, then check out the steps for :ref:`yosys_internals/extending_yosys/contributing:reporting bugs` - - -Why context matters -------------------- - -- if you did `minimize your script`_, and removed commands prior to the failure - to get a smaller design, try to work backwards and find which commands may - have contributed to the design failing -- especially important when the bug is happening inside of a ``synth_*`` script -- example (#4590) - - + say you did all the minimization and found that the error occurs when a call - to ``techmap -map +/xilinx/cells_map.v`` with ``MIN_MUX_INPUTS`` defined - parses a `$_MUX16_` with all inputs set to ``1'x`` - + step through the original script, calling `stat` after each step to find - when the `$_MUX16_` is added - + find that the `$_MUX16_` is introduced by a call to `muxcover`, but all the - inputs are defined, so calling `techmap` now works as expected - - * and from running `bugpoint` with the failing techmap you know that the - cell with index ``2297`` will fail, so you can now call ``select - top/*$2297`` to limit to just that cell, and optionally call ``design - -save pre_bug`` or ``write_rtlil -selected pre_bug.il`` to save this state - - + next you step through the remaining commands and call `dump` after each to - find when the inputs are disconnected - + find that ``opt -full`` has optimized away portions of the circuit, leading - to `opt_expr` setting the undriven mux inputs to ``x``, but failing to - remove the now unnecessary `$_MUX16_` - -- in this example, you might've stopped with the minimal reproducer, fixed the - bug in ``+/xilinx/cells_map.v``, and carried on -- but by following the failure back you've managed to identify a problem with - `opt_expr` that could be causing other problems either now or in the future diff --git a/docs/source/yosys_internals/extending_yosys/advanced_bugpoint.rst b/docs/source/yosys_internals/extending_yosys/advanced_bugpoint.rst new file mode 100644 index 000000000..32d334581 --- /dev/null +++ b/docs/source/yosys_internals/extending_yosys/advanced_bugpoint.rst @@ -0,0 +1,208 @@ +Identifying the root cause of bugs +================================== + +This document references Yosys internals and is intended for people interested +in solving or investigating Yosys bugs. This also applies if you are using a +fuzzing tool; fuzzers have a tendency to find many variations of the same bug, +so identifying the root cause is important for avoiding issue spam. + +- if the failing command was part of a larger script, such as one of the + :doc:`/using_yosys/synthesis/synth`, you could try to follow the design + through the script + + + sometimes when a command is raising an error, you're seeing a symptom rather + than the underlying issue + + an earlier command may be putting the design in an invalid state which isn't + picked up until the error is raised + + check out :ref:`yosys_internals/extending_yosys/advanced_bugpoint:why context matters` + +- if you're familiar with C/C++ you might try to have a look at the source + code of the command that's failing + + + even if you can't fix the problem yourself, it can be very helpful for + anyone else investigating if you're able to identify where exactly the + issue is + +Minimizing scripts +------------------ + +.. TODO:: disclaimer this is intended as advanced usage, and generally not necessary for bug reports + +If you're using a command line prompt, such as ``yosys -p 'synth_xilinx' -o +design.json design.v``, consider converting it to a script. It's generally much +easier to iterate over changes to a script in a file rather than one on the +command line, as well as being better for sharing with others. + +.. code-block:: yoscrypt + :caption: example script, ``script.ys``, for prompt ``yosys -p 'synth_xilinx' -o design.json design.v`` + + read_verilog design.v + synth_xilinx + write_json design.json + +Next up you want to remove everything *after* the error occurs. Using the +``-L`` flag can help here, allowing you to specify a file to log to, such as +``yosys -L out.log -s script.ys``. Most commands will print a header message +when they begin; something like ``2.48. Executing HIERARCHY pass (managing +design hierarchy).`` The last header message will usually be the failing +command. There are some commands which don't print a header message, so you may +want to add ``echo on`` to the start of your script. The `echo` command echoes +each command executed, along with any arguments given to it. For the +`hierarchy` example above this might be ``yosys> hierarchy -check``. + +.. note:: + + It may also be helpful to use the `log` command to add messages which you can + then search for either in the terminal or the logfile. This can be quite + useful if your script contains script-passes, like the + :doc:`/using_yosys/synthesis/synth`, which call many sub-commands and you're + not sure exactly which script-pass is calling the failing command. + +If your final command calls sub-commands, replace it with its contents and +repeat the previous step. In the case of the +:doc:`/using_yosys/synthesis/synth`, as well as certain other script-passes, you +can use the ``-run`` option to simplify this. For example we can replace +``synth -top -lut`` with the :ref:`replace_synth`. The options ``-top + -lut`` can be provided to each `synth` step, or to just the step(s) where +it is relevant, as done here. + +.. code-block:: yoscrypt + :caption: example replacement script for `synth` command + :name: replace_synth + + synth -top -run :coarse + synth -lut -run coarse:fine + synth -lut -run fine:check + synth -run check: + +Say we ran :ref:`replace_synth` and were able to remove the ``synth -run +check:`` and still got our error, then we check the log and we see the last +thing before the error was ``7.2. Executing MEMORY_MAP pass (converting memories +to logic and flip-flops)``. By checking the output of ``yosys -h synth`` (or the +`synth` help page) we can see that the `memory_map` pass is called in the +``fine`` step. We can then update our script to the following: + +.. code-block:: yoscrypt + :caption: example replacement script for `synth` when `memory_map` is failing + + synth -top -run :fine + opt -fast -full + memory_map + +By giving `synth` the option ``-run :fine``, we are telling it to run from the +beginning of the script until the ``fine`` step, where we then give it the exact +commands to run. There are some cases where the commands given in the help +output are not an exact match for what is being run, but are instead a +simplification. If you find that replacing the script-pass with its contents +causes the error to disappear, or change, try calling the script-pass with +``echo on`` to see exactly what commands are being called and what options are +used. + +.. warning:: + + Before continuing further, *back up your code*. The following steps can + remove context and lead to over-minimizing scripts, hiding underlying issues. + Check out :ref:`yosys_internals/extending_yosys/advanced_bugpoint:Why + context matters` to learn more. + +When a problem is occurring many steps into a script, minimizing the design at +the start of the script isn't always enough to identify the cause of the issue. +Each extra step of the script can lead to larger sections of the input design +being needed for the specific problem to be preserved until it causes a crash. +So to find the smallest possible reproducer it can sometimes be helpful to +remove commands prior to the failure point. + +The simplest way to do this is by writing out the design, resetting the current +state, and reading back the design: + +.. code-block:: yoscrypt + + write_rtlil ; design -reset; read_rtlil ; + +In most cases, this can be inserted immediately before the failing command while +still producing the error, allowing you to :ref:`minimize your +RTLIL` with the +```` output. For our previous example with `memory_map`, if +:ref:`map_reset` still gives the same error, then we should now be able to call +``yosys design.il -p 'memory_map'`` to reproduce it. + +.. code-block:: yoscrypt + :caption: resetting the design immediately before failure + :name: map_reset + + synth -top -run :fine + opt -fast -full + write_rtlil design.il; design -reset; read_rtlil design.il; + memory_map + +If that doesn't give the error (or doesn't give the same error), then you should +try to move the write/reset/read earlier in the script until it does. If you +have no idea where exactly you should put the reset, the best way is to use a +"binary search" type approach, reducing the possible options by half after each +attempt. + +.. note:: + + By default, `write_rtlil` doesn't include platform specific IP blocks and + other primitive cell models which are typically loaded with a ``read_verilog + -lib`` command at the start of the synthesis script. You may have to + duplicate these commands *after* the call to ``design -reset``. It is also + possible to write out *everything* with ``select =*; write_rtlil -selected + ``. + +As an example, your script has 16 commands in it before failing on the 17th. If +resetting immediately before the 17th doesn't reproduce the error, try between +the 8th and 9th (8 is half of the total 16). If that produces the error then +you can remove everything before the `read_rtlil` and try reset again in the +middle of what's left, making sure to use a different name for the output file +so that you don't overwrite what you've already got. If the error isn't +produced then you need to go earlier still, so in this case you would do between +the 4th and 5th (4 is half of the previous 8). Repeat this until you can't +reduce the remaining commands any further. + +A more conservative, but more involved, method is to remove or comment out +commands prior to the failing command. Each command, or group of commands, can +be disabled one at a time while checking if the error still occurs, eventually +giving the smallest subset of commands needed to take the original input through +to the error. The difficulty with this method is that depending on your design, +some commands may be necessary even if they aren't needed to reproduce the +error. For example, if your design includes ``process`` blocks, many commands +will fail unless you run the `proc` command. While this approach can do a +better job of maintaining context, it is often easier to *recover* the context +after the design has been minimized for producing the error. For more on +recovering context, checkout +:ref:`yosys_internals/extending_yosys/advanced_bugpoint:Why context matters`. + + +Why context matters +------------------- + +- if you did minimized your script, and removed commands prior to the failure + to get a smaller design, try to work backwards and find which commands may + have contributed to the design failing +- especially important when the bug is happening inside of a ``synth_*`` script +- example (#4590) + + + say you did all the minimization and found that the error occurs when a call + to ``techmap -map +/xilinx/cells_map.v`` with ``MIN_MUX_INPUTS`` defined + parses a `$_MUX16_` with all inputs set to ``1'x`` + + step through the original script, calling `stat` after each step to find + when the `$_MUX16_` is added + + find that the `$_MUX16_` is introduced by a call to `muxcover`, but all the + inputs are defined, so calling `techmap` now works as expected + + * and from running `bugpoint` with the failing techmap you know that the + cell with index ``2297`` will fail, so you can now call ``select + top/*$2297`` to limit to just that cell, and optionally call ``design + -save pre_bug`` or ``write_rtlil -selected pre_bug.il`` to save this state + + + next you step through the remaining commands and call `dump` after each to + find when the inputs are disconnected + + find that ``opt -full`` has optimized away portions of the circuit, leading + to `opt_expr` setting the undriven mux inputs to ``x``, but failing to + remove the now unnecessary `$_MUX16_` + +- in this example, you might've stopped with the minimal reproducer, fixed the + bug in ``+/xilinx/cells_map.v``, and carried on +- but by following the failure back you've managed to identify a problem with + `opt_expr` that could be causing other problems either now or in the future diff --git a/docs/source/yosys_internals/extending_yosys/index.rst b/docs/source/yosys_internals/extending_yosys/index.rst index 4ee21517b..72843ecd6 100644 --- a/docs/source/yosys_internals/extending_yosys/index.rst +++ b/docs/source/yosys_internals/extending_yosys/index.rst @@ -11,6 +11,7 @@ of interest for developers looking to customise Yosys builds. extensions build_verific functional_ir + advanced_bugpoint contributing test_suites