From 88b348f9ff57e68898508756dddd171b6b48e4b2 Mon Sep 17 00:00:00 2001 From: Krystine Sherwin <93062060+KrystalDelusion@users.noreply.github.com> Date: Tue, 24 Feb 2026 12:56:56 +1300 Subject: [PATCH] docs: Add information on CI Mostly bullet points, giving a brief description of what runs and when. Mention public/private runners, and manually triggering jobs (on forks). Adds tips for when the different parts of the test suite are run (and under what OS). --- .../yosys_internals/extending_yosys/ci.rst | 153 ++++++++++++++++++ .../yosys_internals/extending_yosys/index.rst | 1 + .../extending_yosys/test_suites.rst | 48 ++++-- 3 files changed, 193 insertions(+), 9 deletions(-) create mode 100644 docs/source/yosys_internals/extending_yosys/ci.rst diff --git a/docs/source/yosys_internals/extending_yosys/ci.rst b/docs/source/yosys_internals/extending_yosys/ci.rst new file mode 100644 index 000000000..e52d1ebf4 --- /dev/null +++ b/docs/source/yosys_internals/extending_yosys/ci.rst @@ -0,0 +1,153 @@ +Continuous Integration +====================== + +.. seealso:: Information about test suites is covered in :doc:`test_suites` + +- combination of public (github) runners and private runner (for building with + Verific) + +Manually triggering jobs +------------------------ + +- most jobs can run on ``workflow_dispatch``, allowing manual starting of a job + on any branch +- jobs which run on a public (github) runner can generally be run on any fork of + the repo +- `learn more about GitHub Actions + `_ + +Workflows +--------- + +- located in :file:`.github/workflows` +- most jobs will be cancelled if a newer commit is pushed, prioritizing the most + up-to-date version instead + + - jobs running on the main branch will never be cancelled + +- most jobs will skip documentation-only changes (:file:`docs` directory and + readme file) + +Compiler testing +~~~~~~~~~~~~~~~~ + +- ``test-compile.yml`` +- most of these use ``make compile-only``, meaning no linking (or executing) +- run on pushes to main and PRs + +- gcc + + - oldest supported + - newest supported + +- clang + + - latest ubuntu, oldest supported + - latest ubuntu, newest supported + - macOS x86 + - macOS arm + - verific configurations (``test-verific-cfg.yml`` + manually triggered only) + +- C++17 +- C++20 (newest compiler versions only) + +Build testing +~~~~~~~~~~~~~ + +- ``test-build.yml`` + + - run on pushes to main and PRs + - builds once (per OS) out-of-tree, using the same build artifact across tests + - mostly covered by :doc:`test_suites` + - ``make`` + - ``make unit-test`` + - ``make vanilla-test`` + - ``make -C docs test`` + +- ``test-sanitizers.yml`` + + - uses private runner + - run on pushes to main and during PR merges + - ``make vanilla-test`` with ``SANITIZER=undefined,address`` + + +Automatic testing with Verific +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- ``test-verific.yml`` +- uses private runner +- run on pushes to main and during PR merges +- build and test Yosys+Verific + + - Vanilla tests + - Functional tests + - SBY tests (using latest version of SBY) + - Yosys+Verific specific tests + +- also builds pyosys and runs pyosys tests + +Extra build flows +~~~~~~~~~~~~~~~~~ + +- compiled and linked but not executed +- ``extra-builds.yml`` + + - run on pushes to main and PRs + - WASI (smoke test for YoWASP) + - VS (not released, not maintained?) + - nix (flake no longer updating, not maintained?) + +- ``wheels.yml`` + + - run once per week on latest main: + - python wheels for PyPI release (pyosys) + +Building documentation +~~~~~~~~~~~~~~~~~~~~~~ + +- uses private runner +- runs even on documentation-only changes, but still skips readme-only changes +- ``prepare-docs.yml`` + + - run on all pushes and PRs + - prepare docs artifact from Yosys executable (``make docs/prep``) + - triggers ReadtheDocs build for main (latest), tagged versions (releases), + and certain WIP doc previews + +- ``test-build.yml`` + + - run on pushes to main and PRs + - ``make docs``, building both html and latexpdf targets (independently) + + - will fail on warnings to replicate ReadtheDocs behavior (which itself was + configured to prevent updating live documentation with an incomplete or + missing artifact) + +CodeQL +~~~~~~ + +- ``codeql.yml`` +- static code analysis +- run every day on latest main + +Vendor sources +~~~~~~~~~~~~~~ + +- ``source-vendor.yml`` +- download and package source code with all submodules +- used for releases (``yosys.tar.gz``) +- run on pushes to YosysHQ/yosys + + +Composite actions +----------------- + +- consolidate common setup steps +- :file:`.github/actions/setup-build-env/action.yml` + + - consistent build environment setup + - broken into build/docs/test dependencies (currently only on linux) + +- :file:`.github/actions/setup-iverilog/action.yml` + - iverilog install (including dependencies) diff --git a/docs/source/yosys_internals/extending_yosys/index.rst b/docs/source/yosys_internals/extending_yosys/index.rst index 72843ecd6..1b8c9993b 100644 --- a/docs/source/yosys_internals/extending_yosys/index.rst +++ b/docs/source/yosys_internals/extending_yosys/index.rst @@ -14,4 +14,5 @@ of interest for developers looking to customise Yosys builds. advanced_bugpoint contributing test_suites + ci diff --git a/docs/source/yosys_internals/extending_yosys/test_suites.rst b/docs/source/yosys_internals/extending_yosys/test_suites.rst index d3422b23a..821eb7659 100644 --- a/docs/source/yosys_internals/extending_yosys/test_suites.rst +++ b/docs/source/yosys_internals/extending_yosys/test_suites.rst @@ -3,6 +3,22 @@ Testing Yosys .. todo:: adding tests (makefile-tests vs seed-tests) +.. todo:: pyosys tests + + (part of :ref:`yosys_internals/extending_yosys/ci:Automatic testing with Verific`) + +.. TODO:: test-cells + + (run on pushes to main and PRs, under Linux) + +The majority of the included test suite is run automatically on the `Yosys Git +repo`_. For more about compiler tests and our CI process, check the :doc:`ci` +document. For the most up to date information, including OS versions of test +runners, refer to `the GitHub actions page`_. + +.. _Yosys Git repo: https://github.com/YosysHQ/yosys +.. _the GitHub actions page: https://github.com/YosysHQ/yosys/actions + Running the included test suite ------------------------------- @@ -30,6 +46,17 @@ with ``-assert-count``. Usually it's unnecessary to "register" the test anywhere as if it's being added to an existing directory, depending on how the ``run-test.sh`` in that directory works. +.. tip:: + + Automatically run on pushes to main and PRs, under both macOS and Linux + (``test-build.yml``) + +.. TODO:: testing with sanitizers + + - Automatically run on pushes to main and during PR merges, under both macOS + and Linux, using clang's UBSAN and ASAN (``test-sanitizers.yml``) + - run locally with :makevar:`SANITIZER` (clean build first) + Unit tests ~~~~~~~~~~ @@ -47,6 +74,11 @@ Running the unit tests requires the following additional packages: Unit tests can be run with ``make unit-test``. +.. tip:: + + Automatically run on pushes to main and PRs, under both macOS and Linux + (``test-build.yml``) + Functional tests ~~~~~~~~~~~~~~~~ @@ -78,6 +110,11 @@ instructions `_. Then, set the :makevar:`ENABLE_FUNCTIONAL_TESTS` make variable when calling ``make test`` and the functional tests will be run as well. +.. tip:: + + Automatically run as part of + :ref:`yosys_internals/extending_yosys/ci:Automatic testing with Verific` + Docs tests ~~~~~~~~~~ @@ -87,17 +124,10 @@ documentation, which can be run by calling ``make test`` from the also includes checking some macro commands to ensure that descriptions of them are kept up to date, and is mostly intended for CI. +.. tip:: -Automatic testing ------------------ + Automatically run on pushes to main and PRs, under Linux (``test-build.yml``) -The `Yosys Git repo`_ has automatic testing of builds and running of the -included test suite on both Ubuntu and macOS, as well as across range of -compiler versions. For up to date information, including OS versions, refer to -`the git actions page`_. - -.. _Yosys Git repo: https://github.com/YosysHQ/yosys -.. _the git actions page: https://github.com/YosysHQ/yosys/actions .. How to add a unit test