mirror of
https://github.com/YosysHQ/yosys
synced 2026-03-01 11:16:55 +00:00
docs: Add information on CI
Mostly bullet points, giving a brief description of what runs and when. Mention public/private runners, and manually triggering jobs (on forks). Adds tips for when the different parts of the test suite are run (and under what OS).
This commit is contained in:
Krystine Sherwin
parent
53d8eb43ff
commit
88b348f9ff
3 changed files with 193 additions and 9 deletions
153
docs/source/yosys_internals/extending_yosys/ci.rst
Normal file
153
docs/source/yosys_internals/extending_yosys/ci.rst
Normal file
|
|
@ -0,0 +1,153 @@
|
|||
Continuous Integration
|
||||
======================
|
||||
|
||||
.. seealso:: Information about test suites is covered in :doc:`test_suites`
|
||||
|
||||
- combination of public (github) runners and private runner (for building with
|
||||
Verific)
|
||||
|
||||
Manually triggering jobs
|
||||
------------------------
|
||||
|
||||
- most jobs can run on ``workflow_dispatch``, allowing manual starting of a job
|
||||
on any branch
|
||||
- jobs which run on a public (github) runner can generally be run on any fork of
|
||||
the repo
|
||||
- `learn more about GitHub Actions
|
||||
<https://docs.github.com/en/actions/get-started>`_
|
||||
|
||||
Workflows
|
||||
---------
|
||||
|
||||
- located in :file:`.github/workflows`
|
||||
- most jobs will be cancelled if a newer commit is pushed, prioritizing the most
|
||||
up-to-date version instead
|
||||
|
||||
- jobs running on the main branch will never be cancelled
|
||||
|
||||
- most jobs will skip documentation-only changes (:file:`docs` directory and
|
||||
readme file)
|
||||
|
||||
Compiler testing
|
||||
~~~~~~~~~~~~~~~~
|
||||
|
||||
- ``test-compile.yml``
|
||||
- most of these use ``make compile-only``, meaning no linking (or executing)
|
||||
- run on pushes to main and PRs
|
||||
|
||||
- gcc
|
||||
|
||||
- oldest supported
|
||||
- newest supported
|
||||
|
||||
- clang
|
||||
|
||||
- latest ubuntu, oldest supported
|
||||
- latest ubuntu, newest supported
|
||||
- macOS x86
|
||||
- macOS arm
|
||||
- verific configurations (``test-verific-cfg.yml``
|
||||
manually triggered only)
|
||||
|
||||
- C++17
|
||||
- C++20 (newest compiler versions only)
|
||||
|
||||
Build testing
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
- ``test-build.yml``
|
||||
|
||||
- run on pushes to main and PRs
|
||||
- builds once (per OS) out-of-tree, using the same build artifact across tests
|
||||
- mostly covered by :doc:`test_suites`
|
||||
- ``make``
|
||||
- ``make unit-test``
|
||||
- ``make vanilla-test``
|
||||
- ``make -C docs test``
|
||||
|
||||
- ``test-sanitizers.yml``
|
||||
|
||||
- uses private runner
|
||||
- run on pushes to main and during PR merges
|
||||
- ``make vanilla-test`` with ``SANITIZER=undefined,address``
|
||||
|
||||
|
||||
Automatic testing with Verific
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
- ``test-verific.yml``
|
||||
- uses private runner
|
||||
- run on pushes to main and during PR merges
|
||||
- build and test Yosys+Verific
|
||||
|
||||
- Vanilla tests
|
||||
- Functional tests
|
||||
- SBY tests (using latest version of SBY)
|
||||
- Yosys+Verific specific tests
|
||||
|
||||
- also builds pyosys and runs pyosys tests
|
||||
|
||||
Extra build flows
|
||||
~~~~~~~~~~~~~~~~~
|
||||
|
||||
- compiled and linked but not executed
|
||||
- ``extra-builds.yml``
|
||||
|
||||
- run on pushes to main and PRs
|
||||
- WASI (smoke test for YoWASP)
|
||||
- VS (not released, not maintained?)
|
||||
- nix (flake no longer updating, not maintained?)
|
||||
|
||||
- ``wheels.yml``
|
||||
|
||||
- run once per week on latest main:
|
||||
- python wheels for PyPI release (pyosys)
|
||||
|
||||
Building documentation
|
||||
~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
- uses private runner
|
||||
- runs even on documentation-only changes, but still skips readme-only changes
|
||||
- ``prepare-docs.yml``
|
||||
|
||||
- run on all pushes and PRs
|
||||
- prepare docs artifact from Yosys executable (``make docs/prep``)
|
||||
- triggers ReadtheDocs build for main (latest), tagged versions (releases),
|
||||
and certain WIP doc previews
|
||||
|
||||
- ``test-build.yml``
|
||||
|
||||
- run on pushes to main and PRs
|
||||
- ``make docs``, building both html and latexpdf targets (independently)
|
||||
|
||||
- will fail on warnings to replicate ReadtheDocs behavior (which itself was
|
||||
configured to prevent updating live documentation with an incomplete or
|
||||
missing artifact)
|
||||
|
||||
CodeQL
|
||||
~~~~~~
|
||||
|
||||
- ``codeql.yml``
|
||||
- static code analysis
|
||||
- run every day on latest main
|
||||
|
||||
Vendor sources
|
||||
~~~~~~~~~~~~~~
|
||||
|
||||
- ``source-vendor.yml``
|
||||
- download and package source code with all submodules
|
||||
- used for releases (``yosys.tar.gz``)
|
||||
- run on pushes to YosysHQ/yosys
|
||||
|
||||
|
||||
Composite actions
|
||||
-----------------
|
||||
|
||||
- consolidate common setup steps
|
||||
- :file:`.github/actions/setup-build-env/action.yml`
|
||||
|
||||
- consistent build environment setup
|
||||
- broken into build/docs/test dependencies (currently only on linux)
|
||||
|
||||
- :file:`.github/actions/setup-iverilog/action.yml`
|
||||
- iverilog install (including dependencies)
|
||||
|
|
@ -14,4 +14,5 @@ of interest for developers looking to customise Yosys builds.
|
|||
advanced_bugpoint
|
||||
contributing
|
||||
test_suites
|
||||
ci
|
||||
|
||||
|
|
|
|||
|
|
@ -3,6 +3,22 @@ Testing Yosys
|
|||
|
||||
.. todo:: adding tests (makefile-tests vs seed-tests)
|
||||
|
||||
.. todo:: pyosys tests
|
||||
|
||||
(part of :ref:`yosys_internals/extending_yosys/ci:Automatic testing with Verific`)
|
||||
|
||||
.. TODO:: test-cells
|
||||
|
||||
(run on pushes to main and PRs, under Linux)
|
||||
|
||||
The majority of the included test suite is run automatically on the `Yosys Git
|
||||
repo`_. For more about compiler tests and our CI process, check the :doc:`ci`
|
||||
document. For the most up to date information, including OS versions of test
|
||||
runners, refer to `the GitHub actions page`_.
|
||||
|
||||
.. _Yosys Git repo: https://github.com/YosysHQ/yosys
|
||||
.. _the GitHub actions page: https://github.com/YosysHQ/yosys/actions
|
||||
|
||||
Running the included test suite
|
||||
-------------------------------
|
||||
|
||||
|
|
@ -30,6 +46,17 @@ with ``-assert-count``. Usually it's unnecessary to "register" the test anywhere
|
|||
as if it's being added to an existing directory, depending
|
||||
on how the ``run-test.sh`` in that directory works.
|
||||
|
||||
.. tip::
|
||||
|
||||
Automatically run on pushes to main and PRs, under both macOS and Linux
|
||||
(``test-build.yml``)
|
||||
|
||||
.. TODO:: testing with sanitizers
|
||||
|
||||
- Automatically run on pushes to main and during PR merges, under both macOS
|
||||
and Linux, using clang's UBSAN and ASAN (``test-sanitizers.yml``)
|
||||
- run locally with :makevar:`SANITIZER` (clean build first)
|
||||
|
||||
Unit tests
|
||||
~~~~~~~~~~
|
||||
|
||||
|
|
@ -47,6 +74,11 @@ Running the unit tests requires the following additional packages:
|
|||
|
||||
Unit tests can be run with ``make unit-test``.
|
||||
|
||||
.. tip::
|
||||
|
||||
Automatically run on pushes to main and PRs, under both macOS and Linux
|
||||
(``test-build.yml``)
|
||||
|
||||
Functional tests
|
||||
~~~~~~~~~~~~~~~~
|
||||
|
||||
|
|
@ -78,6 +110,11 @@ instructions <https://github.com/Z3Prover/z3>`_.
|
|||
Then, set the :makevar:`ENABLE_FUNCTIONAL_TESTS` make variable when calling
|
||||
``make test`` and the functional tests will be run as well.
|
||||
|
||||
.. tip::
|
||||
|
||||
Automatically run as part of
|
||||
:ref:`yosys_internals/extending_yosys/ci:Automatic testing with Verific`
|
||||
|
||||
Docs tests
|
||||
~~~~~~~~~~
|
||||
|
||||
|
|
@ -87,17 +124,10 @@ documentation, which can be run by calling ``make test`` from the
|
|||
also includes checking some macro commands to ensure that descriptions of them
|
||||
are kept up to date, and is mostly intended for CI.
|
||||
|
||||
.. tip::
|
||||
|
||||
Automatic testing
|
||||
-----------------
|
||||
Automatically run on pushes to main and PRs, under Linux (``test-build.yml``)
|
||||
|
||||
The `Yosys Git repo`_ has automatic testing of builds and running of the
|
||||
included test suite on both Ubuntu and macOS, as well as across range of
|
||||
compiler versions. For up to date information, including OS versions, refer to
|
||||
`the git actions page`_.
|
||||
|
||||
.. _Yosys Git repo: https://github.com/YosysHQ/yosys
|
||||
.. _the git actions page: https://github.com/YosysHQ/yosys/actions
|
||||
|
||||
..
|
||||
How to add a unit test
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue