Previously CMake was not aware of which headers files the generation
of `mem_initializer.cpp` depended on. Consequently this could result
in broken incremental builds if
* Existing headers that declare memory initializers/finalizers change.
* New headers are added that declare memory initializers/finalizer.
Now the `z3_add_component()` CMake function has been modifed so that
it now takes an optional `MEMORY_INIT_FINALIZER_HEADERS` argument
which allows the headers that declare memory initializers/finalizers
to be explicitly listed.
With this information CMake will now regenerate `mem_initializer.cpp`
correctly.
This required the `mk_mem_initializer_cpp_internal()` function to be
changed to take a list of header files rather than a list of component
source directories. The two consumers (CMake and Python/Makefile build
systems) of this function have been modified to work with this change.
This partially fixes#1030.
Previously CMake was not aware of which headers files the generation
of `gparams_register_modules.cpp` depended on. Consequently this could result
in broken incremental builds if
* Existing headers that declared module description/parameters change.
* New headers are added that declare module description/parameters.
* `.pyg` files that generate header files that declare module
description/parameters change
Now the `z3_add_component()` CMake function has been modifed so that
* All header files that are generated from `.pyg` files are added as
dependencies and are scanned from module description/parameter
declarations. This implicit dependency of `gparams_register_modules.cpp`
depending on other generated header files seems unnecessary complex. We
should revisit this design decision once the Python/Makefile build
system is deprecated.
* The function now takes an optional `EXTRA_REGISTER_MODULE_HEADERS`
argument which allows the headers that declare module
description/paramters to be explicitly listed.
With this information CMake will now regenerate `gparams_register_modules.cpp`
correctly.
This required the `mk_gparams_register_modules_internal()` function to be
changed to take a list of header files rather than a list of component
source directories. The two consumers (CMake and Python/Makefile build
systems) of this function have been modified to work with this change.
This partially fixes#1030.
Previously CMake was not aware of which headers files the generation
of `install_tactic.cpp` depended on. Consequently this could result
in broken incremental builds if
* Existing headers that declared tactics/probes changed.
* New tactics/probes were added to new header files.
Now the `z3_add_component()` CMake function has been modifed to take an
optional `TACTIC_HEADERS` argument which allows the headers that declare
tactics/probes to be explicitly listed. The necessary component
declarations have been modified to declare their tactic/probe header
files.
With this information CMake will now regenerate `install_tactic.cpp`
correctly.
This required the `mk_install_tactic_cpp_internal()` function to be
changed to take a list of header files rather than a list of component
source directories. The two consumers (CMake and Python/Makefile build
systems) of this function have been modified to work with this change.
This partially fixes#1030.
and move that into `mk_genfile_common.py`. Then adapt `mk_util.py` and
`mk_consts_files.py` to call into the code at its new location.
The purpose of this change is to have Python code common to the Python
and CMake build systems separate from Python code that is only used for
the Python build system.
enum values is not consistent between python 2 or 3. The root cause
of the problem was a dictionary's keys being iterated over which has
no defined order.
This has been fixed by iterating over the dictionary's items and
ordering by values. We could order by key rather than the values but
seeing as these represent an enum, ordering by value makes more sense.
and move that into ``mk_genfile_common.py``. Then adapt ``mk_util.py`` and
``mk_consts_files.py`` to call into the code at its new location.
The purpose of this change is to have Python code common to the Python
and CMake build systems separate from Python code that is only used for
the Python build system.
The *.cmx files are now installed using ocamlfind. They contain information from the
compiler that can be used during optimization (the upcoming OCaml 4.03.0 issues
warning 58 if those files are missing from a package).
and move that into ``mk_genfile_common.py``. Then adapt ``mk_util.py`` and
``mk_consts_files.py`` to call into the code at its new location.
The purpose of this change is to have Python code common to the Python
and CMake build systems separate from Python code that is only used for
the Python build system.
This will still use 'ar' if AR isn't set in the environment, but lets
us override the default archive tool at configure time.
Just like CC and CXX, this doesn't apply to a ./configure for Windows.
``mk_genfile_common.py``. A new function ``mk_hpp_from_pyg()`` has been added
which provides a more sensible interface (hides the nasty ``exec()`` stuff)
to create the ``*.hpp`` files from ``*.pyg`` files.
Both ``mk_util.py`` and ``pyg2hpp.py`` have been modified to use the new
interface.
Whilst I'm here reindent ``pyg2hpp.py``.
The purpose of this change is to have Python code common to the Python
and CMake build systems separate from Python code that is only used for
the Python build system.
I've tested this change by making sure that the all the ``*.hpp``
files generated from ``*.pyg`` files match the files generated
before this change.
``mk_genfile_common.py`` and adapt ``mk_util.py`` and
``mk_pat_db.py`` to use the code at its new location.
Whilst I'm here reindent ``mk_mem_initializer_cpp.py``.
The purpose of this change is to have Python code common to the Python
and CMake build systems separate from Python code that is only used for
the Python build system.
``mk_genfile_common.py`` and adapt ``mk_util.py`` and
``mk_mem_initializer_cpp.py`` to use the code at its new location. The interface
has been changed slightly so that ``mk_mem_initializer_cpp_internal()`` now
returns the path the generated file. The motivation behind this is so that
clients of the function know the path of the generated file.
Whilst I'm here reindent ``mk_mem_initializer_cpp.py`` and the relevant
code in ``mk_util.py``.
The purpose of this change is to have Python code common to the Python
and CMake build systems separate from Python code that is only used for
the Python build system.
``mk_genfile_common.py`` and adapt ``mk_util.py`` and
``mk_install_tactic_cpp.py`` to use the code at its new location. The interface
has been changed slightly so that ``mk_install_tactic_cpp_internal()`` now
returns the path the generated file. The motivation behind this is so that
clients of the function know the path of the generated file.
Whilst I'm here reindent ``mk_install_tactic_cpp.py`` and the relevant
code in ``mk_util.py``.
The purpose of this change is to have Python code common to the Python
and CMake build systems separate from Python code that is only used for
the Python build system.
to ``mk_genfile_common.py`` and adapt ``mk_util.py`` and
``mk_gparams_register_modules_cpp.py`` to use the code at its new
location. The interface has been changed slightly so that
``mk_gparams_register_modules_internal()`` now returns the path
to the generated file. The motivation behind this so that clients
of the function know the path to the generated file.
Whilst I'm here reindent ``mk_gparams_register_modules_cpp.py``
and the relevant code in ``mk_util.py``.
Also remove duplicated code that is now available in
``mk_genfile_common.py``.
The purpose of this change is to have Python code common to the Python
and CMake build systems separate from Python code that is only used for
the Python build system.
``mk_genfile_common.py`` and adapt ``mk_util.py`` and
``mk_def_file.py`` to use the code at its new location.
Whilst I'm here also reindent ``mk_def_file.py`` and make it
use some of the code in ``mk_genfile_common.py`` to avoid code
duplication.
The purpose of this change is to have Python code common to the Python
and CMake build systems separate from Python code that is only used for
the Python build system.
``mk_genfile_common.py`` and adapt ``mk_util.py`` and
``mk_consts_files.py`` to use the code at its new location. The
interface has been changed slightly so that
``mk_z3consts_py_internal()`` now returns the path to the generated
file. The motivation behind this is so that clients of the function
know the path to the generated file.
Whilst I'm here also reindent ``mk_consts_files.py`` and move some of
its code into ``mk_genfile_common.py`` so it can be shared.
Also update Z3_GENERATED_FILE_EXTRA_DEPENDENCIES in the CMake build
so it knows about ``mk_genfile_common.py``.
The purpose of this change is to have Python code common to the Python
and CMake build systems separate from Python code that is only used for
the Python build system.
function ``mk_z3consts_py_internal()`` which called by a new
``mk_consts_files.py`` script. This will script will allow the const
declarations for the different Z3 language bindings to be generated.
Right now only support for python is implemented but more can be added
in the future.
function ``mk_def_file_internal()`` which is called by a
new ``mk_def_file.py`` script. This will allow other build systems to
generate the ``api_dll.def`` file.
be used as a script for other build systems and is callable via
a new ``generate_files()`` function from ``mk_util.py``. This removes
the horrible ``_execfile()`` hack that was previously in ``mk_util.py``.
Unfortunately lots of bad code is still in ``update_api.py`` but
fixing all of its problems is too much work right now.
function ``mk_gparams_register_modules_internal()`` which is called by a
new ``mk_gparams_register_modules_cpp.py`` script. This will allow other build systems to
generate the ``gparams_register_modules.cpp`` files.
function ``mk_mem_initializer_cpp_internal()`` which is called by a
new ``mk_mem_initializer_cpp.py`` script. This will allow other build systems to
generate the ``mem_initializer.cpp`` files.
function ``mk_install_tactic_cpp_internal()`` which is called by a
new ``mk_install_tactic_cpp.py`` script. This will allow other build systems to
generate the ``install_tactic.cpp`` files.
build directory would cause REV_BUILD_DIR to be set incorrectly and
would lead to a broken Makefile being generated.
What would happen before:
```
$ python scripts/mk_make.py --build FOO_1
...
REV_BUILD_DIR='..'
```
```
$ python scripts/mk_make.py --build FOO_1/
...
REV_BUILD_DIR='../..'
```
^^^^^ that's wrong. It should be REV_BUILD_DIR='..'
To fix this the ``reverse_path()`` function has been taught to ignore empty
components in ``p.split(os.sep)``.
when using Python2.
The output from ``check_output()`` has ``unicode`` type under
Python 2 but type ``str`` under Python 3. This type ended up being
used inside the ``MakeRuleCmd`` class which asserts that it receives
paths of type ``str``. To fix the problem under Python 2 the asserts
have been made weaker by also allowing the paths to be of type
``unicode``.
variable. Previously it would try to install into the system (e.g.
``/usr/lib/ocaml``) regardless of the value of DESTDIR.
Unfortunately it looks like packagers who use DESTDIR to do staged
installs will need to have their packages patch their user's OCaml
``ld.conf`` file manually at package install time (not ``make install``
time) with the extra path to the Z3 Ocaml package directory. We could
use the ``touch`` command to create an empty ``ld.conf`` before running
``ocamlfind install`` but that adds the wrong path to ``ld.conf``
because it contains DESTDIR.
bindings is enabled. That is really unhelpful behaviour. Instead emit a
warning. I would prefer an error message but apparently being able to
build but not install the OCaml bindings is desirable.
Whilst I'm here also print information about ocamlfind where it should
have been mentioned.
Previous method is_osx_hack didn't exist, possibly wasn't cleaned up. This
change has been tested and Z3 compiles successfully within a 2.7 virtualenv
running on OS X 10.10.5.
for the ".NET" bindings example.
Previously there was a hack that would to path separator replacement
( '/' -> '\' ) which breaks paths under non-windows platforms. The hack
has been made slightly better by doing ( '/' -> os.path.sep). This
preserves the existing behaviour under Windows and unbreaks the build
on non-windows platforms.
I'm not entirely sure why the path replacement needs to be done in
the first place. I thought Windows was supposed to support using
'/' as a path separator (as well as '/'). Maybe ``csc.exe`` doesn't
support these kind of paths?
the install prefix on OSX less aggressive by not stating that this might
lead to a broken install (even though it might!).
Also emit a similar warning during the configuration step.
This partially addresses #361
the install prefix on OSX less aggressive by not stating that this might
lead to a broken install (even though it might!).
Also emit a similar warning during the configuration step.
This partially addresses #361
under OSX when the Python ``site-packages`` directory doesn't live
under the install prefix. Also disallow installing under Windows
as it doesn't really make sense.
was not created before trying to create files in it.
The ``install_deps()`` method was also dead. It looks like that
should have been named ``mk_install_deps()`` but even if we give it
the right name it does the wrong thing. That method is supposed
emit target names (i.e. dependencies) not commands.
".NET" bindings. This file is required for Monodevelop to find the
bindings because Monodevelop uses pkg-config to find packages
(it doesn't use the GAC).
For lack of a better name the GAC (and pkg-config) package name is now
``Microsoft.Z3.Sharp``. I don't want to call it ``Microsoft.Z3`` because
someone may want to create a ``Microsoft.Z3.pc`` file in the future for
the native Z3 library (i.e. C++ or C bindings).
In addition there is a new utility function ``configure_file()``
which reads a template file, applies some substitutions and writes
the result to another file. This very similar to what CMake does.
There is a new environment variable ``Z3_INSTALL_PKGCONFIG_DIR``
which allows pkgconfig directory to be controlled for the install.
under non Windows systems (i.e. Using mono).
Building these bindings is unfortunately on by default because
I didn't want to change the command line interface (i.e. ``--nodotnet``)
which people might be relying on. This should really be changed to
match the other binding flags (e.g. ``--java``) but I will leave
this for now.
To perform the build a C# compiler and the GAC utility are required.
The script will try to automatically detect them but the user can
override this by setting the ``CSC`` and ``GACUTIL`` environment
variables.
In order for the ".NET bindings" to be installed the assembly
(``Microsoft.Z3.dll``) needs to have a strong name which means
we need a Strong name key file which is what the
``Microsoft.Z3.mono.snk`` is for. This is the public and private
key so this key **must never** be used for checking integrity. Instead its
only purpose is to avoid any name clashes in the GAC.
It is also worth noting that slightly different flags needs to
be passed to the C# compiler on non Windows platforms. I don't
understand why some of the flags are being used on Windows but I left
a comment there that hopefully someone can fix...
variables:
Z3_INSTALL_BIN_DIR - defaults to "bin"
Z3_INSTALL_LIB_DIR - defaults to "lib"
Z3_INSTALL_INCLUDE_DIR - defaults to "include"
This has two advantages
* We no longer hard code strings like "bin" all over the place
* Packagers can easily control where things get installed.
create_relative_symbolic_link(out, '/usr/lib64/libz3.so',
'/usr/lib/python3.5/site-package/libz3.so') would create an incorrect relative
path because it would consider ``/usr/lib`` to a be a path prefix of
``/usr/lib64``.
which would try to uninstall components that were never installed.
This bug would cause the following line to be emitted in the
``Makefile`` under the ``uninstall`` rule even though there was
no corresponding rule to install the file under the ``install`` rule.
```
@rm -f $(DESTDIR)$(PREFIX)/bin/test-z3$(EXE_EXT)
```
duplicated in too many places by refactoring the installation and
removal of the Python bindings to use the ``MakeRuleCmd`` class.
In order to make this change:
* ``PYTHON_PACKAGE_DIR`` no longer contains the text ``$(PREFIX)``
* ``PYTHON_PACKAGE_DIR`` **MUST BE** inside the installation prefix
duplicated in too many places and being worried that someone might
forget to use it when installing additional components.
To acheive this the new ``MakeRuleCmd`` class provides
several class methods to generate commonly needed commands used in
make file rules.
Most of the build system has been changed to use these helper methods
apart from stuff related to the Python bindings. This can't be changed
until we fix how PYTHON_PACKAGE_DIR is handled. Right it not guaranteed
to live under the install prefix but this is a requirement when using
the ``MakeRuleCmd`` methods.
There were several problems with the existing implementation.
* When specifying ``--prefix`` the implementation would assume the
path was ``$(PREFIX)/lib/python-<VERSION>/dist-packages``. This
is incorrect. ``dist-packages`` is Debian (and its derivatives,
i.e Ubuntu) specific and won't work on other Linux distributions
such as Arch Linux.
* When generating the ``Makefile``, ``$(PREFIX)`` was only emitted
during the Python installation when ``--prefix`` was passed on
the command line. When ``--prefix`` was not passed the absolute
path to the Python package directory was emitted. This is not
very consistent.
This patch checks that the detected Python package directory lives
under the install prefix and emits an error if it does not as this
indicates that the installation will be broken. If the Python package
directory does live under the install prefix it replaces that prefix
with the ``$(PREFIX)`` variable when emitting the ``Makefile`` for
consistency with the other install commands.
If a user really wants to install to a particular Python package
directory they can force it with the newly added ``--pypkgdir=``
flag.
when installing the Python bindings.
If ``DESTDIR`` is set the bindings will now be installed under this
path but ``$(PREFIX)`` only appears in the ``Makefile`` if ``--prefix``
was set which seems a little broken (we'll fix this soon).
The creation of the Python ``site-packages`` (and ``__pycache__`` for Python
3) directories has been moved to build time instead of configure time
because we don't know what ``DESTDIR`` will be set to at configure time.
DESTDIR make file variable (https://www.gnu.org/prep/standards/html_node/DESTDIR.html)
for ``install`` and ``uninstall`` targets.
Typically packagers build packages like so
```
$ ./configure --prefix=/usr/
$ make
$ make DESTDIR=/some/path/ install
```
Doing this installs the files into a directory ``/some/path`` but places
files inside that directory using the layout in ``--prefix`` (e.g.
``/some/path/usr/bin/z3``). The ``/some/path`` directory can then be
packaged (e.g. tarballed) for later installation.
The ``DESTDIR`` is not set in the Makefile and thus is empty by default
which maintains the existing ``make install`` behaviour.
Unfortunately this situation isn't fixed for the Python bindings (and
possibly others) yet as more invasive changes are needed here. I'll fix
this in later commits.
