From e4266ead987dcc45c4235450fe113eff06f2fa15 Mon Sep 17 00:00:00 2001 From: Pekka Enberg Date: Fri, 4 Sep 2020 11:44:43 +0300 Subject: [PATCH] Improve build documentation This improves the build documentation beyond just packaging: - Explain how the configure.py step works - Explain how to build just executables and tests (for development) - Explain how to build for specific build mode if you didn't specify a build mode in configure.py step - Fix build artifact locations, missing .debs, and add executables and tests Message-Id: <20200904084443.495137-1-penberg@iki.fi> --- README.md | 4 +- docs/building-packages.md | 60 -------------- docs/building.md | 169 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 171 insertions(+), 62 deletions(-) delete mode 100644 docs/building-packages.md create mode 100644 docs/building.md diff --git a/README.md b/README.md index e6817a64bc..f045f09fca 100644 --- a/README.md +++ b/README.md @@ -38,11 +38,11 @@ $ ./tools/toolchain/dbuild ninja build/release/scylla For further information, please see: * [Developer documentation] for more information on building Scylla. -* [Packaging documentation] on how to build Scylla packages for different Linux distributions. +* [Build documentation] on how to build Scylla binaries, tests, and packages. * [Docker image build documentation] for information on how to build Docker images. [developer documentation]: HACKING.md -[packaging documentation]: docs/building-packages.md +[build documentation]: docs/building.md [docker image build documentation]: dist/docker/redhat/README.md ## Running Scylla diff --git a/docs/building-packages.md b/docs/building-packages.md deleted file mode 100644 index 5b0c189c83..0000000000 --- a/docs/building-packages.md +++ /dev/null @@ -1,60 +0,0 @@ -# Building Scylla Packages - -This document describes how to build Scylla's packages. - -The build system generates _relocatable packages_, which means that the packages contain all the dependencies they need, and you can, therefore, install and run the same binaries on all Linux distributions. -The relocatable package tarball is used as a base for building the Linux distribution specific packages, `.rpm`s and `.deb`s. - -## Building - -In these instructions, we use `dbuild` to build the packages, but you can also build packages without it. - -The first step is to run `configure.py`, which generates the `build.ninja` file (equivalent of a `Makefile`): - -``` -./tools/toolchain/dbuild ./configure.py --mode= -``` - -(where `mode` is `release`, `dev`, or `debug`) - -The second step is to build the packages with the `dist` target. - -``` -./tools/toolchain/dbuild ninja-build dist -``` - -## Artifacts - -The `dist` target generates the following tarballs (*relocatable packages*), which can be installed on any Linux distribution: - -### Relocatable package artifacts - -* `build//dist/tar/scylla-package.tar.gz` -* `build//dist/tar/scylla-python3-package.tar.gz` -* `build//dist/tar/scylla-jmx-package.tar.gz` -* `build//dist/tar/scylla-tools-package.tar.gz` - -### RPM package artifacts - -* `build/dist//redhat/RPMS/x86_64/scylla-*.rpm` -* `build/dist//redhat/RPMS/x86_64/scylla-conf-*.rpm` -* `build/dist//redhat/RPMS/x86_64/scylla-debuginfo-*.rpm` -* `build/dist//redhat/RPMS/x86_64/scylla-kernel-conf-*.rpm` -* `build/dist//redhat/RPMS/x86_64/scylla-server-*.rpm` -* `build/redhat/RPMS/x86_64/scylla-python3-*.rpm` -* `scylla-jmx/build/redhat/RPMS/noarch/scylla-jmx-*.rpm` -* `scylla-tools/build/redhat/RPMS/noarch/scylla-tools-*.rpm` -* `scylla-tools/build/redhat/RPMS/noarch/scylla-tools-core-*.rpm` - -### Debian package artifacts - -## Verifying - -To verify built Scylla packages, run the following command: - -``` -ninja-build dist-check -``` - -Please note that you need to run `dist-check` on the host because it -requires Docker to perform the verification. diff --git a/docs/building.md b/docs/building.md new file mode 100644 index 0000000000..2639f84c2d --- /dev/null +++ b/docs/building.md @@ -0,0 +1,169 @@ +# Building Scylla + +This document describes how to build Scylla's executables, tests, and packages. + +*Please note that these instructions use `dbuild` -- a Docker-based development environment -- to build Scylla. +However, `dbuild` is optional and you can also build on your host machine by running the same commands without the `dbuild` prefix.* + +## Table of Contents + +* [Getting Started](#getting-started) +* [Configuring](#configuring) +* [Building](#building) +* [Packaging](#packaging) +* [Artifacts](#artifacts) + +## Getting Started + +To build everything, run: + +```console +git clone https://github.com/scylladb/scylla +git submodule update --init --force --recursive +./tools/toolchain/dbuild ./configure.py --mode= +./tools/toolchain/dbuild ninja +``` + +where `mode` is `dev` for development builds, `release` for release builds, and `debug` for debug builds. + +Alternatively, you can also pass the `mode` directly to the `ninja` command: + +```console +git submodule update --init --force --recursive +./tools/toolchain/dbuild ./configure.py +./tools/toolchain/dbuild ninja +``` + +After the build completes, you can find build artifacts in `build/` directory. + +You can run unit tests with: + +```console +./tools/toolchain/dbuild test +``` + +or launch a Scylla server locally with: + +```console +./tools/toolchain/dbuild ./build/dev/scylla --workdir tmp --developer-mode 1 --smp 1 --memory 1G +``` + +That's it! + +## Configuring + +The `configure.py` script, which is run as the first step of a build, generates a `build.ninja` build file (similar to a `Makefile`) for the [Ninja] build tool that we use for building. + +To configure the build system for a specific build mode, run: + +```console +./tools/toolchain/dbuild ./configure.py --mode= +``` + +The `mode` flag determines build flags for the `scylla` executable: + +* `release` is the release build that targets fast execution time (slow build, fast execution). +* `dev` is the development build that targets fast build times (fast build, reasonable execution) +* `debug` is the debug build that targets error discovery (slow build, slow execution). + +If you don't specify a build mode, the `build.ninja` will contain configuration for _all build modes_. + +[Ninja]: https://ninja.org/ + +## Building + +To build Scylla executables and tests, run: + +```console +./tools/toolchain/dbuild ninja build +``` + +This will build Scylla for the build mode specified by the `configure.py` command. + +Alternatively, if you did not specify a build mode in the `configure.py` step, you can build executables and tests for a specific build mode with: + +```console +./tools/toolchain/dbuild ninja -build +``` + +## Testing + +To run unit tests, run: + +```console +./tools/toolchain/dbuild ninja test +``` + +Alternatively, to run unit tests for a specific build mode, run: + +```console +./tools/toolchain/dbuild ninja -test +``` + +## Packaging + +The build system generates _relocatable packages_, which means that the packages contain all the dependencies they need, and you can, therefore, install and run the same binaries on all Linux distributions. +The relocatable package tarball is used as a base for building the Linux distribution specific packages, `.rpm`s and `.deb`s. + +To build packages, run: + +```console +./tools/toolchain/dbuild ninja dist +``` + +Alternatively, to run unit tests for a specific build mode, run: + +```console +./tools/toolchain/dbuild ninja dist- +``` + +To verify the packages, run: + +```console +ninja dist-check +``` + +*Please note that you need to run `dist-check` on the host because it requires Docker to perform the verification.* + +## Artifacts + +This section lists all the artifacts generated by the build system. + +**Executables**: + +* `build//scylla` + +**Tests**: + +* `build//test/**` + +**Tarballs**: + +* `build//dist/tar/scylla-package.tar.gz` +* `build//dist/tar/scylla-python3-package.tar.gz` +* `build//dist/tar/scylla-jmx-package.tar.gz` +* `build//dist/tar/scylla-tools-package.tar.gz` + +**.rpms:** + +* `build/dist//redhat/RPMS/x86_64/scylla-*.rpm` +* `build/dist//redhat/RPMS/x86_64/scylla-conf-*.rpm` +* `build/dist//redhat/RPMS/x86_64/scylla-debuginfo-*.rpm` +* `build/dist//redhat/RPMS/x86_64/scylla-kernel-conf-*.rpm` +* `build/dist//redhat/RPMS/x86_64/scylla-server-*.rpm` +* `tools/python3/build/redhat/RPMS/x86_64/scylla-python3-*.rpm` +* `tools/jmx/build/redhat/RPMS/noarch/scylla-jmx-*.rpm` +* `tools/java/build/redhat/RPMS/noarch/scylla-tools-*.rpm` +* `tools/java/build/redhat/RPMS/noarch/scylla-tools-core-*.rpm` + +**.debs:** + +* `build/dist/dev/debian/scylla-conf_*.deb` +* `build/dist/dev/debian/scylla_*.deb` +* `build/dist/dev/debian/scylla-server_*.deb` +* `build/dist/dev/debian/scylla-kernel-conf_*.deb` +* `build/dist/dev/debian/scylla-server-dbg_*.deb` +* `tools/python3/build/debian/scylla-python3_*.deb` +* `tools/jmx/build/debian/scylla-jmx_*.deb` +* `tools/java/build/debian/scylla-tools-core_*.deb` +* `tools/java/build/debian/scylla-tools_*.deb`