1.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
1.2 +++ b/build/docs/build-overview.rst Wed Dec 31 06:09:35 2014 +0100
1.3 @@ -0,0 +1,117 @@
1.4 +.. _build_overview:
1.5 +
1.6 +=====================
1.7 +Build System Overview
1.8 +=====================
1.9 +
1.10 +This document provides an overview on how the build system works. It is
1.11 +targeted at people wanting to learn about internals of the build system.
1.12 +It is not meant for persons who casually interact with the build system.
1.13 +That being said, knowledge empowers, so consider reading on.
1.14 +
1.15 +The build system is composed of many different components working in
1.16 +harmony to build the source tree. We begin with a graphic overview.
1.17 +
1.18 +.. graphviz::
1.19 +
1.20 + digraph build_components {
1.21 + rankdir="LR";
1.22 + "configure" -> "config.status" -> "build backend" -> "build output"
1.23 + }
1.24 +
1.25 +Phase 1: Configuration
1.26 +======================
1.27 +
1.28 +Phase 1 centers around the ``configure`` script, which is a bash shell script.
1.29 +The file is generated from a file called ``configure.in`` which is written in M4
1.30 +and processed using Autoconf 2.13 to create the final configure script.
1.31 +You don't have to worry about how you obtain a ``configure`` file: the build
1.32 +system does this for you.
1.33 +
1.34 +The primary job of ``configure`` is to determine characteristics of the system
1.35 +and compiler, apply options passed into it, and validate everything looks OK to
1.36 +build. The primary output of the ``configure`` script is an executable file
1.37 +in the object directory called ``config.status``. ``configure`` also produces
1.38 +some additional files (like ``autoconf.mk``). However, the most important file
1.39 +in terms of architecture is ``config.status``.
1.40 +
1.41 +The existence of a ``config.status`` file may be familiar to those who have worked
1.42 +with Autoconf before. However, Mozilla's ``config.status`` is different from almost
1.43 +any other ``config.status`` you've ever seen: it's written in Python! Instead of
1.44 +having our ``configure`` script produce a shell script, we have it generating
1.45 +Python.
1.46 +
1.47 +Now is as good a time as any to mention that Python is prevalent in our build
1.48 +system. If we need to write code for the build system, we do it in Python.
1.49 +That's just how we roll. For more, see :ref:`python`.
1.50 +
1.51 +``config.status`` contains 2 parts: data structures representing the output of
1.52 +``configure`` and a command-line interface for preparing/configuring/generating
1.53 +an appropriate build backend. (A build backend is merely a tool used to build
1.54 +the tree - like GNU Make or Tup). These data structures essentially describe
1.55 +the current state of the system and what the existing build configuration looks
1.56 +like. For example, it defines which compiler to use, how to invoke it, which
1.57 +application features are enabled, etc. You are encouraged to open up
1.58 +``config.status`` to have a look for yourself!
1.59 +
1.60 +Once we have emitted a ``config.status`` file, we pass into the realm of
1.61 +phase 2.
1.62 +
1.63 +Phase 2: Build Backend Preparation and the Build Definition
1.64 +===========================================================
1.65 +
1.66 +Once ``configure`` has determined what the current build configuration is,
1.67 +we need to apply this to the source tree so we can actually build.
1.68 +
1.69 +What essentially happens is the automatically-produced ``config.status`` Python
1.70 +script is executed as soon as ``configure`` has generated it. ``config.status``
1.71 +is charged with the task of tell a tool how to build the tree. To do this,
1.72 +``config.status`` must first scan the build system definition.
1.73 +
1.74 +The build system definition consists of various ``moz.build`` files in the tree.
1.75 +There is roughly one ``moz.build`` file per directory or per set of related directories.
1.76 +Each ``moz.build`` files defines how its part of the build config works. For
1.77 +example it says *I want these C++ files compiled* or *look for additional
1.78 +information in these directories.* config.status starts with the ``moz.build``
1.79 +file from the root directory and then descends into referenced ``moz.build``
1.80 +files by following ``DIRS`` variables or similar.
1.81 +
1.82 +As the ``moz.build`` files are read, data structures describing the overall
1.83 +build system definition are emitted. These data structures are then fed into a
1.84 +build backend, which then performs actions, such as writing out files to
1.85 +be read by a build tool. e.g. a ``make`` backend will write a
1.86 +``Makefile``.
1.87 +
1.88 +When ``config.status`` runs, you'll see the following output::
1.89 +
1.90 + Reticulating splines...
1.91 + Finished reading 1096 moz.build files into 1276 descriptors in 2.40s
1.92 + Backend executed in 2.39s
1.93 + 2188 total backend files. 0 created; 1 updated; 2187 unchanged
1.94 + Total wall time: 5.03s; CPU time: 3.79s; Efficiency: 75%
1.95 +
1.96 +What this is saying is that a total of *1096* ``moz.build`` files were read.
1.97 +Altogether, *1276* data structures describing the build configuration were
1.98 +derived from them. It took *2.40s* wall time to just read these files and
1.99 +produce the data structures. The *1276* data structures were fed into the
1.100 +build backend which then determined it had to manage *2188* files derived
1.101 +from those data structures. Most of them already existed and didn't need
1.102 +changed. However, *1* was updated as a result of the new configuration.
1.103 +The whole process took *5.03s*. Although, only *3.79s* was in
1.104 +CPU time. That likely means we spent roughly *25%* of the time waiting on
1.105 +I/O.
1.106 +
1.107 +For more on how ``moz.build`` files work, see :ref:`mozbuild-files`.
1.108 +
1.109 +Phase 3: Invokation of the Build Backend
1.110 +========================================
1.111 +
1.112 +When most people think of the build system, they think of phase 3. This is
1.113 +where we take all the code in the tree and produce Firefox or whatever
1.114 +application you are creating. Phase 3 effectively takes whatever was
1.115 +generated by phase 2 and runs it. Since the dawn of Mozilla, this has been
1.116 +make consuming Makefiles. However, with the transition to moz.build files,
1.117 +you may soon see non-Make build backends, such as Tup or Visual Studio.
1.118 +
1.119 +When building the tree, most of the time is spent in phase 3. This is when
1.120 +header files are installed, C++ files are compiled, files are preprocessed, etc.
