michael@0: .. _build_overview: michael@0: michael@0: ===================== michael@0: Build System Overview michael@0: ===================== michael@0: michael@0: This document provides an overview on how the build system works. It is michael@0: targeted at people wanting to learn about internals of the build system. michael@0: It is not meant for persons who casually interact with the build system. michael@0: That being said, knowledge empowers, so consider reading on. michael@0: michael@0: The build system is composed of many different components working in michael@0: harmony to build the source tree. We begin with a graphic overview. michael@0: michael@0: .. graphviz:: michael@0: michael@0: digraph build_components { michael@0: rankdir="LR"; michael@0: "configure" -> "config.status" -> "build backend" -> "build output" michael@0: } michael@0: michael@0: Phase 1: Configuration michael@0: ====================== michael@0: michael@0: Phase 1 centers around the ``configure`` script, which is a bash shell script. michael@0: The file is generated from a file called ``configure.in`` which is written in M4 michael@0: and processed using Autoconf 2.13 to create the final configure script. michael@0: You don't have to worry about how you obtain a ``configure`` file: the build michael@0: system does this for you. michael@0: michael@0: The primary job of ``configure`` is to determine characteristics of the system michael@0: and compiler, apply options passed into it, and validate everything looks OK to michael@0: build. The primary output of the ``configure`` script is an executable file michael@0: in the object directory called ``config.status``. ``configure`` also produces michael@0: some additional files (like ``autoconf.mk``). However, the most important file michael@0: in terms of architecture is ``config.status``. michael@0: michael@0: The existence of a ``config.status`` file may be familiar to those who have worked michael@0: with Autoconf before. However, Mozilla's ``config.status`` is different from almost michael@0: any other ``config.status`` you've ever seen: it's written in Python! Instead of michael@0: having our ``configure`` script produce a shell script, we have it generating michael@0: Python. michael@0: michael@0: Now is as good a time as any to mention that Python is prevalent in our build michael@0: system. If we need to write code for the build system, we do it in Python. michael@0: That's just how we roll. For more, see :ref:`python`. michael@0: michael@0: ``config.status`` contains 2 parts: data structures representing the output of michael@0: ``configure`` and a command-line interface for preparing/configuring/generating michael@0: an appropriate build backend. (A build backend is merely a tool used to build michael@0: the tree - like GNU Make or Tup). These data structures essentially describe michael@0: the current state of the system and what the existing build configuration looks michael@0: like. For example, it defines which compiler to use, how to invoke it, which michael@0: application features are enabled, etc. You are encouraged to open up michael@0: ``config.status`` to have a look for yourself! michael@0: michael@0: Once we have emitted a ``config.status`` file, we pass into the realm of michael@0: phase 2. michael@0: michael@0: Phase 2: Build Backend Preparation and the Build Definition michael@0: =========================================================== michael@0: michael@0: Once ``configure`` has determined what the current build configuration is, michael@0: we need to apply this to the source tree so we can actually build. michael@0: michael@0: What essentially happens is the automatically-produced ``config.status`` Python michael@0: script is executed as soon as ``configure`` has generated it. ``config.status`` michael@0: is charged with the task of tell a tool how to build the tree. To do this, michael@0: ``config.status`` must first scan the build system definition. michael@0: michael@0: The build system definition consists of various ``moz.build`` files in the tree. michael@0: There is roughly one ``moz.build`` file per directory or per set of related directories. michael@0: Each ``moz.build`` files defines how its part of the build config works. For michael@0: example it says *I want these C++ files compiled* or *look for additional michael@0: information in these directories.* config.status starts with the ``moz.build`` michael@0: file from the root directory and then descends into referenced ``moz.build`` michael@0: files by following ``DIRS`` variables or similar. michael@0: michael@0: As the ``moz.build`` files are read, data structures describing the overall michael@0: build system definition are emitted. These data structures are then fed into a michael@0: build backend, which then performs actions, such as writing out files to michael@0: be read by a build tool. e.g. a ``make`` backend will write a michael@0: ``Makefile``. michael@0: michael@0: When ``config.status`` runs, you'll see the following output:: michael@0: michael@0: Reticulating splines... michael@0: Finished reading 1096 moz.build files into 1276 descriptors in 2.40s michael@0: Backend executed in 2.39s michael@0: 2188 total backend files. 0 created; 1 updated; 2187 unchanged michael@0: Total wall time: 5.03s; CPU time: 3.79s; Efficiency: 75% michael@0: michael@0: What this is saying is that a total of *1096* ``moz.build`` files were read. michael@0: Altogether, *1276* data structures describing the build configuration were michael@0: derived from them. It took *2.40s* wall time to just read these files and michael@0: produce the data structures. The *1276* data structures were fed into the michael@0: build backend which then determined it had to manage *2188* files derived michael@0: from those data structures. Most of them already existed and didn't need michael@0: changed. However, *1* was updated as a result of the new configuration. michael@0: The whole process took *5.03s*. Although, only *3.79s* was in michael@0: CPU time. That likely means we spent roughly *25%* of the time waiting on michael@0: I/O. michael@0: michael@0: For more on how ``moz.build`` files work, see :ref:`mozbuild-files`. michael@0: michael@0: Phase 3: Invokation of the Build Backend michael@0: ======================================== michael@0: michael@0: When most people think of the build system, they think of phase 3. This is michael@0: where we take all the code in the tree and produce Firefox or whatever michael@0: application you are creating. Phase 3 effectively takes whatever was michael@0: generated by phase 2 and runs it. Since the dawn of Mozilla, this has been michael@0: make consuming Makefiles. However, with the transition to moz.build files, michael@0: you may soon see non-Make build backends, such as Tup or Visual Studio. michael@0: michael@0: When building the tree, most of the time is spent in phase 3. This is when michael@0: header files are installed, C++ files are compiled, files are preprocessed, etc.