services/docs/metrics.rst@97036ab72558 (annotated)
services/docs/metrics.rst
Tue, 06 Jan 2015 21:39:09 +0100
- author
- Michael Schloh von Bennewitz <michael@schloh.com>
- date
- Tue, 06 Jan 2015 21:39:09 +0100
- branch
- TOR_BUG_9701
- changeset 8
- 97036ab72558
- permissions
- -rw-r--r--
Conditionally force memory storage according to privacy.thirdparty.isolate;
This solves Tor bug #9701, complying with disk avoidance documented in
https://www.torproject.org/projects/torbrowser/design/#disk-avoidance.
| michael@0 | 1 | .. _services_metrics: |
| michael@0 | 2 | |
| michael@0 | 3 | ============================ |
| michael@0 | 4 | Metrics Collection Framework |
| michael@0 | 5 | ============================ |
| michael@0 | 6 | |
| michael@0 | 7 | The ``services/metrics`` directory contains a generic data metrics |
| michael@0 | 8 | collecting and persisting framework for Gecko applications. |
| michael@0 | 9 | |
| michael@0 | 10 | Overview |
| michael@0 | 11 | ======== |
| michael@0 | 12 | |
| michael@0 | 13 | The Metrics framework by itself doesn't do much: it simply provides a |
| michael@0 | 14 | generic mechanism for collecting and persisting data. It is up to users |
| michael@0 | 15 | of this framework to drive collection and do something with the obtained |
| michael@0 | 16 | data. A consumer of this framework is :ref:`firefox_health_report`. |
| michael@0 | 17 | |
| michael@0 | 18 | Relationship to Telemetry |
| michael@0 | 19 | ------------------------- |
| michael@0 | 20 | |
| michael@0 | 21 | Telemetry provides similar features to code in this directory. The two |
| michael@0 | 22 | may be unified in the future. |
| michael@0 | 23 | |
| michael@0 | 24 | Usage |
| michael@0 | 25 | ===== |
| michael@0 | 26 | |
| michael@0 | 27 | To use the code in this directory, import Metrics.jsm. e.g. |
| michael@0 | 28 | |
| michael@0 | 29 | Components.utils.import("resource://gre/modules/Metrics.jsm"); |
| michael@0 | 30 | |
| michael@0 | 31 | This exports a *Metrics* object which holds references to the main JS |
| michael@0 | 32 | types and functions provided by this feature. Read below for what those |
| michael@0 | 33 | types are. |
| michael@0 | 34 | |
| michael@0 | 35 | Metrics Types |
| michael@0 | 36 | ============= |
| michael@0 | 37 | |
| michael@0 | 38 | ``Metrics.jsm`` exports a number of types. They are documented in the |
| michael@0 | 39 | sections below. |
| michael@0 | 40 | |
| michael@0 | 41 | Metrics.Provider |
| michael@0 | 42 | ---------------- |
| michael@0 | 43 | |
| michael@0 | 44 | ``Metrics.Provider`` is an entity that collects and manages data. Providers |
| michael@0 | 45 | are typically domain-specific: if you need to collect a new type of data, |
| michael@0 | 46 | you create a ``Metrics.Provider`` type that does this. |
| michael@0 | 47 | |
| michael@0 | 48 | Metrics.Measurement |
| michael@0 | 49 | ------------------- |
| michael@0 | 50 | |
| michael@0 | 51 | A ``Metrics.Measurement`` represents a collection of related pieces/fields |
| michael@0 | 52 | of data. |
| michael@0 | 53 | |
| michael@0 | 54 | All data recorded by the metrics framework is modeled as |
| michael@0 | 55 | ``Metrics.Measurement`` instances. Instances of ``Metrics.Measurement`` |
| michael@0 | 56 | are essentially data structure descriptors. |
| michael@0 | 57 | |
| michael@0 | 58 | Each ``Metrics.Measurement`` consists of a name and version to identify |
| michael@0 | 59 | itself (and its data) as well as a list of *fields* that this measurement |
| michael@0 | 60 | holds. A *field* is effectively an entry in a data structure. It consists |
| michael@0 | 61 | of a name and strongly enumerated type. |
| michael@0 | 62 | |
| michael@0 | 63 | Metrics.Storage |
| michael@0 | 64 | --------------- |
| michael@0 | 65 | |
| michael@0 | 66 | This entity is responsible for persisting collected data and state. |
| michael@0 | 67 | |
| michael@0 | 68 | It currently uses SQLite to store data, but this detail is abstracted away |
| michael@0 | 69 | in order to facilitate swapping of storage backends. |
| michael@0 | 70 | |
| michael@0 | 71 | Metrics.ProviderManager |
| michael@0 | 72 | ----------------------- |
| michael@0 | 73 | |
| michael@0 | 74 | High-level entity coordinating activity among several ``Metrics.Provider`` |
| michael@0 | 75 | instances. |
| michael@0 | 76 | |
| michael@0 | 77 | Providers and Measurements |
| michael@0 | 78 | ========================== |
| michael@0 | 79 | |
| michael@0 | 80 | The most important types in this framework are ``Metrics.Provider`` and |
| michael@0 | 81 | ``Metrics.Measurement``, henceforth known as ``Provider`` and |
| michael@0 | 82 | ``Measurement``, respectively. As you will see, these two types go |
| michael@0 | 83 | hand in hand. |
| michael@0 | 84 | |
| michael@0 | 85 | A ``Provider`` is an entity that *provides* data about a specific subsystem |
| michael@0 | 86 | or feature. They do this by recording data to specific ``Measurement`` |
| michael@0 | 87 | types. Both ``Provider`` and ``Measurement`` are abstract base types. |
| michael@0 | 88 | |
| michael@0 | 89 | A ``Measurement`` implementation defines a name and version. More |
| michael@0 | 90 | importantly, it also defines its storage requirements and how |
| michael@0 | 91 | previously-stored values are serialized. |
| michael@0 | 92 | |
| michael@0 | 93 | Storage allocation is performed by communicating with the SQLite |
| michael@0 | 94 | backend. There is a startup function that tells SQLite what fields the |
| michael@0 | 95 | measurement is recording. The storage backend then registers these in |
| michael@0 | 96 | the database. Internally, this is creating a new primary key for |
| michael@0 | 97 | individual fields so later storage operations can directly reference |
| michael@0 | 98 | these primary keys in order to retrieve data without having to perform |
| michael@0 | 99 | complicated joins. |
| michael@0 | 100 | |
| michael@0 | 101 | A ``Provider`` can be thought of as a collection of ``Measurement`` |
| michael@0 | 102 | implementations. e.g. an Addons provider may consist of a measurement |
| michael@0 | 103 | for all *current* add-ons as well as a separate measurement for |
| michael@0 | 104 | historical counts of add-ons. A provider's primary role is to take |
| michael@0 | 105 | metrics data and write it to various measurements. This effectively |
| michael@0 | 106 | persists the data to SQLite. |
| michael@0 | 107 | |
| michael@0 | 108 | Data is emitted from providers in either a push or pull based mechanism. |
| michael@0 | 109 | In push-based scenarios, the provider likely subscribes to external |
| michael@0 | 110 | events (e.g. observer notifications). An event of interest can occur at |
| michael@0 | 111 | any time. When it does, the provider immediately writes the event of |
| michael@0 | 112 | interest to storage or buffers it for eventual writing. In pull-based |
| michael@0 | 113 | scenarios, the provider is periodically queried and asked to populate |
| michael@0 | 114 | data. |
| michael@0 | 115 | |
| michael@0 | 116 | SQLite Storage |
| michael@0 | 117 | ============== |
| michael@0 | 118 | |
| michael@0 | 119 | ``Metrics.Storage`` provides an interface for persisting metrics data to a |
| michael@0 | 120 | SQLite database. |
| michael@0 | 121 | |
| michael@0 | 122 | The storage API organizes values by fields. A field is a named member of |
| michael@0 | 123 | a ``Measurement`` that has specific type and retention characteristics. |
| michael@0 | 124 | Some example field types include: |
| michael@0 | 125 | |
| michael@0 | 126 | * Last text value |
| michael@0 | 127 | * Last numeric value for a given day |
| michael@0 | 128 | * Discrete text values for a given day |
| michael@0 | 129 | |
| michael@0 | 130 | See ``storage.jsm`` for more. |
