The Tor Browser: browser/experiments/docs/manifest.rst@b8a032363ba2 (annotated)

browser/experiments/docs/manifest.rst@b8a032363ba2 (annotated)

browser/experiments/docs/manifest.rst

Thu, 22 Jan 2015 13:21:57 +0100

author: Michael Schloh von Bennewitz <michael@schloh.com>
date: Thu, 22 Jan 2015 13:21:57 +0100
branch: TOR_BUG_9701
changeset 15: b8a032363ba2
permissions: -rw-r--r--

Incorporate requested changes from Mozilla in review:
https://bugzilla.mozilla.org/show_bug.cgi?id=1123480#c6

 .. _experiments_manifests:
 =====================
 Experiments Manifests
 =====================
 *Experiments Manifests* are documents that describe the set of active
 experiments a client may run.
 *Experiments Manifests* are fetched periodically by clients. When
 fetched, clients look at the experiments within the manifest and
 determine which experiments are applicable. If an experiment is
 applicable, the client may download and start the experiment.
 Manifest Format
 ===============
 Manifests are JSON documents where the main element is an object.
 The *schema* of the object is versioned and defined by the presence
 of a top-level ``version`` property, whose integer value is the
 schema version used by that manifest. Each version is documented
 in the sections below.
 Version 1
 ---------
 Version 1 is the original manifest format.
 The following properties may exist in the root object:
 experiments
    An array of objects describing candidate experiments. The format of
    these objects is documented below.
    An array is used to create an explicit priority of experiments.
    Experiments listed at the beginning of the array take priority over
    experiments that follow.
 Experiments Objects
 ^^^^^^^^^^^^^^^^^^^
 Each object in the ``experiments`` array may contain the following
 properties:
 id
    (required) String identifier of this experiment. The identifier should
    be treated as opaque by clients. It is used to uniquely identify an
    experiment for all of time.
 xpiURL
    (required) String URL of the XPI that implements this experiment.
    If the experiment is activated, the client will download and install this
    XPI.
 xpiHash
    (required) String hash of the XPI that implements this experiment.
    The value is composed of a hash identifier followed by a colon
    followed by the hash value. e.g.
    `sha1:f677428b9172e22e9911039aef03f3736e7f78a7`. `sha1` and `sha256`
    are the two supported hashing mechanisms. The hash value is the hex
    encoding of the binary hash.
    When the client downloads the XPI for the experiment, it should compare
    the hash of that XPI against this value. If the hashes don't match,
    the client should not install the XPI.
    Clients may also use this hash as a means of determining when an
    experiment's XPI has changed and should be refreshed.
 startTime
    Integer seconds since UNIX epoch that this experiment should
    start. Clients should not start an experiment if *now()* is less than
    this value.
 maxStartTime
    (optional) Integer seconds since UNIX epoch after which this experiment
    should no longer start.
    Some experiments may wish to impose hard deadlines after which no new
    clients should activate the experiment. This property may be used to
    facilitate that.
 endTime
    Integer seconds since UNIX epoch after which this experiment
    should no longer run. Clients should cease an experiment when the current
    time is beyond this value.
 maxActiveSeconds
    Integer seconds defining the max wall time this experiment should be
    active for.
    The client should deactivate the experiment this many seconds after
    initial activation.
    This value only involves wall time, not browser activity or session time.
 appName
    Array of application names this experiment should run on.
    An application name comes from ``nsIXULAppInfo.name``. It is a value
    like ``Firefox``, ``Fennec``, or `B2G`.
    The client should compare its application name against the members of
    this array. If a match is found, the experiment is applicable.
 minVersion
    (optional) String version number of the minimum application version this
    experiment should run on.
    A version number is something like ``27.0.0`` or ``28``.
    The client should compare its version number to this value. If the client's
    version is greater or equal to this version (using a version-aware comparison
    function), the experiment is applicable.
    If this is not specified, there is no lower bound to versions this
    experiment should run on.
 maxVersion
    (optional) String version number of the maximum application version this
    experiment should run on.
    This is similar to ``minVersion`` except it sets the upper bound for
    application versions.
    If the client's version is less than or equal to this version, the
    experiment is applicable.
    If this is not specified, there is no upper bound to versions this
    experiment should run on.
 version
    (optional) Array of application versions this experiment should run on.
    This is similar to ``minVersion`` and ``maxVersion`` except only a
    whitelisted set of specific versions are allowed.
    The client should compare its version to members of this array. If a match
    is found, the experiment is applicable.
 minBuildID
    (optional) String minimum Build ID this experiment should run on.
    Build IDs are values like ``201402261424``.
    The client should perform a string comparison of its Build ID against this
    value. If its value is greater than or equal to this value, the experiment
    is applicable.
 maxBuildID
    (optional) String maximum Build ID this experiment should run on.
    This is similar to ``minBuildID`` except it sets the upper bound
    for Build IDs.
    The client should perform a string comparison of its Build ID against
    this value. If its value is less than or equal to this value, the
    experiment is applicable.
 buildIDs
    (optional) Array of Build IDs this experiment should run on.
    This is similar to ``minBuildID`` and ``maxBuildID`` except only a
    whitelisted set of Build IDs are considered.
    The client should compare its Build ID to members of this array. If a
    match is found, the experiment is applicable.
 os
    (optional) Array of operating system identifiers this experiment should
    run on.
    Values for this array come from ``nsIXULRuntime.OS``.
    The client will compare its operating system identifier to members
    of this array. If a match is found, the experiment is applicable to the
    client.
 channel
    (optional) Array of release channel identifiers this experiment should run
    on.
    The client will compare its channel to members of this array. If a match
    is found, the experiment is applicable.
    If this property is not defined, the client should assume the experiment
    is to run on all channels.
 locale
    (optional) Array of locale identifiers this experiment should run on.
    A locale identifier is a string like ``en-US`` or ``zh-CN`` and is
    obtained by looking at
    ``nsIXULChromeRegistry.getSelectedLocale("global")``.
    The client should compare its locale identifier to members of this array.
    If a match is found, the experiment is applicable.
    If this property is not defined, the client should assume the experiment
    is to run on all locales.
 sample
    (optional) Decimal number indicating the sampling rate for this experiment.
    This will contain a value between ``0.0`` and ``1.0``. The client should
    generate a random decimal between ``0.0`` and ``1.0``. If the randomly
    generated number is less than or equal to the value of this field, the
    experiment is applicable.
 disabled
    (optional) Boolean value indicating whether an experiment is disabled.
    Normally, experiments are deactivated after a certain time has passed or
    after the experiment itself determines it no longer needs to run (perhaps
    it collected sufficient data already).
    This property serves as a backup mechanism to remotely disable an
    experiment before it was scheduled to be disabled. It can be used to
    kill experiments that are found to be doing wrong or bad things or that
    aren't useful.
    If this property is not defined or is false, the client should assume
    the experiment is active and a candidate for activation.
 frozen
    (optional) Boolean value indicating this experiment is frozen and no
    longer accepting new enrollments.
    If a client sees a true value in this field, it should not attempt to
    activate an experiment.
 jsfilter
     (optional) JavaScript code that will be evaluated to determine experiment
     applicability.
     This property contains the string representation of JavaScript code that
     will be evaluated in a sandboxed environment using JavaScript's
     ``eval()``.
     The string is expected to contain the definition of a JavaScript function
     ``filter(context)``. This function receives as its argument an object
     holding application state. See the section below for the definition of
     this object.
     The purpose of this property is to allow experiments to define complex
     rules and logic for evaluating experiment applicability in a manner
     that is privacy conscious and doesn't require the transmission of
     excessive data.
     The return value of this filter indicates whether the experiment is
     applicable. Functions should return true if the experiment is
     applicable.
     If an experiment is not applicable, they should throw an Error whose
     message contains the reason the experiment is not applicable. This
     message may be logged and sent to remote servers, so it should not
     contain private or otherwise sensitive data that wouldn't normally
     be submitted.
     If a falsey (or undefined) value is returned, the client should
     assume the experiment is not applicable.
     If this property is not defined, the client does not consider a custom
     JavaScript filter function when determining whether an experiment is
     applicable.
 JavaScript Filter Context Objects
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 The object passed to a ``jsfilter`` ``filter()`` function contains the
 following properties:
 healthReportSubmissionEnabled
    This property contains a boolean indicating whether Firefox Health
    Report has its data submission flag enabled (whether Firefox Health
    Report is sending data to remote servers).
 healthReportPayload
    This property contains the current Firefox Health Report payload.
    The payload format is documented at :ref:`healthreport_dataformat`.
 telemetryPayload
    This property contains the current Telemetry payload.
 The evaluation sandbox for the JavaScript filters may be destroyed
 immediately after ``filter()`` returns. This function should not assume
 async code will finish.
 Experiment Applicability and Client Behavior
 ============================================
 The point of an experiment manifest is to define which experiments are
 available and where and how to run them. This section explains those
 rules in more detail.
 Many of the properties in *Experiment Objects* are related to determining
 whether an experiment should run on a given client. This evaluation is
 performed client side.
 . Multiple conditions in an experiment
 ---------------------------------------
 If multiple conditions are defined for an experiment, the client should
 combine each condition with a logical *AND*: all conditions must be
 satisfied for an experiment to run. If one condition fails, the experiment
 is not applicable.
 . Active experiment disappears from manifest
 ---------------------------------------------
 If a specific experiment disappears from the manifest, the client should
 continue conducting an already-active experiment. Furthermore, the
 client should remember what the expiration events were for an experiment
 and honor them.
 The rationale here is that we want to prevent an accidental deletion
 or temporary failure on the server to inadvertantly deactivate
 supposed-to-be-active experiments. We also don't want premature deletion
 of an experiment from the manifest to result in indefinite activation
 periods.
 . Inactive experiment disappears from manifest
 -----------------------------------------------
 If an inactive but scheduled-to-be-active experiment disappears from the
 manifest, the client should not activate the experiment.
 If that experiment reappears in the manifest, the client should not
 treat that experiment any differently than any other new experiment. Put
 another way, the fact an inactive experiment disappears and then
 reappears should not be significant.
 The rationale here is that server operators should have complete
 control of an inactive experiment up to it's go-live date.
 . Re-evaluating applicability on manifest refresh
 --------------------------------------------------
 When an experiment manifest is refreshed or updated, the client should
 re-evaluate the applicability of each experiment therein.
 The rationale here is that the server may change the parameters of an
 experiment and want clients to pick those up.
 . Activating a previously non-applicable experiment
 ----------------------------------------------------
 If the conditions of an experiment change or the state of the client
 changes to allow an experiment to transition from previously
 non-applicable to applicable, the experiment should be activated.
 For example, if a client is running version 28 and the experiment
 initially requires version 29 or above, the client will not mark the
 experiment as applicable. But if the client upgrades to version 29 or if
 the manifest is updated to require 28 or above, the experiment will
 become applicable.
 . Deactivating a previously active experiment
 ----------------------------------------------
 If the conditions of an experiment change or the state of the client
 changes and an active experiment is no longer applicable, that
 experiment should be deactivated.
 . Calculation of sampling-based applicability
 ----------------------------------------------
 For calculating sampling-based applicability, the client will associate
 a random value between ``0.0`` and ``1.0`` for each observed experiment
 ID. This random value will be generated the first time sampling
 applicability is evaluated. This random value will be persisted and used
 in future applicability evaluations for this experiment.
 By saving and re-using the value, the client is able to reliably and
 consistently evaluate applicability, even if the sampling threshold
 in the manifest changes.
 Clients should retain the randomly-generated sampling value for
 experiments that no longer appear in a manifest for a period of at least
 days. The rationale is that if an experiment disappears and reappears
 from a manifest, the client will not have multiple opportunities to
 generate a random value that satisfies the sampling criteria.
 . Incompatible version numbers
 -------------------------------
 If a client receives a manifest with a version number that it doesn't
 recognize, it should ignore the manifest.
 . Usage of old manifests
 -------------------------
 If a client experiences an error fetching a manifest (server not
 available) or if the manifest is corrupt, not readable, or compatible,
 the client may use a previously-fetched (cached) manifest.
 . Updating XPIs
 -----------------
 If the URL or hash of an active experiment's XPI changes, the client
 should fetch the new XPI, uninstall the old XPI, and install the new
 XPI.
 Examples
 ========
 Here is an example manifest::
    {
      "version": 1,
      "experiments": [
        {
          "id": "da9d7f4f-f3f9-4f81-bacd-6f0626ffa360",
          "xpiURL": "https://experiments.mozilla.org/foo.xpi",
          "xpiHash": "sha1:cb1eb32b89d86d78b7326f416cf404548c5e0099",
          "startTime": 1393000000,
          "endTime": 1394000000,
          "appName": ["Firefox", "Fennec"],
          "minVersion": "28",
          "maxVersion": "30",
          "os": ["windows", "linux", "osx"],
          "jsfilter": "function filter(context) { return context.healthReportEnabled; }"
        }
      ]
    }

The Tor Browser / annotate

browser/experiments/docs/manifest.rst@b8a032363ba2 (annotated)

browser/experiments/docs/manifest.rst