1.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
1.2 +++ b/services/healthreport/docs/architecture.rst Wed Dec 31 06:09:35 2014 +0100
1.3 @@ -0,0 +1,223 @@
1.4 +.. _healthreport_architecture:
1.5 +
1.6 +============
1.7 +Architecture
1.8 +============
1.9 +
1.10 +``healthreporter.jsm`` contains the main interface for FHR, the
1.11 +``HealthReporter`` type. An instance of this is created by the
1.12 +:ref:`data_reporting_service`.
1.13 +
1.14 +``providers.jsm`` contains numerous ``Metrics.Provider`` and
1.15 +``Metrics.Measurement`` used for collecting application metrics. If you
1.16 +are looking for the FHR probes, this is where they are.
1.17 +
1.18 +Storage
1.19 +=======
1.20 +
1.21 +Firefox Health Report stores data in 3 locations:
1.22 +
1.23 +* Metrics measurements and provider state is stored in a SQLite database
1.24 + (via ``Metrics.Storage``).
1.25 +* Service state (such as the IDs of documents uploaded) is stored in a
1.26 + JSON file on disk (via OS.File).
1.27 +* Lesser state and run-time options are stored in preferences.
1.28 +
1.29 +Preferences
1.30 +===========
1.31 +
1.32 +Preferences controlling behavior of Firefox Health Report live in the
1.33 +``datareporting.healthreport.*`` branch.
1.34 +
1.35 +Service and Data Control
1.36 +------------------------
1.37 +
1.38 +The follow preferences control behavior of the service and data upload.
1.39 +
1.40 +service.enabled
1.41 + Controls whether the entire health report service runs. The overall
1.42 + service performs data collection, storing, and submission.
1.43 +
1.44 + This is the primary kill switch for Firefox Health Report
1.45 + outside of the build system variable. i.e. if you are using an
1.46 + official Firefox build and wish to disable FHR, this is what you
1.47 + should set to false to prevent FHR from not only submitting but
1.48 + also collecting data.
1.49 +
1.50 +uploadEnabled
1.51 + Whether uploading of data is enabled. This is the preference the
1.52 + checkbox in the preferences UI reflects. If this is
1.53 + disabled, FHR still collects data - it just doesn't upload it.
1.54 +
1.55 +service.loadDelayMsec
1.56 + How long (in milliseconds) after initial application start should FHR
1.57 + wait before initializing.
1.58 +
1.59 + FHR may initialize sooner than this if the FHR service is requested.
1.60 + This will happen if e.g. the user goes to ``about:healthreport``.
1.61 +
1.62 +service.loadDelayFirstRunMsec
1.63 + How long (in milliseconds) FHR should wait to initialize on first
1.64 + application run.
1.65 +
1.66 + FHR waits longer than normal to initialize on first application run
1.67 + because first-time initialization can use a lot of I/O to initialize
1.68 + the SQLite database and this I/O should not interfere with the
1.69 + first-run user experience.
1.70 +
1.71 +documentServerURI
1.72 + The URI of a Bagheera server that FHR should interface with for
1.73 + submitting documents.
1.74 +
1.75 + You typically do not need to change this.
1.76 +
1.77 +documentServerNamespace
1.78 + The namespace on the document server FHR should upload documents to.
1.79 +
1.80 + You typically do not need to change this.
1.81 +
1.82 +infoURL
1.83 + The URL of a page containing more info about FHR, it's privacy
1.84 + policy, etc.
1.85 +
1.86 +about.reportUrl
1.87 + The URL to load in ``about:healthreport``.
1.88 +
1.89 +service.providerCategories
1.90 + A comma-delimited list of category manager categories that contain
1.91 + registered ``Metrics.Provider`` records. Read below for how provider
1.92 + registration works.
1.93 +
1.94 +If the entire service is disabled, you lose data collection. This means
1.95 +that **local** data analysis won't be available because there is no data
1.96 +to analyze! Keep in mind that Firefox Health Report can be useful even
1.97 +if it's not submitting data to remote servers!
1.98 +
1.99 +Logging
1.100 +-------
1.101 +
1.102 +The following preferences allow you to control the logging behavior of
1.103 +Firefox Health Report.
1.104 +
1.105 +logging.consoleEnabled
1.106 + Whether to write log messages to the web console. This is true by
1.107 + default.
1.108 +
1.109 +logging.consoleLevel
1.110 + The minimum log level FHR messages must have to be written to the
1.111 + web console. By default, only FHR warnings or errors will be written
1.112 + to the web console. During normal/expected operation, no messages of
1.113 + this type should be produced.
1.114 +
1.115 +logging.dumpEnabled
1.116 + Whether to write log messages via ``dump()``. If true, FHR will write
1.117 + messages to stdout/stderr.
1.118 +
1.119 + This is typically only enabled when developing FHR.
1.120 +
1.121 +logging.dumpLevel
1.122 + The minimum log level messages must have to be written via
1.123 + ``dump()``.
1.124 +
1.125 +State
1.126 +-----
1.127 +
1.128 +currentDaySubmissionFailureCount
1.129 + How many submission failures the client has encountered while
1.130 + attempting to upload the most recent document.
1.131 +
1.132 +lastDataSubmissionFailureTime
1.133 + The time of the last failed document upload.
1.134 +
1.135 +lastDataSubmissionRequestedTime
1.136 + The time of the last document upload attempt.
1.137 +
1.138 +lastDataSubmissionSuccessfulTime
1.139 + The time of the last successful document upload.
1.140 +
1.141 +nextDataSubmissionTime
1.142 + The time the next data submission is scheduled for. FHR will not
1.143 + attempt to upload a new document before this time.
1.144 +
1.145 +pendingDeleteRemoteData
1.146 + Whether the client currently has a pending request to delete remote
1.147 + data. If true, the client will attempt to delete all remote data
1.148 + before an upload is performed.
1.149 +
1.150 +FHR stores various state in preferences.
1.151 +
1.152 +Registering Providers
1.153 +=====================
1.154 +
1.155 +Firefox Health Report providers are registered via the category manager.
1.156 +See ``HealthReportComponents.manifest`` for providers defined in this
1.157 +directory.
1.158 +
1.159 +Essentially, the category manager receives the name of a JS type and the
1.160 +URI of a JSM to import that exports this symbol. At run-time, the
1.161 +providers registered in the category manager are instantiated.
1.162 +
1.163 +Providers are registered via the category manager to make registration
1.164 +simple and less prone to errors. Any XPCOM component can create a
1.165 +category manager entry. Therefore, new data providers can be added
1.166 +without having to touch core Firefox Health Report code. Additionally,
1.167 +category manager registration means providers are more likely to be
1.168 +registered on FHR's terms, when it wants. If providers were registered
1.169 +in code at application run-time, there would be the risk of other
1.170 +components prematurely instantiating FHR (causing a performance hit if
1.171 +performed at an inopportune time) or semi-complicated code around
1.172 +observers or listeners. Category manager entries are only 1 line per
1.173 +provider and leave FHR in control: they are simple and safe.
1.174 +
1.175 +Document Generation and Lifecycle
1.176 +=================================
1.177 +
1.178 +FHR will attempt to submit a JSON document containing data every 24 wall
1.179 +clock hours.
1.180 +
1.181 +At upload time, FHR will query the database for **all** information from
1.182 +the last 180 days and assemble this data into a JSON document. We
1.183 +attempt to upload this JSON document with a client-generated UUID to the
1.184 +configured server.
1.185 +
1.186 +Before we attempt upload, the generated UUID is stored in the JSON state
1.187 +file on local disk. At this point, the client assumes the document with
1.188 +that UUID has been successfully stored on the server.
1.189 +
1.190 +If the client is aware of other document UUIDs that presumably exist on
1.191 +the server, those UUIDs are sent with the upload request so the client
1.192 +can request those UUIDs be deleted. This helps ensure that each client
1.193 +only has 1 document/UUID on the server at any one time.
1.194 +
1.195 +Importance of Persisting UUIDs
1.196 +------------------------------
1.197 +
1.198 +The choices of how, where, and when document UUIDs are stored and updated
1.199 +are very important. One should not attempt to change things unless she
1.200 +has a very detailed understanding of why things are the way they are.
1.201 +
1.202 +The client is purposefully very conservative about forgetting about
1.203 +generated UUIDs. In other words, once a UUID is generated, the client
1.204 +deliberately holds on to that UUID until it's very confident that UUID
1.205 +is no longer stored on the server. The reason we do this is because
1.206 +*orphaned* documents/UUIDs on the server can lead to faulty analysis,
1.207 +such as over-reporting the number of Firefox installs that stop being
1.208 +used.
1.209 +
1.210 +When uploading a new UUID, we update the state and save the state file
1.211 +to disk *before* an upload attempt because if the upload succeeds but
1.212 +the response never makes it back to the client, we want the client to
1.213 +know about the uploaded UUID so it can delete it later to prevent an
1.214 +orphan.
1.215 +
1.216 +We maintain a list of UUIDs locally (not simply the last UUID) because
1.217 +multiple upload attempts could fail the same way as the previous
1.218 +paragraph describes and we have no way of knowing which (if any)
1.219 +actually succeeded. The safest approach is to assume every document
1.220 +produced managed to get uploaded some how.
1.221 +
1.222 +We store the UUIDs on a file on disk and not anywhere else because we
1.223 +want storage to be robust. We originally stored UUIDs in preferences,
1.224 +which only flush to disk periodically. Writes to preferences were
1.225 +apparently getting lost. We switched to writing directly to files to
1.226 +eliminate this window.
