michael@0: .. _healthreport_architecture: michael@0: michael@0: ============ michael@0: Architecture michael@0: ============ michael@0: michael@0: ``healthreporter.jsm`` contains the main interface for FHR, the michael@0: ``HealthReporter`` type. An instance of this is created by the michael@0: :ref:`data_reporting_service`. michael@0: michael@0: ``providers.jsm`` contains numerous ``Metrics.Provider`` and michael@0: ``Metrics.Measurement`` used for collecting application metrics. If you michael@0: are looking for the FHR probes, this is where they are. michael@0: michael@0: Storage michael@0: ======= michael@0: michael@0: Firefox Health Report stores data in 3 locations: michael@0: michael@0: * Metrics measurements and provider state is stored in a SQLite database michael@0: (via ``Metrics.Storage``). michael@0: * Service state (such as the IDs of documents uploaded) is stored in a michael@0: JSON file on disk (via OS.File). michael@0: * Lesser state and run-time options are stored in preferences. michael@0: michael@0: Preferences michael@0: =========== michael@0: michael@0: Preferences controlling behavior of Firefox Health Report live in the michael@0: ``datareporting.healthreport.*`` branch. michael@0: michael@0: Service and Data Control michael@0: ------------------------ michael@0: michael@0: The follow preferences control behavior of the service and data upload. michael@0: michael@0: service.enabled michael@0: Controls whether the entire health report service runs. The overall michael@0: service performs data collection, storing, and submission. michael@0: michael@0: This is the primary kill switch for Firefox Health Report michael@0: outside of the build system variable. i.e. if you are using an michael@0: official Firefox build and wish to disable FHR, this is what you michael@0: should set to false to prevent FHR from not only submitting but michael@0: also collecting data. michael@0: michael@0: uploadEnabled michael@0: Whether uploading of data is enabled. This is the preference the michael@0: checkbox in the preferences UI reflects. If this is michael@0: disabled, FHR still collects data - it just doesn't upload it. michael@0: michael@0: service.loadDelayMsec michael@0: How long (in milliseconds) after initial application start should FHR michael@0: wait before initializing. michael@0: michael@0: FHR may initialize sooner than this if the FHR service is requested. michael@0: This will happen if e.g. the user goes to ``about:healthreport``. michael@0: michael@0: service.loadDelayFirstRunMsec michael@0: How long (in milliseconds) FHR should wait to initialize on first michael@0: application run. michael@0: michael@0: FHR waits longer than normal to initialize on first application run michael@0: because first-time initialization can use a lot of I/O to initialize michael@0: the SQLite database and this I/O should not interfere with the michael@0: first-run user experience. michael@0: michael@0: documentServerURI michael@0: The URI of a Bagheera server that FHR should interface with for michael@0: submitting documents. michael@0: michael@0: You typically do not need to change this. michael@0: michael@0: documentServerNamespace michael@0: The namespace on the document server FHR should upload documents to. michael@0: michael@0: You typically do not need to change this. michael@0: michael@0: infoURL michael@0: The URL of a page containing more info about FHR, it's privacy michael@0: policy, etc. michael@0: michael@0: about.reportUrl michael@0: The URL to load in ``about:healthreport``. michael@0: michael@0: service.providerCategories michael@0: A comma-delimited list of category manager categories that contain michael@0: registered ``Metrics.Provider`` records. Read below for how provider michael@0: registration works. michael@0: michael@0: If the entire service is disabled, you lose data collection. This means michael@0: that **local** data analysis won't be available because there is no data michael@0: to analyze! Keep in mind that Firefox Health Report can be useful even michael@0: if it's not submitting data to remote servers! michael@0: michael@0: Logging michael@0: ------- michael@0: michael@0: The following preferences allow you to control the logging behavior of michael@0: Firefox Health Report. michael@0: michael@0: logging.consoleEnabled michael@0: Whether to write log messages to the web console. This is true by michael@0: default. michael@0: michael@0: logging.consoleLevel michael@0: The minimum log level FHR messages must have to be written to the michael@0: web console. By default, only FHR warnings or errors will be written michael@0: to the web console. During normal/expected operation, no messages of michael@0: this type should be produced. michael@0: michael@0: logging.dumpEnabled michael@0: Whether to write log messages via ``dump()``. If true, FHR will write michael@0: messages to stdout/stderr. michael@0: michael@0: This is typically only enabled when developing FHR. michael@0: michael@0: logging.dumpLevel michael@0: The minimum log level messages must have to be written via michael@0: ``dump()``. michael@0: michael@0: State michael@0: ----- michael@0: michael@0: currentDaySubmissionFailureCount michael@0: How many submission failures the client has encountered while michael@0: attempting to upload the most recent document. michael@0: michael@0: lastDataSubmissionFailureTime michael@0: The time of the last failed document upload. michael@0: michael@0: lastDataSubmissionRequestedTime michael@0: The time of the last document upload attempt. michael@0: michael@0: lastDataSubmissionSuccessfulTime michael@0: The time of the last successful document upload. michael@0: michael@0: nextDataSubmissionTime michael@0: The time the next data submission is scheduled for. FHR will not michael@0: attempt to upload a new document before this time. michael@0: michael@0: pendingDeleteRemoteData michael@0: Whether the client currently has a pending request to delete remote michael@0: data. If true, the client will attempt to delete all remote data michael@0: before an upload is performed. michael@0: michael@0: FHR stores various state in preferences. michael@0: michael@0: Registering Providers michael@0: ===================== michael@0: michael@0: Firefox Health Report providers are registered via the category manager. michael@0: See ``HealthReportComponents.manifest`` for providers defined in this michael@0: directory. michael@0: michael@0: Essentially, the category manager receives the name of a JS type and the michael@0: URI of a JSM to import that exports this symbol. At run-time, the michael@0: providers registered in the category manager are instantiated. michael@0: michael@0: Providers are registered via the category manager to make registration michael@0: simple and less prone to errors. Any XPCOM component can create a michael@0: category manager entry. Therefore, new data providers can be added michael@0: without having to touch core Firefox Health Report code. Additionally, michael@0: category manager registration means providers are more likely to be michael@0: registered on FHR's terms, when it wants. If providers were registered michael@0: in code at application run-time, there would be the risk of other michael@0: components prematurely instantiating FHR (causing a performance hit if michael@0: performed at an inopportune time) or semi-complicated code around michael@0: observers or listeners. Category manager entries are only 1 line per michael@0: provider and leave FHR in control: they are simple and safe. michael@0: michael@0: Document Generation and Lifecycle michael@0: ================================= michael@0: michael@0: FHR will attempt to submit a JSON document containing data every 24 wall michael@0: clock hours. michael@0: michael@0: At upload time, FHR will query the database for **all** information from michael@0: the last 180 days and assemble this data into a JSON document. We michael@0: attempt to upload this JSON document with a client-generated UUID to the michael@0: configured server. michael@0: michael@0: Before we attempt upload, the generated UUID is stored in the JSON state michael@0: file on local disk. At this point, the client assumes the document with michael@0: that UUID has been successfully stored on the server. michael@0: michael@0: If the client is aware of other document UUIDs that presumably exist on michael@0: the server, those UUIDs are sent with the upload request so the client michael@0: can request those UUIDs be deleted. This helps ensure that each client michael@0: only has 1 document/UUID on the server at any one time. michael@0: michael@0: Importance of Persisting UUIDs michael@0: ------------------------------ michael@0: michael@0: The choices of how, where, and when document UUIDs are stored and updated michael@0: are very important. One should not attempt to change things unless she michael@0: has a very detailed understanding of why things are the way they are. michael@0: michael@0: The client is purposefully very conservative about forgetting about michael@0: generated UUIDs. In other words, once a UUID is generated, the client michael@0: deliberately holds on to that UUID until it's very confident that UUID michael@0: is no longer stored on the server. The reason we do this is because michael@0: *orphaned* documents/UUIDs on the server can lead to faulty analysis, michael@0: such as over-reporting the number of Firefox installs that stop being michael@0: used. michael@0: michael@0: When uploading a new UUID, we update the state and save the state file michael@0: to disk *before* an upload attempt because if the upload succeeds but michael@0: the response never makes it back to the client, we want the client to michael@0: know about the uploaded UUID so it can delete it later to prevent an michael@0: orphan. michael@0: michael@0: We maintain a list of UUIDs locally (not simply the last UUID) because michael@0: multiple upload attempts could fail the same way as the previous michael@0: paragraph describes and we have no way of knowing which (if any) michael@0: actually succeeded. The safest approach is to assume every document michael@0: produced managed to get uploaded some how. michael@0: michael@0: We store the UUIDs on a file on disk and not anywhere else because we michael@0: want storage to be robust. We originally stored UUIDs in preferences, michael@0: which only flush to disk periodically. Writes to preferences were michael@0: apparently getting lost. We switched to writing directly to files to michael@0: eliminate this window.