About Lineage Traceability

Lineage traceability is a record of the CATALYST Enterprise jobs and parameters supplied by users during processing of ARD related jobs for a given image dataset. The lineage data are stored in a .lineage.json file. The intent of a lineage file is to provide an analyst (or scientist) with sufficient information for them to duplicate the processes and achieve the same results. Lineage data can also be helpful in the Quality Control process as it offers an insight to the process (jobs that were run) and parameters that were used, alongside the results. Lineage data also record the specific version of the software, or algorithm, which can be useful in identifying scenes for reprocessing when algorithms are updated.

Lineage data is needed, especially for final products, so that users can produce identical results when re-processing their scenes; or for achieving equivalent results when processing a new batch of data. In other words, it can provide consistency of results as well as an algorithm version tracking capability. The .lineage.json file is also used in the creation of CARD4L-SR and ISO 19915-2 compliant metadata.

The information captured in the .lineage.json file are the job level (parent/child/grandchild etc) and all the parameters entered by the user, default values set in the settings configuration for parent and child jobs along with such things as the job title, description, operating platform/system, software version and start date and time.

Essentially, with lineage traceability, the user can track the process the data underwent during processing. The information in the lineage file can be used to repeat the process on the same or different dataset. If a scientist wants to reproduce an ARD product using new data for comparison with an already existing ARD dataset, they would examine the .lineage.json file and be able to trace the complete processing from end-to-end with all the processing parameters.

The key switch for enabling and disabling the lineage trace capability is found in the custom_settings.py file. The custom_settings.py file is located in the installation folder under the python sub-directory: ~\PGS\config\custom_settings.py.

In the custom_settings.py file, locate the key enableLineageCreation and set the value to True to enable and False to disable lineage tracing. The example below will enable lineage tracing.

	# Whether to create lineage information
	'enableLineageCreation': True,