3.3.8. pci.api.inputsource module¶

A framework that provides an abstraction to iterate over various sources (including file system, MFILE). The static method :func`create_input_source<InputSourceFactoryBuilder.create_input_source> creates an input source agnostic iterator that can be used to simplify processing of files.

New in version 2018.

3.3.8.1. Examples¶

The following example prints each file in folder ‘/data’:

from pci.api.inputsource import InputSourceFactoryBuilder

# Create default input source factory. This factory comes with the following
# builders: FolderSourceBuilder, MFileSourceBuilder, IndexFileSourceBuilder, FileListSourceBuilder
factory = InputSourceFactoryBuilder().build_factory()
# The following will use FolderSourceBuilder to create an instance of
# FolderSource:
input_source = factory.create_input_source('/data')
# input_source can be iterated
for f in input_source:
    print("file: {}".format(f))

If the folder ‘/data’, contains the files ‘strip_1.pix’, ‘Mosaic.Pix’, ‘THUMBNAIL.TIF’, and ‘tmpimage.pix’, the output will be as follows:

file: /data/strip_1.pix
file: /data/Mosaic.Pix
file: /data/THUMBNAIL.TIF
file: /data/tmpimage.pix

3.3.8.2. Parsing source string¶

pci.api.inputsource.split_source_string(source, delimiters)¶

Return a list, which is the result of splitting the string source around a delimiter in delimiters. The string is split by using the first delimiter to yield a list with two or more parts.

Example:

from pci.api.inputsource import split_source_string
print(split_source_string('/mnt/data/GF1;/mnt/data/RD2;/mnt/data/WV2', ';'))
print(split_source_string('/mnt/data/GF1;/mnt/data/RD2;/mnt/data/WV2', ',;'))
print(split_source_string('/mnt/data/GF1;/mnt/data/RD2,/mnt/data/WV2', ',;'))
print(split_source_string('/mnt/data/GF1;/mnt/data/RD2;/mnt/data/WV2', None))

Output:

['/mnt/data/GF1', '/mnt/data/RD2', '/mnt/data/WV2']
['/mnt/data/GF1', '/mnt/data/RD2', '/mnt/data/WV2']
['/mnt/data/GF1;/mnt/data/RD2', '/mnt/data/WV2']
['/mnt/data/GF1;/mnt/data/RD2;/mnt/data/WV2']

pci.api.inputsource.split_source_strings(sources, delimiters)¶

Return a list, which is the result of splitting the list strings sources around a delimiter in delimiters. Each string is split using the first delimiter to yield a list with two or more parts.

Example:

from pci.api.inputsource import split_source_strings
print(split_source_strings(['/mnt/data/GF1,/mnt/data/RD2', '/mnt/data/WV2;/mnt/data/gdb'], ';,'))

Output:

['/mnt/data/GF2', '/mnt/data/RD2', '/mnt/data/WV2', '/mnt/data/gdb/']

3.3.8.3. Constants¶

class pci.api.inputsource.SearchType¶

Bases: object

Constants for different file system search types.

SEARCH_DIRECTORY_ONLY = 100¶: Search for directories only

SEARCH_FILE_ONLY = 200¶: Search for files only

SEARCH_BOTH_FILE_AND_DIRECTORY = 300¶: Search for directories and files

3.3.8.4. Input sources¶

class pci.api.inputsource.InputSourceCollection(source, **extra_param)¶

Bases: Iterable

Abstract class for input source container.

Called by subclass to initialize InputSourceCollection. source is a input source string

extra_param is an arbitrary keyword arguments. This class supports the following keywords:

mask - set file masks by calling InputSourceCollection.set_masks()

case_insensitive - set case insensitivity of the file masks

get_source()¶: Get the input original source resource locator string.

get_mask()¶: Get the current file masks.

set_masks(mask)¶: Replace current file masks with mask. The parameter mask can be a string or a list of strings or None. If None or an empty list, then the file mask is removed.

is_case_insensitive()¶: Return True if the mask is case insensitive, False otherwise

is_empty()¶: Return True if input source is empty, False otherwise

has_at_least(count)¶: Return True if this input source has more than count number of files, False otherwise

has_mask()¶: Return True, if this input source has at least one mask, False otherwise.

abstract is_selective_set()¶

Return True if the input source contains an internal list of filenames and therefore will not need to generate filenames using os.walk().

This implementation always throws a NotImplementedError.

set_recursive(recursive)¶: If recursive is True, the input source will search for files recursively. if supported by the input source.

class pci.api.inputsource.FolderSource(path, **extra_param)¶

Bases: InputSourceCollection

An input source collection for iterating over files and folders on a file system

path is valid path to a directory.

For example:

'/data'
r'D:\data'

extra_param is an arbitrary keyword arguments. This class supports the following keywords:

mask - set file masks by calling InputSourceCollection.set_masks()

case_insensitive - set case insensitivity of the file masks

recursive - if True, the path will searched recursively.

search_type - The type of serach tp performs, either search for files, directory or both, valid arguments are SearchType.SEARCH_FILE_ONLY, SearchType.SEARCH_DIRECTORY_ONLY, SearchType.SEARCH_BOTH_FILE_AND_DIRECTORY

is_selective_set()¶: Return True if the input source contains an internal list of filenames and therefore will not need to generate filenames using os.walk().

set_recursive(recursive)¶: If recursive is True, the FolderSource will search for files recursively.

get_mask()¶: Get the current file masks.

get_source()¶: Get the input original source resource locator string.

has_at_least(count)¶: Return True if this input source has more than count number of files, False otherwise

has_mask()¶: Return True, if this input source has at least one mask, False otherwise.

is_case_insensitive()¶: Return True if the mask is case insensitive, False otherwise

is_empty()¶: Return True if input source is empty, False otherwise

set_masks(mask)¶: Replace current file masks with mask. The parameter mask can be a string or a list of strings or None. If None or an empty list, then the file mask is removed.

class pci.api.inputsource.MFileSource(mfilename, **extra_param)¶

Bases: InputSourceCollection

An input source collection for iterating files in PCI MFile

mfilename is path to an existing mfile.

For example:

'orthos.mfile'

extra_param is an arbitrary keyword arguments. This class supports the following keywords:

mask - set file masks by calling InputSourceCollection.set_masks()

case_insensitive - set case insensitivity of the file masks

recursive - if True, the path will searched recursively.

is_selective_set()¶: Return True if the input source contains an internal list of filenames and therefore will not need to generate filenames using os.walk().

set_recursive(recursive)¶: If recursive is True, the FolderSource will search for files recursively.

get_mask()¶: Get the current file masks.

get_source()¶: Get the input original source resource locator string.

has_at_least(count)¶: Return True if this input source has more than count number of files, False otherwise

has_mask()¶: Return True, if this input source has at least one mask, False otherwise.

is_case_insensitive()¶: Return True if the mask is case insensitive, False otherwise

is_empty()¶: Return True if input source is empty, False otherwise

set_masks(mask)¶: Replace current file masks with mask. The parameter mask can be a string or a list of strings or None. If None or an empty list, then the file mask is removed.

class pci.api.inputsource.IndexFileSource(indexFilename, **extra_param)¶

Bases: InputSourceCollection

An input source collection for iterating files in PCIDSK index file

indexFilename is path to an existing index file.

For example:

'Image_Footprints.pix'

extra_param is an arbitrary keyword arguments. This class supports the following keywords:

mask - set file masks by calling InputSourceCollection.set_masks()

case_insensitive - set case insensitivity of the file masks

recursive - if True, the path will be searched recursively. filename_field_index - indicate in which column the filename is stored.

is_selective_set()¶: Return True if the input source contains an internal list of filenames and therefore will not need to generate filenames using os.walk().

set_recursive(recursive)¶: If recursive is True, the FolderSource will search for files recursively.

get_mask()¶: Get the current file masks.

get_source()¶: Get the input original source resource locator string.

has_at_least(count)¶: Return True if this input source has more than count number of files, False otherwise

has_mask()¶: Return True, if this input source has at least one mask, False otherwise.

is_case_insensitive()¶: Return True if the mask is case insensitive, False otherwise

is_empty()¶: Return True if input source is empty, False otherwise

set_masks(mask)¶: Replace current file masks with mask. The parameter mask can be a string or a list of strings or None. If None or an empty list, then the file mask is removed.

class pci.api.inputsource.FileListSource(files, **extra_param)¶

Bases: InputSourceCollection

An input source collection for iterating files from a collection

files is a path to a file or list of files.

For example:

'Image2.pix'
['/hdd/data/Ads_image1.tiff', '/collection/data/Image6.pix']
[r'D:\data\Ads_image1.tiff', r'C:\collection\data\Image6.pix']

extra_param is an arbitrary keyword arguments. This class supports the following keywords:

mask - set file masks by calling InputSourceCollection.set_masks()

case_insensitive - set case insensitivity of the file masks

is_selective_set()¶: Return True if the input source contains an internal list of filenames and therefore will not need to generate filenames using os.walk().

get_mask()¶: Get the current file masks.

get_source()¶: Get the input original source resource locator string.

has_at_least(count)¶: Return True if this input source has more than count number of files, False otherwise

has_mask()¶: Return True, if this input source has at least one mask, False otherwise.

is_case_insensitive()¶: Return True if the mask is case insensitive, False otherwise

is_empty()¶: Return True if input source is empty, False otherwise

set_masks(mask)¶: Replace current file masks with mask. The parameter mask can be a string or a list of strings or None. If None or an empty list, then the file mask is removed.

set_recursive(recursive)¶: If recursive is True, the input source will search for files recursively. if supported by the input source.

class pci.api.inputsource.AggregateSource(input_sources, **extra_param)¶

Bases: InputSourceCollection

This class provides a proxy for iterating over multiple input sourcecs

input_sources is a list of InputSourceCollection objects

extra_param is an arbitrary keyword arguments. This class supports the following keywords:

mask - set file masks by calling InputSourceCollection.set_masks()

case_insensitive - set case insensitivity of the file masks

is_selective_set()¶: Returns True if any of the aggregated input sources is a selective set.

has_mask()¶: Returns True, if any of the input sources has a mask.

get_mask()¶: Get the current file masks.

get_source()¶: Get the input original source resource locator string.

has_at_least(count)¶: Return True if this input source has more than count number of files, False otherwise

is_case_insensitive()¶: Return True if the mask is case insensitive, False otherwise

is_empty()¶: Return True if input source is empty, False otherwise

set_masks(mask)¶: Replace current file masks with mask. The parameter mask can be a string or a list of strings or None. If None or an empty list, then the file mask is removed.

set_recursive(recursive)¶: If recursive is True, the input source will search for files recursively. if supported by the input source.

3.3.8.5. Exceptions¶

class pci.api.inputsource.InputSourceError(message=None)¶

Bases: Exception

This exception is used by the builder to abort building of input source

args¶

with_traceback()¶: Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

3.3.8.6. Factories¶

class pci.api.inputsource.InputSourceFactory(split_delimiters=';')¶

Bases: object

Factory to create InputSourceCollection objects based on the source string. This factory iterates over all the registered builders and tries to build an InputSourceCollection object.

Initialize with a list of delimiters, split_delimiters for parsing source strings for multiple input sources. If None, then source string will not be split.

register_builder(builder)¶: Register builder of type InputSourceBuilder

create_input_source(source, **extra_param)¶: Creates an InputSourceCollection object from source string source and pass the variable keyword argument to the created object. If there are multiple sources in the source string, then an AggregateSource object is created. If none of the registered builder cannot build an InputSourceCollection object then None is returned.

create_input_source_list(source, **extra_param)¶: Creates a list of InputSourceCollection objects from source string source and passes the variable keyword argument to each created objects. If none of the registered builders can build an InputSourceCollection object then empty list is returned.

set_split_delimiter(split_delimiters)¶: Change the current delimiters to split_delimiters. None = don’t split.

get_split_delimiters()¶: Returns the list of delimiters

class pci.api.inputsource.InputSourceFactoryBuilder¶

Bases: object

A builder that builds an instance of the InputSourceFactory class.

build_factory(split_delimiters=';', **extra_param)¶

Build InputSourceFactory with pre-registered builders

InputSourceFactory object

3.3.8.7. Input source builders¶

class pci.api.inputsource.InputSourceBuilder¶

Bases: object

Abstract Input source builder class.

abstract build(source, **extra_param)¶

Build an InputSourceCollection object based on the source string, source and initializing it with the variable keyword parameter extra_param. On success InputSourceCollection object is returned, otherwise None is returned.

This implementation always throws a NotImplementedError.

class pci.api.inputsource.FolderSourceBuilder¶

Bases: InputSourceBuilder

A builder that builds a FolderSource object for a valid directory path.

build(source, **extra_param)¶: Build a new FolderSource object if the source string, source is a path to an existing directory and initialize it with the variable keyword parameter extra_param. On success FolderSource object is returned, otherwise None is returned.

class pci.api.inputsource.MFileSourceBuilder¶

Bases: InputSourceBuilder

A builder that builds a MFileSource object for a file with extension ‘.mfile’ or ‘.txt’.

build(source, **extra_param)¶: Build a new MFileSource object if the source string, source is a path to an existing file with extension ‘.mfile’ or ‘.txt’ and initialize it with the variable keyword parameter extra_param. On success MFileSource object is returned, otherwise None is returned.

class pci.api.inputsource.IndexFileSourceBuilder¶

Bases: InputSourceBuilder

A builder that builds a IndexFileSource object for a PCIDSK file with vector layer containing index of files.

build(source, **extra_param)¶: Build a new IndexFileSource object if the source string, source is a path to an existing PCIDSK file with vector layer that contains index of other files and initialize it with the variable keyword parameter extra_param. On success IndexFileSource object is returned, otherwise None is returned.

class pci.api.inputsource.FileListSourceBuilder¶

Bases: InputSourceBuilder

A builder that builds a FileListSource object for a file or list of files.

build(source, **extra_param)¶: Build a new FileListSource object if the source string, source is a path to an existing file or list of files and initialize it with the variable keyword parameter extra_param. On success FileListSource object is returned, otherwise None is returned.

3.3.8.8. Handle single file¶

class pci.api.inputsource.SingleFileUtil¶

Bases: object

Static methods dealing with a single file from an InputSourceCollection.

NoError = 0¶: No error, input source has exactly one file.

InvalidInputSource = 100¶: Invalid input source.

EmptyInputSource = 200¶: The input source is empty.

MoreThanOneFileInInputSource = 300¶: There are more than one file in the input source.

static check_for_error(input_source)¶: Check if the input source, input_source has only one input file. Returns SingleFileUtil.NoError, SingleFileUtil.InvalidInputSource, SingleFileUtil.EmptyInputSource, SingleFileUtil.MoreThanOneFileInInputSource

static get_file(input_source)¶: Return the first file from the InputSourceCollection object.

3.3.8. pci.api.inputsource module¶

3.3.8.1. Examples¶

3.3.8.2. Parsing source string¶

3.3.8.3. Constants¶

3.3.8.4. Input sources¶

3.3.8.5. Exceptions¶

3.3.8.6. Factories¶

3.3.8.7. Input source builders¶

3.3.8.8. Handle single file¶

Table of Contents

Previous topic

Next topic