17.2.5.1.1. cis.data_io.products package¶
17.2.5.1.1.1. The abstract AProduct module¶
-
class
cis.data_io.products.AProduct.
AProduct
¶ Bases:
object
Abstract class for the various possible data products. This just defines the interface which the subclasses must implement.
-
create_coords
(filenames)¶ Reads the coordinates from one or more files. Note that this method may have to make certain assumptions about the file in order to return a single coordinate set. The user should be warned through the logger if this is the case.
Parameters: filenames (list) – List of filenames to read coordinates from Returns: CommonData
object
-
create_data_object
(filenames, variable)¶ Create and return an
CommonData
object for a given variable from one or more files.Parameters: - filenames (list) – List of filenames of files to read
- variable (str) – Variable to read from the files
Returns: An
CommonData
object representing the specified variableRaises: - FileIOError – Unable to read a file
- InvalidVariableError – Variable not present in file
-
get_file_format
(filename)¶ Returns a file format hierarchy separated by slashes, of the form
TopLevelFormat/SubFormat/SubFormat/Version
. E.g.NetCDF/GASSP/1.0
,ASCII/ASCIIHyperpoint
orHDF4/CloudSat
. This is mainly used within the ceda_di indexing tool. If not set it will default to the products name.A filename of an example file can be provided to enable the determination of, for example, a dataset version number.
Parameters: filename (str) – Filename of file to be inspected Returns: File format, of the form [parent/]format/specific instance/version
, or the class nameReturn type: str Raises: FileFormatError if there is an error
-
get_file_signature
()¶ This method should return a list of regular expressions, which CIS uses to decide which data product to use for a given file. If more than one regular expression is provided in the list then the file can match any of the expressions. The first product with a signature that matches the filename will be used. The order in which the products are searched is determined by the priority property, highest value first; internal products generally have a priority of 10.
For example, this would match all files with a name containing the string ‘CODE’ and with the ‘nc’ extension.:
return [r'.*CODE*.nc']
Note
If the signature has matched the framework will call
AProduct.get_file_type_error()
, this gives the product a chance to open the file and check the contents.Returns: A list of regex to match the product’s file naming convention. Return type: list
-
get_file_type_error
(filename)¶ Check a single file to see if it is of the correct type, and if not return a list of errors. If the return is None then there are no errors and this is the correct data product to use for this file.
This method gives a mechanism for a data product to identify itself as the correct product when a specific enough file signature cannot be provided. For example GASSP is a type of NetCDF file and so filenames end with .nc but so do other NetCDF files, so the data product opens the file and looks for the GASSP version attribute, and if it doesn’t find it returns an error.
Parameters: filename (str) – The filename for the file Returns: List of errors, or None Return type: list or None
-
get_variable_names
(filenames, data_type=None)¶ Get a list of available variable names from the filenames list passed in. This general implementation can be overridden in specific products to include/exclude variables which may or may not be relevant. The data_type parameter can be used to specify extra information.
Parameters: - filenames (list) – List of string filenames of files to be read from
- data_type (str) – ‘SD’ or ‘VD’ to specify only return SD or VD variables from HDF files. This may take on other values in specific product implementations.
Returns: A set of variable names as strings
Return type: str
-
priority
= 10¶
-
valid_dimensions
= None¶
-
-
exception
cis.data_io.products.AProduct.
ProductPluginException
(message, original_exception)¶ Bases:
exceptions.Exception
Represents an error which has occurred inside of a Product plugin
-
original_exception
= None¶
-
-
cis.data_io.products.AProduct.
get_coordinates
(filenames, product=None)¶ Top level routine for calling the correct product’s
create_coords()
routine.Parameters: - filenames (list) – A list of filenames to read data from
- product (str) – The product to read data with - this should be a string which matches the name of one of the subclasses of AProduct
Returns: A
CoordList
object
-
cis.data_io.products.AProduct.
get_data
(filenames, variable, product=None)¶ Top level routine for calling the correct product’s
create_data_object()
routine.Parameters: - filenames (list) – A list of filenames to read data from
- variable (str) – The variable to create the
CommonData
object from - product (str) – The product to read data with - this should be a string which matches the name of one of the
subclasses of
AProduct
. If none is supplied it is guessed from the filename signature.
Returns: A
CommonData
variable
-
cis.data_io.products.AProduct.
get_file_format
(filenames, product=None)¶ Returns the files format of throws FileFormatError if there is an error in the format
Parameters: - filenames (list) – the filenames to read
- product (str) – the product to use, if not specified search
Returns: File format
Raises ClassNotFoundError: if there is no reader for this class
-
cis.data_io.products.AProduct.
get_product_full_name
(filenames, product=None)¶ Get the full name of the product which would read this file
Parameters: - filenames (list) – list of filenames to read
- product (str) – specified product to use
-
cis.data_io.products.AProduct.
get_variables
(filenames, product=None, data_type=None)¶ Top level routine for calling the correct product’s
get_variable_names()
routine.Parameters: - filenames (list) – A list of filenames to read the variables from
- product (str) – The product to read data with - this should be a string which matches the name of one of the subclasses of AProduct
Returns: A set of variable names as strings