Module preprocessor.numpy_input

NumPy data can represent virtually any kind of numerical data model, including images, timeseries data and other abstractions. Generally it is simpler to use the preprocessor.image module when working with images, but other data types or special occasions might call for this more abstract representation.

Typical usage:

preprocessor = (
    tb.NumpyInputPreprocessor.builder().
    .dtype("float32")
    .expand_target_dims()
    .target_column("target")
)

Classes

class NumpyInputPreprocessor (target_column: str, expand_target_dims: bool, dtype: Optional[str], transpose: List[int], target_is_path: bool, target_dtype: Optional[str])

Subclasses

  • preprocessor.numpy_input.NumpyNumpyPreprocessor
  • preprocessor.numpy_input.NumpyTorchPreprocessor

Static methods

def builder() -> NumpyInputPreprocessor

Instance variables

var target_column
var target_dtype
var target_is_path

Methods

def preprocess_array(self, ary) -> numpy.ndarray
def preprocess_target(self, target)
class NumpyInputPreprocessorBuilder

Preprocessor to generate numpy.ndarray representations of the data

Ancestors

Methods

def dtype(self, dtype: Optional[str]) -> NumpyInputPreprocessorBuilder

Cast an output numpy array to a given dtype. If unset, the Protocol will choose. Ignored for non numpy outputs.

Args

dtype
The dtype that a numpy output will be cast into.

Returns

NumpyInputPreprocessorBuilder
This class instance, useful for chaining.
def expand_target_dims(self, val: bool = True) -> NumpyInputPreprocessorBuilder

Expands the dimensions of the targets/labels, leaves the input data unchanged.

Args

val
True to expand, False to not. (default: True)

Returns

NumpyInputPreprocessorBuilder
This class instance, useful for chaining.
def target_column(self, column_name: str) -> NumpyInputPreprocessorBuilder

Sets which column from the asset's record data to use as a target.

Args

column_name
The name of the column to take as target information

Returns

NumpyInputPreprocessorBuilder
This class instance, useful for chaining.
def target_dtype(self, target_dtype: Optional[str] = None) -> NumpyInputPreprocessorBuilder

Convert target data type.

Args

target_dtype
string which correlates to numpy dtype

Returns

NumpyInputPreprocessorBuilder
This class instance, useful for chaining.
def target_is_path(self, target_is_path: Optional[bool] = False) -> NumpyInputPreprocessorBuilder

Target column is a path to a numpy file. If this is set to false, then target column will be used directly.

Args

target_is_path
set to true is target should be used as path to numpy file

Returns

NumpyInputPreprocessorBuilder
This class instance, useful for chaining.
def transpose(self, transpose: List[int] = None) -> NumpyInputPreprocessorBuilder

Apply transpose to numpy array. Can be used to change channel first dataset to channel last dataset depending on protocol being run.

Args

transpose
List of channels to tranpose. See np.transpose docs for

more info.

Returns

NumpyInputPreprocessorBuilder
This class instance, useful for chaining.

Inherited members