instancelib.feature_extraction package

Submodules

• instancelib.feature_extraction.base module
  • BaseVectorizer
    • BaseVectorizer.fit()
    • BaseVectorizer.fit_transform()
    • BaseVectorizer.fitted
    • BaseVectorizer.name
    • BaseVectorizer.transform()
  • SeparateContextVectorizer
    • SeparateContextVectorizer.fit()
    • SeparateContextVectorizer.fit_transform()
    • SeparateContextVectorizer.fitted
    • SeparateContextVectorizer.transform()
  • StackVectorizer
    • StackVectorizer.fit()
    • StackVectorizer.fit_transform()
    • StackVectorizer.fitted
    • StackVectorizer.transform()
    • StackVectorizer.vectorizers
• instancelib.feature_extraction.doc2vec module
  • Doc2VecVectorizer
    • Doc2VecVectorizer.fit()
    • Doc2VecVectorizer.fit_transform()
    • Doc2VecVectorizer.innermodel
    • Doc2VecVectorizer.load()
    • Doc2VecVectorizer.save()
    • Doc2VecVectorizer.tokenizer
    • Doc2VecVectorizer.transform()
  • get_line_docs()
  • split_tokenizer()
• instancelib.feature_extraction.textinstance module
  • TextInstanceVectorizer
    • TextInstanceVectorizer.fit()
    • TextInstanceVectorizer.fit_transform()
    • TextInstanceVectorizer.fitted
    • TextInstanceVectorizer.transform()
• instancelib.feature_extraction.textsklearn module
  • SklearnVectorizer
    • SklearnVectorizer.fit()
    • SklearnVectorizer.fit_transform()
    • SklearnVectorizer.innermodel
    • SklearnVectorizer.name
    • SklearnVectorizer.transform()

Module contents

class instancelib.feature_extraction.BaseVectorizer

Bases: ABC, Generic[DT]

This is the ABC specifies a generic vectorizer. Vectorizers transform raw data examples into feature vectors. Given a data type DT, it specifies the methods fit() that initializes or fits the vectorizer. The method transform() transforms the data into vector form.

abstract fit(x_data, **kwargs)

Fit the vectorizer according to the data in the given Sequence.

Parameters:

x_data (Sequence[DT]) – A Sequence of examples with type DT.

Returns:

A fitted vectorizer for data with type DT

Return type:

BaseVectorizer[DT]

Examples

Assume the creation of a vectorizer and a sequence of data examples in the variable data_list

>>> vectorizer = BaseVectorizer[DT]()
>>> vectorizer = vectorizer.fit(data_list)

Parameters:

kwargs (Any) –

abstract fit_transform(x_data, **kwargs)

Transform a list of data to a feature matrix. The transformation is based on the data contained in the parameter x_data. Subsequent transformations with transform() will be based on the fit of the data provided in this call.

Parameters:

x_data (Sequence[DT]) – A sequence of raw data examples with length n_examples

Returns:

A feature matrix with shape (n_examples, n_features)

Return type:

npt.NDArray[Any]

Examples

Assume the vectorizer is fitted

>>> x_mat = vectorizer.fit_transform(x_data)

Parameters:

kwargs (Any) –

property fitted: bool

Check if the vectorizer has been fitted

Returns:

True if the vectorizer has been fitted

Return type:

bool

property name: str

abstract transform(x_data, **kwargs)

Transform a list raw data points to a feature matrix according to the fitted vectorizer

Parameters:

x_data (Sequence[DT]) – A sequence of raw data examples with length n_examples

Returns:

A feature matrix with shape (n_examples, n_features)

Return type:

npt.NDArray[Any]

Examples

Assume the vectorizer is fitted

>>> x_mat = vectorizer.transform(x_data)

Parameters:

kwargs (Any) –

class instancelib.feature_extraction.SklearnVectorizer(vectorizer, storage_location=None, filename=None)

Bases: BaseVectorizer[str], SaveableInnerModel

Parameters:

• vectorizer (BaseEstimator) –

• storage_location (Optional[str]) –

• filename (Optional[str]) –

fit(x_data, **kwargs)

Fit the vectorizer according to the data in the given Sequence.

Parameters:

x_data (Sequence[DT]) – A Sequence of examples with type DT.

Returns:

A fitted vectorizer for data with type DT

Return type:

BaseVectorizer[DT]

Examples

Assume the creation of a vectorizer and a sequence of data examples in the variable data_list

>>> vectorizer = BaseVectorizer[DT]()
>>> vectorizer = vectorizer.fit(data_list)

Parameters:

kwargs (Any) –

fit_transform(x_data, **kwargs)

Transform a list of data to a feature matrix. The transformation is based on the data contained in the parameter x_data. Subsequent transformations with transform() will be based on the fit of the data provided in this call.

Parameters:

x_data (Sequence[DT]) – A sequence of raw data examples with length n_examples

Returns:

A feature matrix with shape (n_examples, n_features)

Return type:

npt.NDArray[Any]

Examples

Assume the vectorizer is fitted

>>> x_mat = vectorizer.fit_transform(x_data)

Parameters:

kwargs (Any) –

innermodel: BaseEstimator

property name: str

transform(x_data, **kwargs)

Transform a list raw data points to a feature matrix according to the fitted vectorizer

Parameters:

x_data (Sequence[DT]) – A sequence of raw data examples with length n_examples

Returns:

A feature matrix with shape (n_examples, n_features)

Return type:

npt.NDArray[Any]

Examples

Assume the vectorizer is fitted

>>> x_mat = vectorizer.transform(x_data)

Parameters:

kwargs (Any) –

class instancelib.feature_extraction.TextInstanceVectorizer(vectorizer)

Bases: BaseVectorizer[Instance[Any, str, ndarray, Any]]

Parameters:

vectorizer (BaseVectorizer[str]) –

fit(x_data, **kwargs)

Fit the vectorizer according to the data in the given Sequence.

Parameters:

x_data (Sequence[DT]) – A Sequence of examples with type DT.

Returns:

A fitted vectorizer for data with type DT

Return type:

BaseVectorizer[DT]

Examples

Assume the creation of a vectorizer and a sequence of data examples in the variable data_list

>>> vectorizer = BaseVectorizer[DT]()
>>> vectorizer = vectorizer.fit(data_list)

Parameters:

kwargs (Any) –

fit_transform(x_data, **kwargs)

Transform a list of data to a feature matrix. The transformation is based on the data contained in the parameter x_data. Subsequent transformations with transform() will be based on the fit of the data provided in this call.

Parameters:

x_data (Sequence[DT]) – A sequence of raw data examples with length n_examples

Returns:

A feature matrix with shape (n_examples, n_features)

Return type:

npt.NDArray[Any]

Examples

Assume the vectorizer is fitted

>>> x_mat = vectorizer.fit_transform(x_data)

Parameters:

kwargs (Any) –

property fitted: bool

Check if the vectorizer has been fitted

Returns:

True if the vectorizer has been fitted

Return type:

bool

transform(x_data, **kwargs)

Transform a list raw data points to a feature matrix according to the fitted vectorizer

Parameters:

x_data (Sequence[DT]) – A sequence of raw data examples with length n_examples

Returns:

A feature matrix with shape (n_examples, n_features)

Return type:

npt.NDArray[Any]

Examples

Assume the vectorizer is fitted

>>> x_mat = vectorizer.transform(x_data)

Parameters:

kwargs (Any) –