instancelib.instances.memory module

class instancelib.instances.memory.AbstractMemoryProvider(instances)[source]

Bases: InstanceProvider[InstanceType, KT, DT, VT, RT], ABC, Generic[InstanceType, KT, DT, VT, RT]

Parameters:

instances (Iterable[TypeVar(InstanceType, bound= Instance[Any, Any, Any, Any])]) –

add_child(parent, child)[source]

Register a parent child relation between two instances

Parameters:
Return type:

None

bulk_get_all()[source]

Returns a list of all instances in this provider.

Returns:

A list of all instances in this provider

Return type:

List[Instance[KT, DT, VT, RT]]

Warning

When using this method on very large providers with lazily loaded instances, this may yield Out of Memory errors, as all the data will be loaded into RAM. Use with caution!

bulk_get_vectors(keys)[source]

Given a list of instance keys, return the vectors

Parameters:

keys (Sequence[KT]) – A list of vectors

Returns:

A tuple of two sequences, one with keys and one with vectors. The indices match, so the instance with keys[2] has as vector vectors[2]

Return type:

Tuple[Sequence[KT], Sequence[VT]]

Warning

Some underlying implementations do not preserve the ordering of the parameter keys. Therefore, always use the keys variable from the returned tuple for the correct matching.

children: Dict[TypeVar(KT), Set[TypeVar(KT)]]
clear()[source]
Return type:

None

abstract static construct(*args, **kwargs)[source]
Parameters:

  • args (Any) –

  • kwargs (Any) –

Return type:

TypeVar(InstanceType, bound= Instance[Any, Any, Any, Any])

dictionary: Dict[TypeVar(KT), TypeVar(InstanceType, bound= Instance[Any, Any, Any, Any])]
discard_children(parent)[source]

Discard the children that are registered to this parent

Parameters:

parent (Union[KT, Instance[KT, DT, VT, RT]]) – The parent from which you want to get the children from.

Return type:

None

property empty: bool

Determines if the provider does not contain instances

Returns:

True if the provider is empty

Return type:

bool

classmethod from_data(raw_data)[source]
Parameters:

raw_data (Sequence[TypeVar(DT)]) –

Return type:

Self

classmethod from_data_and_indices(indices, raw_data, vectors=None)[source]
Parameters:
Return type:

AbstractMemoryProvider[TypeVar(InstanceType, bound= Instance[Any, Any, Any, Any]), TypeVar(KT), TypeVar(DT), TypeVar(VT), TypeVar(RT)]

get_all()[source]

Get an iterator that iterates over all instances

Yields:

InstanceType – An iterator that iterates over all instances

Return type:

Iterator[TypeVar(InstanceType, bound= Instance[Any, Any, Any, Any])]

get_children(parent)[source]

Get the children that are registered to this parent

Parameters:

parent (Union[KT, Instance[KT, DT, VT, RT]]) – The parent from which you want to get the children from.

Returns:

A list containing the children

Return type:

Sequence[InstanceType]

get_children_keys(parent)[source]

Get the children that are registered to this parent

Parameters:

parent (Union[KT, Instance[KT, DT, VT, RT]]) – The parent from which you want to get the children from.

Returns:

A list containing the children

Return type:

Sequence[InstanceType]

get_parent(child)[source]

Get the parent of a child

Parameters:

child (Union[KT, Instance[KT, DT, VT, RT]]) – A child instance from which you want to get the children from.

Returns:

The parent of this child instance

Return type:

InstanceType

Raises:

KeyError – If there is no parent associated with this Instance

parents: Dict[TypeVar(KT), TypeVar(KT)]
classmethod shuffle(provider, mapping)[source]

Reorder the provider according to the given mapping

Parameters:

  • provider (InstanceProvider[InstanceType, _T, DT, VT, RT]) – The provider that needs to be reordered

  • mapping (Mapping[_T, KT]) – The mapping that maps old identifiers to new identifiers

Returns:

The shuffled

Return type:

Self

class instancelib.instances.memory.DataPoint(identifier, data, vector=None, representation=None)[source]

Bases: Instance[KT, DT, VT, RT], Generic[KT, DT, VT, RT]

Parameters:
property data: DT

Return the raw data of this instance

Returns:

The Raw Data

Return type:

DT

classmethod from_instance(instance)[source]
Parameters:

instance (Instance[TypeVar(KT), TypeVar(DT), TypeVar(VT), TypeVar(RT)]) –

property identifier: KT

Get the identifier of the instance

Returns:

The identifier key of the instance

Return type:

KT

property representation: RT

Return a representation for annotation

Returns:

A representation of the raw data

Return type:

RT

property vector: VT | None

Get the vector represenation of the raw data

Returns:

The Vector

Return type:

Optional[VT]

class instancelib.instances.memory.DataPointProvider(instances)[source]

Bases: AbstractMemoryProvider[DataPoint[Union[KT, UUID], DT, VT, RT], Union[KT, UUID], DT, VT, RT], Generic[KT, DT, VT, RT]

Parameters:

instances (Iterable[TypeVar(InstanceType, bound= Instance[Any, Any, Any, Any])]) –

children: Dict[TypeVar(KT), Set[TypeVar(KT)]]
static construct(*args, **kwargs)[source]
Parameters:

  • args (Any) –

  • kwargs (Any) –

create(*args, **kwargs)[source]

Create a new instance of type InstanceType. The created instance is subsequently added to the provider.

Note: The number of arguments and keyword arguments may differ in actual implementation, so there are no standard arguments.

Returns:

The new instance Type

Return type:

InstanceType

Parameters:

  • args (Any) –

  • kwargs (Any) –

dictionary: Dict[TypeVar(KT), TypeVar(InstanceType, bound= Instance[Any, Any, Any, Any])]
parents: Dict[TypeVar(KT), TypeVar(KT)]
class instancelib.instances.memory.MemoryBucketProvider(dataset, instances)[source]

Bases: AbstractBucketProvider[InstanceType, KT, DT, VT, RT], Generic[InstanceType, KT, DT, VT, RT]

Parameters:
dataset: InstanceProvider[TypeVar(InstanceType, bound= Instance[Any, Any, Any, Any]), TypeVar(KT), TypeVar(DT), TypeVar(VT), TypeVar(RT)]

The original dataset. All data will remain there

property empty: bool

Determines if the provider does not contain instances

Returns:

True if the provider is empty

Return type:

bool