kaishi.tabular.filters.duplicate_rows_after_concatenation

Class definition for filtering duplicate rows after concatenation.

Module Contents

class kaishi.tabular.filters.duplicate_rows_after_concatenation.FilterDuplicateRowsAfterConcatenation

Bases: kaishi.core.pipeline_component.PipelineComponent

Filter duplicate rows in the concatenated dataframe (dataset will be concatenated if it hasn’t been already).

__call__(self, dataset)

Perform filter on a given tabular dataset.

Parameters

dataset (kaishi.tabular.dataset.TabularDataset) – tabular dataset to perform operation on