olaf.pipeline package¶
Subpackages¶
- olaf.pipeline.data_preprocessing package
- olaf.pipeline.pipeline_component package
- Subpackages
- olaf.pipeline.pipeline_component.axiom_extraction package
- olaf.pipeline.pipeline_component.candidate_term_enrichment package
- olaf.pipeline.pipeline_component.concept_relation_extraction package
- Submodules
- olaf.pipeline.pipeline_component.concept_relation_extraction.agglomerative_clustering_concept_extraction module
- olaf.pipeline.pipeline_component.concept_relation_extraction.agglomerative_clustering_relation_extraction module
- olaf.pipeline.pipeline_component.concept_relation_extraction.candidate_terms_to_concepts module
- olaf.pipeline.pipeline_component.concept_relation_extraction.candidate_terms_to_relations module
- olaf.pipeline.pipeline_component.concept_relation_extraction.concept_cooc_metarelation_extraction module
- olaf.pipeline.pipeline_component.concept_relation_extraction.knowledge_based_concept_extraction module
- olaf.pipeline.pipeline_component.concept_relation_extraction.knowledge_based_relation_extraction module
- olaf.pipeline.pipeline_component.concept_relation_extraction.llm_based_concept_extraction module
- olaf.pipeline.pipeline_component.concept_relation_extraction.llm_based_relation_extraction module
- olaf.pipeline.pipeline_component.concept_relation_extraction.synonym_concept_extraction module
- olaf.pipeline.pipeline_component.concept_relation_extraction.synonym_relation_extraction module
- Module contents
- olaf.pipeline.pipeline_component.concept_relation_hierarchy package
- olaf.pipeline.pipeline_component.term_extraction package
- Submodules
- olaf.pipeline.pipeline_component.term_extraction.c_value_term_extraction module
- olaf.pipeline.pipeline_component.term_extraction.llm_term_extraction module
- olaf.pipeline.pipeline_component.term_extraction.manual_candidate_terms module
- olaf.pipeline.pipeline_component.term_extraction.pos_term_extraction module
- olaf.pipeline.pipeline_component.term_extraction.term_extraction_schema module
- olaf.pipeline.pipeline_component.term_extraction.tfidf_term_extraction module
- Module contents
- Submodules
- olaf.pipeline.pipeline_component.pipeline_component_schema module
- Module contents
- Subpackages
Submodules¶
olaf.pipeline.pipeline_schema module¶
- class olaf.pipeline.pipeline_schema.Pipeline(spacy_model: Language, pipeline_components: List[PipelineComponent] | None = None, preprocessing_components: List[DataPreprocessing] | None = None, corpus_loader: CorpusLoader | None = None, corpus: List[Doc] | None = None, seed_kr: KnowledgeRepresentation | None = None)[source]¶
Bases:
objectA Pipeline is the library main class. It orchestrates the pipeline starting from raw texts to build the final knowledge representation.
The corpus loader is responsible for the conversion for raw text to spacy document. We separate data preprocessing to explicitly enable pipelines without preprocessing.
Parameters¶
- spacy_model: spacy.language.Language
The spacy model used to represent text corpus.
- pipeline_components: List[PipelineComponent]
The ontology learning pipeline components that build the knowledge representation from the corpus.
- preprocessing_components: List[DataPreprocessing]
The pipeline components specific to preprocessing.
- corpus_loader: CorpusLoader
The component that loads the text corpus in the format used by the framework, i.e., a List[spacy.tokens.doc.Doc].
- corpus: List[spacy.tokens.doc.Doc]
The preprocessed corpus the knowledge representation is built from.
- kr: KnowledgeRepresentation
The knowledge extracted from the corpus.
- candidate_terms: Set[CandidateTerms]
The candidate terms extracted and processed to create concept and relations.
- add_pipeline_component(pipeline_component: PipelineComponent) None[source]¶
Add a component to the pipeline.
Parameters¶
- pipeline_componentPipelineComponent
The pipeline component to add.
- add_preprocessing_component(preprocessing_component: DataPreprocessing) None[source]¶
Add a preprocessing component to the pipeline.
Parameters¶
- preprocessing_componentDataPreprocessing
The preprocessing pipeline component to add.
- build() None[source]¶
Effectively build the pipeline, making the instance runnable. This method check each components and the constrained order.
- remove_pipeline_component(pipeline_component: PipelineComponent) None[source]¶
Remove a component from the pipeline.
Parameters¶
- pipeline_componentPipelineComponent
The pipeline component to remove.
- remove_preprocessing_component(preprocessing_component: DataPreprocessing) None[source]¶
Remove a preprocessing component from the pipeline.
Parameters¶
- preprocessing_componentDataPreprocessing
The preprocessing pipeline component to remove.