Packages

class TextMatcherInternalModel extends AnnotatorModel[TextMatcherInternalModel] with HasSimpleAnnotate[TextMatcherInternalModel] with TextMatcherInternalParams with CheckLicense

Instantiated model of the TextMatcherInternal. For usage and examples see the documentation of the main class.

Linear Supertypes
CheckLicense, TextMatcherInternalParams, HasSimpleAnnotate[TextMatcherInternalModel], AnnotatorModel[TextMatcherInternalModel], CanBeLazy, RawAnnotator[TextMatcherInternalModel], HasOutputAnnotationCol, HasInputAnnotationCols, HasOutputAnnotatorType, ParamsAndFeaturesWritable, HasFeatures, DefaultParamsWritable, MLWritable, Model[TextMatcherInternalModel], Transformer, PipelineStage, Logging, Params, Serializable, Serializable, Identifiable, AnyRef, Any
Ordering
  1. Grouped
  2. Alphabetic
  3. By Inheritance
Inherited
  1. TextMatcherInternalModel
  2. CheckLicense
  3. TextMatcherInternalParams
  4. HasSimpleAnnotate
  5. AnnotatorModel
  6. CanBeLazy
  7. RawAnnotator
  8. HasOutputAnnotationCol
  9. HasInputAnnotationCols
  10. HasOutputAnnotatorType
  11. ParamsAndFeaturesWritable
  12. HasFeatures
  13. DefaultParamsWritable
  14. MLWritable
  15. Model
  16. Transformer
  17. PipelineStage
  18. Logging
  19. Params
  20. Serializable
  21. Serializable
  22. Identifiable
  23. AnyRef
  24. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new TextMatcherInternalModel()

    internal constructor for writabale annotator

  2. new TextMatcherInternalModel(uid: String)

    uid

    internally renquired UID to make it writable

Type Members

  1. type AnnotationContent = Seq[Row]
    Attributes
    protected
    Definition Classes
    AnnotatorModel
  2. type AnnotatorType = String
    Definition Classes
    HasOutputAnnotatorType

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def $[T](param: Param[T]): T
    Attributes
    protected
    Definition Classes
    Params
  4. def $$[T](feature: StructFeature[T]): T
    Attributes
    protected
    Definition Classes
    HasFeatures
  5. def $$[K, V](feature: MapFeature[K, V]): Map[K, V]
    Attributes
    protected
    Definition Classes
    HasFeatures
  6. def $$[T](feature: SetFeature[T]): Set[T]
    Attributes
    protected
    Definition Classes
    HasFeatures
  7. def $$[T](feature: ArrayFeature[T]): Array[T]
    Attributes
    protected
    Definition Classes
    HasFeatures
  8. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  9. def _transform(dataset: Dataset[_], recursivePipeline: Option[PipelineModel]): DataFrame
    Attributes
    protected
    Definition Classes
    AnnotatorModel
  10. def afterAnnotate(dataset: DataFrame): DataFrame
    Attributes
    protected
    Definition Classes
    AnnotatorModel
  11. def annotate(annotations: Seq[Annotation]): Seq[Annotation]

    Searches entities and stores them in the annotation.

    Searches entities and stores them in the annotation. Defines annotator phrase matching depending on whether we are using SBD or not

    returns

    Extracted Entities

    Definition Classes
    TextMatcherInternalModel → HasSimpleAnnotate
  12. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  13. def beforeAnnotate(dataset: Dataset[_]): Dataset[_]
    Attributes
    protected
    Definition Classes
    AnnotatorModel
  14. val buildFromTokens: BooleanParam

    Whether the TextMatcherInternal should take the CHUNK from TOKEN or not

  15. def cartesianTokenVariants(tokens: Seq[Annotation], lemmaDictionary: Map[String, String]): Seq[Seq[String]]
    Attributes
    protected
    Definition Classes
    TextMatcherInternalParams
  16. val caseSensitive: BooleanParam

    Whether to match regardless of case (Default: true)

    Whether to match regardless of case (Default: true)

    Definition Classes
    TextMatcherInternalParams
  17. final def checkSchema(schema: StructType, inputAnnotatorType: String): Boolean
    Attributes
    protected
    Definition Classes
    HasInputAnnotationCols
  18. def checkValidEnvironment(spark: Option[SparkSession], scopes: Seq[String]): Unit
    Definition Classes
    CheckLicense
  19. def checkValidScope(scope: String): Unit
    Definition Classes
    CheckLicense
  20. def checkValidScopeAndEnvironment(scope: String, spark: Option[SparkSession], checkLp: Boolean): Unit
    Definition Classes
    CheckLicense
  21. def checkValidScopesAndEnvironment(scopes: Seq[String], spark: Option[SparkSession], checkLp: Boolean): Unit
    Definition Classes
    CheckLicense
  22. val cleanKeywords: StringArrayParam

    A parameter defining additional keywords to be removed during text processing, in addition to the standard stopwords.

    A parameter defining additional keywords to be removed during text processing, in addition to the standard stopwords.

    These keywords are appended to the default stopwords list and will be excluded from the text when cleanStopWords is enabled.

    By default, this parameter is an empty array, meaning no additional keywords are filtered unless specified.

    Definition Classes
    TextMatcherInternalParams
  23. val cleanStopWords: BooleanParam

    Parameter indicating whether to clean stop words during text processing.

    Parameter indicating whether to clean stop words during text processing. Defaults to true.

    Definition Classes
    TextMatcherInternalParams
  24. final def clear(param: Param[_]): TextMatcherInternalModel.this.type
    Definition Classes
    Params
  25. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  26. final def collapse(rs: List[(Int, Int)], sep: List[(Int, Int)] = Nil): List[(Int, Int)]
    Attributes
    protected
    Annotations
    @tailrec()
  27. def copy(extra: ParamMap): TextMatcherInternalModel
    Definition Classes
    RawAnnotator → Model → Transformer → PipelineStage → Params
  28. def copyValues[T <: Params](to: T, extra: ParamMap): T
    Attributes
    protected
    Definition Classes
    Params
  29. final def defaultCopy[T <: Params](extra: ParamMap): T
    Attributes
    protected
    Definition Classes
    Params
  30. val delimiter: Param[String]

    Value for the delimiter between Phrase, Entity in the entities file (Default: ,)

  31. def dfAnnotate: UserDefinedFunction
    Definition Classes
    HasSimpleAnnotate
  32. val enableLemmatizer: BooleanParam

    A Boolean parameter that controls whether lemmatization should be applied during text processing.

    A Boolean parameter that controls whether lemmatization should be applied during text processing.

    Lemmatization is the process of reducing words to their base or dictionary form (lemma). When this parameter is set to true: - The incoming tokens (words from the input text) are lemmatized. - The predefined entities (the terms you want to match against) are also lemmatized.

    This allows for more flexible and accurate matching. For example, words like "running", "ran", or "runs" will all be reduced to "run", and can match consistently even if the exact form in the text differs.

    Default value is false, meaning lemmatization is disabled unless explicitly turned on.

    Definition Classes
    TextMatcherInternalParams
  33. val enableStemmer: BooleanParam

    A Boolean parameter that controls whether stemming should be applied during text processing.

    A Boolean parameter that controls whether stemming should be applied during text processing.

    Stemming reduces words to their root forms (e.g., "running", "runs", and "runner" → "run"). This can help match different word forms more effectively in tasks such as keyword matching and entity recognition.

    When this parameter is set to true, stemming is applied in addition to the original form: - Input tokens are matched both in their original and stemmed forms. - Target entities can also be matched using their stemmed forms.

    This does not replace original matching — it complements it. Matching is performed using both the original and processed (stemmed) versions to improve recall and flexibility.

    Default value is false.

    Definition Classes
    TextMatcherInternalParams
  34. val entityValue: Param[String]

    Value for the entity metadata field

  35. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  36. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  37. val excludePunctuation: BooleanParam

    A parameter indicating whether punctuation marks should be removed during text processing.

    A parameter indicating whether punctuation marks should be removed during text processing.

    When set to true, most punctuation characters will be excluded from the processed text. This is typically used to clean text by removing non-word characters.

    Defaults to true, meaning punctuation is removed unless explicitly disabled. Some characters may be preserved if specifically handled by other parameters (e.g., safe keywords).

    Definition Classes
    TextMatcherInternalParams
  38. val excludeRegexPatterns: StringArrayParam

    A parameter specifying regular expression patterns used to exclude matching chunks during text processing.

    A parameter specifying regular expression patterns used to exclude matching chunks during text processing.

    Each string in this array is a regex pattern. If a detected chunk matches any of these patterns, it will be discarded and excluded from the final output.

    This is useful for removing unwanted matches based on pattern rules (e.g., specific codes, formats, or noise). By default, this parameter is empty, meaning no chunks are dropped based on regex.

    Definition Classes
    TextMatcherInternalParams
  39. def explainParam(param: Param[_]): String
    Definition Classes
    Params
  40. def explainParams(): String
    Definition Classes
    Params
  41. def extraValidate(structType: StructType): Boolean
    Attributes
    protected
    Definition Classes
    RawAnnotator
  42. def extraValidateMsg: String
    Attributes
    protected
    Definition Classes
    RawAnnotator
  43. final def extractParamMap(): ParamMap
    Definition Classes
    Params
  44. final def extractParamMap(extra: ParamMap): ParamMap
    Definition Classes
    Params
  45. val features: ArrayBuffer[Feature[_, _, _]]
    Definition Classes
    HasFeatures
  46. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  47. def get[T](feature: StructFeature[T]): Option[T]
    Attributes
    protected
    Definition Classes
    HasFeatures
  48. def get[K, V](feature: MapFeature[K, V]): Option[Map[K, V]]
    Attributes
    protected
    Definition Classes
    HasFeatures
  49. def get[T](feature: SetFeature[T]): Option[Set[T]]
    Attributes
    protected
    Definition Classes
    HasFeatures
  50. def get[T](feature: ArrayFeature[T]): Option[Array[T]]
    Attributes
    protected
    Definition Classes
    HasFeatures
  51. final def get[T](param: Param[T]): Option[T]
    Definition Classes
    Params
  52. def getBuildFromTokens: Boolean

    Getter for buildFromTokens param

  53. def getCaseSensitive: Boolean

    Setter for case sensitivity (Default: true)

  54. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  55. def getCleanKeywords: Array[String]

    Retrieves the list of keywords to be filtered out.

    Retrieves the list of keywords to be filtered out.

    returns

    an array of strings representing the keywords.

    Definition Classes
    TextMatcherInternalParams
  56. def getCleanStopWords: Boolean

    Retrieves the current state of the cleanStopWords parameter.

    Retrieves the current state of the cleanStopWords parameter.

    returns

    true if the cleanStopWords option is enabled, false otherwise.

    Definition Classes
    TextMatcherInternalParams
  57. final def getDefault[T](param: Param[T]): Option[T]
    Definition Classes
    Params
  58. def getDelimiter: String

    Get the Phrase, Entity delimiter in the entities file (Default: ,)

  59. def getEnableLemmatizer: Boolean

    Gets the current state of the lemmatizer enablement setting.

    Gets the current state of the lemmatizer enablement setting.

    returns

    true if the lemmatizer is enabled, false otherwise.

    Definition Classes
    TextMatcherInternalParams
  60. def getEnableStemmer: Boolean

    Retrieves the current value of the enableStemmer parameter.

    Retrieves the current value of the enableStemmer parameter.

    returns

    true if stemming is enabled, false otherwise

    Definition Classes
    TextMatcherInternalParams
  61. def getEntityValue: String

    Getter for Value for the entity metadata field

  62. def getExcludeRegexPattern: Array[String]

    Retrieves the list of regex patterns used to exclude specific text matches during processing.

    Retrieves the list of regex patterns used to exclude specific text matches during processing.

    returns

    an array of strings representing the regex patterns to be excluded.

    Definition Classes
    TextMatcherInternalParams
  63. def getInputCols: Array[String]
    Definition Classes
    HasInputAnnotationCols
  64. def getLazyAnnotator: Boolean
    Definition Classes
    CanBeLazy
  65. def getLicenseScopes: Seq[String]
    Attributes
    protected
  66. def getMergeOverlapping: Boolean

    Whether to merge overlapping matched chunks.

    Whether to merge overlapping matched chunks. Defaults false

  67. final def getOrDefault[T](param: Param[T]): T
    Definition Classes
    Params
  68. final def getOutputCol: String
    Definition Classes
    HasOutputAnnotationCol
  69. def getParam(paramName: String): Param[Any]
    Definition Classes
    Params
  70. def getReturnChunks: String

    Retrieves the current value of the returnChunks parameter.

    Retrieves the current value of the returnChunks parameter.

    returns

    A string representing the configured value for the returnChunks setting.

    Definition Classes
    TextMatcherInternalParams
  71. def getSafeKeywords: Array[String]

    Retrieves the list of keywords to be filtered out.

    Retrieves the list of keywords to be filtered out.

    returns

    an array of strings representing the keywords.

    Definition Classes
    TextMatcherInternalParams
  72. def getSkipMatcherAugmentation: Boolean

    Gets whether augmentation for matcher patterns is skipped.

    Gets whether augmentation for matcher patterns is skipped.

    returns

    true if augmentation for matcher patterns is skipped, false otherwise.

    Definition Classes
    TextMatcherInternalParams
  73. def getSkipSourceTextAugmentation: Boolean

    Gets whether augmentation for source text is skipped.

    Gets whether augmentation for source text is skipped.

    returns

    true if augmentation for source text is skipped, false otherwise.

    Definition Classes
    TextMatcherInternalParams
  74. def getStopWords: Array[String]

    Retrieves the list of stop words used within the text matching process.

    Retrieves the list of stop words used within the text matching process.

    returns

    an array of strings representing the stop words.

    Definition Classes
    TextMatcherInternalParams
  75. def getTokenVariants(token: Annotation, lemmaDictionary: Map[String, String]): Seq[String]
    Attributes
    protected
    Definition Classes
    TextMatcherInternalParams
  76. final def hasDefault[T](param: Param[T]): Boolean
    Definition Classes
    Params
  77. def hasParam(paramName: String): Boolean
    Definition Classes
    Params
  78. def hasParent: Boolean
    Definition Classes
    Model
  79. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  80. def initializeLogIfNecessary(isInterpreter: Boolean, silent: Boolean): Boolean
    Attributes
    protected
    Definition Classes
    Logging
  81. def initializeLogIfNecessary(isInterpreter: Boolean): Unit
    Attributes
    protected
    Definition Classes
    Logging
  82. val inputAnnotatorTypes: Array[AnnotatorType]

    input annotator type : DOCUMENT, TOKEN

    input annotator type : DOCUMENT, TOKEN

    Definition Classes
    TextMatcherInternalModel → HasInputAnnotationCols
  83. final val inputCols: StringArrayParam
    Attributes
    protected
    Definition Classes
    HasInputAnnotationCols
  84. final def isDefined(param: Param[_]): Boolean
    Definition Classes
    Params
  85. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  86. final def isSet(param: Param[_]): Boolean
    Definition Classes
    Params
  87. def isTraceEnabled(): Boolean
    Attributes
    protected
    Definition Classes
    Logging
  88. val lazyAnnotator: BooleanParam
    Definition Classes
    CanBeLazy
  89. val lemmaDict: MapFeature[String, String]

    lemmaDict

    lemmaDict

    Definition Classes
    TextMatcherInternalParams
  90. def log: Logger
    Attributes
    protected
    Definition Classes
    Logging
  91. def logDebug(msg: ⇒ String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  92. def logDebug(msg: ⇒ String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  93. def logError(msg: ⇒ String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  94. def logError(msg: ⇒ String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  95. def logInfo(msg: ⇒ String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  96. def logInfo(msg: ⇒ String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  97. def logName: String
    Attributes
    protected
    Definition Classes
    Logging
  98. def logTrace(msg: ⇒ String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  99. def logTrace(msg: ⇒ String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  100. def logWarning(msg: ⇒ String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  101. def logWarning(msg: ⇒ String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  102. def merge(rs: List[(Int, Int)]): List[(Int, Int)]
    Attributes
    protected
  103. val mergeOverlapping: BooleanParam

    whether to merge overlapping matched chunks.

    whether to merge overlapping matched chunks. Defaults false

  104. def msgHelper(schema: StructType): String
    Attributes
    protected
    Definition Classes
    HasInputAnnotationCols
  105. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  106. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  107. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  108. def onWrite(path: String, spark: SparkSession): Unit
    Attributes
    protected
    Definition Classes
    ParamsAndFeaturesWritable
  109. val optionalInputAnnotatorTypes: Array[String]
    Definition Classes
    HasInputAnnotationCols
  110. val outputAnnotatorType: AnnotatorType

    Output annotator type : CHUNK

    Output annotator type : CHUNK

    Definition Classes
    TextMatcherInternalModel → HasOutputAnnotatorType
  111. final val outputCol: Param[String]
    Attributes
    protected
    Definition Classes
    HasOutputAnnotationCol
  112. lazy val params: Array[Param[_]]
    Definition Classes
    Params
  113. var parent: Estimator[TextMatcherInternalModel]
    Definition Classes
    Model
  114. val returnChunks: Param[String]

    A string parameter that defines which version of the matched chunks should be returned: "original" or "matched".

    A string parameter that defines which version of the matched chunks should be returned: "original" or "matched".

    - If set to "original" (default): the returned chunks reflect the exact text spans as they appeared in the original input. This ensures that the begin and end character indices accurately map to the source text.

    - If set to "matched": the returned chunks are based on the processed form that triggered the match, such as a stemmed or lemmatized version of the phrase. This can be useful to see which normalized entity was matched, but the character indices (begin, end) may not align correctly with the original input text.

    Use "original" if accurate text positioning is important (e.g., for highlighting), and "matched" if you want to inspect the normalized form used for the match.

    Definition Classes
    TextMatcherInternalParams
  115. val safeKeywords: StringArrayParam

    A parameter representing an array of keywords that should be preserved during text cleaning, when stopword removal (cleanStopWords) is enabled.

    A parameter representing an array of keywords that should be preserved during text cleaning, when stopword removal (cleanStopWords) is enabled.

    When cleanStopWords is set to true, common stopwords are typically removed from the text. However, keywords specified in safeKeywords will be exempt from removal and retained in the processed text.

    By default, this parameter is an empty array, meaning no exceptions are made unless explicitly provided.

    Definition Classes
    TextMatcherInternalParams
  116. lazy val safeLemmaDict: Map[String, String]
    Definition Classes
    TextMatcherInternalParams
  117. def save(path: String): Unit
    Definition Classes
    MLWritable
    Annotations
    @Since( "1.6.0" ) @throws( ... )
  118. val searchTrieInternal: MapFeature[String, SearchTrieInternal]
  119. val searchTries: MapFeature[String, SearchTrie]

    searchTries for Searching words

  120. def set[T](feature: StructFeature[T], value: T): TextMatcherInternalModel.this.type
    Attributes
    protected
    Definition Classes
    HasFeatures
  121. def set[K, V](feature: MapFeature[K, V], value: Map[K, V]): TextMatcherInternalModel.this.type
    Attributes
    protected
    Definition Classes
    HasFeatures
  122. def set[T](feature: SetFeature[T], value: Set[T]): TextMatcherInternalModel.this.type
    Attributes
    protected
    Definition Classes
    HasFeatures
  123. def set[T](feature: ArrayFeature[T], value: Array[T]): TextMatcherInternalModel.this.type
    Attributes
    protected
    Definition Classes
    HasFeatures
  124. final def set(paramPair: ParamPair[_]): TextMatcherInternalModel.this.type
    Attributes
    protected
    Definition Classes
    Params
  125. final def set(param: String, value: Any): TextMatcherInternalModel.this.type
    Attributes
    protected
    Definition Classes
    Params
  126. final def set[T](param: Param[T], value: T): TextMatcherInternalModel.this.type
    Definition Classes
    Params
  127. def setBuildFromTokens(v: Boolean): TextMatcherInternalModel.this.type

    Setter for buildFromTokens param

  128. def setCaseSensitive(v: Boolean): TextMatcherInternalModel.this.type

    Setter for case sensitivity (Default: true)

  129. def setCleanKeywords(value: ArrayList[String]): TextMatcherInternalModel.this.type
    Definition Classes
    TextMatcherInternalParams
  130. def setCleanKeywords(values: Array[String]): TextMatcherInternalModel.this.type

    Sets the list of keywords to be cleaned during text processing.

    Sets the list of keywords to be cleaned during text processing.

    returns

    This instance with the updated configuration for cleaning keywords.

    Definition Classes
    TextMatcherInternalParams
  131. def setCleanStopWords(v: Boolean): TextMatcherInternalModel.this.type

    Sets whether to clean stop words during text processing.

    Sets whether to clean stop words during text processing.

    v

    Boolean value indicating whether to enable (true) or disable (false) the cleaning of stop words.

    returns

    This instance with the updated configuration for cleaning stop words.

    Definition Classes
    TextMatcherInternalParams
  132. def setDefault[T](feature: StructFeature[T], value: () ⇒ T): TextMatcherInternalModel.this.type
    Attributes
    protected
    Definition Classes
    HasFeatures
  133. def setDefault[K, V](feature: MapFeature[K, V], value: () ⇒ Map[K, V]): TextMatcherInternalModel.this.type
    Attributes
    protected
    Definition Classes
    HasFeatures
  134. def setDefault[T](feature: SetFeature[T], value: () ⇒ Set[T]): TextMatcherInternalModel.this.type
    Attributes
    protected
    Definition Classes
    HasFeatures
  135. def setDefault[T](feature: ArrayFeature[T], value: () ⇒ Array[T]): TextMatcherInternalModel.this.type
    Attributes
    protected
    Definition Classes
    HasFeatures
  136. final def setDefault(paramPairs: ParamPair[_]*): TextMatcherInternalModel.this.type
    Attributes
    protected
    Definition Classes
    Params
  137. final def setDefault[T](param: Param[T], value: T): TextMatcherInternalModel.this.type
    Attributes
    protected[org.apache.spark.ml]
    Definition Classes
    Params
  138. def setDelimiter(v: String): TextMatcherInternalModel.this.type

    Set the Phrase, Entity delimiter in the entities file (Default: ,)

  139. def setEnableLemmatizer(value: Boolean): TextMatcherInternalModel.this.type

    Enables or disables the lemmatizer for text matching.

    Enables or disables the lemmatizer for text matching.

    value

    If true, the lemmatizer will be enabled; if false, it will be disabled.

    returns

    This TextMatcherInternal instance with the updated lemmatizer setting.

    Definition Classes
    TextMatcherInternalParams
  140. def setEnableStemmer(value: Boolean): TextMatcherInternalModel.this.type

    Enables or disables the use of a stemmer for text processing.

    Enables or disables the use of a stemmer for text processing.

    value

    Boolean value indicating whether to enable (true) or disable (false) the stemmer.

    returns

    Instance of this class with updated configuration.

    Definition Classes
    TextMatcherInternalParams
  141. def setEntityValue(v: String): TextMatcherInternalModel.this.type

    Setter for Value for the entity metadata field

  142. def setExcludePunctuation(v: Boolean): TextMatcherInternalModel.this.type

    Sets the value of the excludePunctuation parameter used for text processing.

    Sets the value of the excludePunctuation parameter used for text processing.

    v

    A boolean value indicating whether to exclude punctuation.

    returns

    This instance with the updated excludePunctuation configuration.

    Definition Classes
    TextMatcherInternalParams
  143. def setExcludeRegexPatterns(v: Array[String]): TextMatcherInternalModel.this.type

    Sets the regular expression patterns for excluding specific elements during text processing.

    Sets the regular expression patterns for excluding specific elements during text processing.

    v

    Array of strings where each string represents a regular expression pattern to be used for excluding matching text elements.

    returns

    This instance with the updated configuration for exclude regex patterns.

    Definition Classes
    TextMatcherInternalParams
  144. final def setInputCols(value: String*): TextMatcherInternalModel.this.type
    Definition Classes
    HasInputAnnotationCols
  145. def setInputCols(value: Array[String]): TextMatcherInternalModel.this.type
    Definition Classes
    HasInputAnnotationCols
  146. def setLazyAnnotator(value: Boolean): TextMatcherInternalModel.this.type
    Definition Classes
    CanBeLazy
  147. def setLemmaDict(value: Map[String, String]): TextMatcherInternalModel.this.type

    Sets the internal dictionary used for lemmatization.

    Sets the internal dictionary used for lemmatization.

    value

    a map where keys are words and values are their corresponding lemmas.

    returns

    this

    Definition Classes
    TextMatcherInternalParams
  148. def setMergeOverlapping(v: Boolean): TextMatcherInternalModel.this.type

    Whether to merge overlapping matched chunks.

    Whether to merge overlapping matched chunks. Defaults false

  149. final def setOutputCol(value: String): TextMatcherInternalModel.this.type
    Definition Classes
    HasOutputAnnotationCol
  150. def setParent(parent: Estimator[TextMatcherInternalModel]): TextMatcherInternalModel
    Definition Classes
    Model
  151. def setReturnChunks(v: String): TextMatcherInternalModel.this.type

    Sets the value of the returnChunks parameter used for text processing.

    Sets the value of the returnChunks parameter used for text processing.

    v

    A string value that specifies the configuration for returning chunks.

    returns

    This instance with the updated returnChunks configuration.

    Definition Classes
    TextMatcherInternalParams
  152. def setSafeKeywords(value: ArrayList[String]): TextMatcherInternalModel.this.type
    Definition Classes
    TextMatcherInternalParams
  153. def setSafeKeywords(v: Array[String]): TextMatcherInternalModel.this.type

    Sets the list of safe keywords to be used in text processing.

    Sets the list of safe keywords to be used in text processing.

    v

    Array of strings representing the safe keywords.

    returns

    This instance with the updated configuration for safe keywords.

    Definition Classes
    TextMatcherInternalParams
  154. def setSearchTries(value: Map[String, SearchTrie]): TextMatcherInternalModel.this.type

    SearchTrie of Tokens

  155. def setSearchTriesInternal(value: Map[String, SearchTrieInternal]): TextMatcherInternalModel.this.type
  156. def setSkipMatcherAugmentation(value: Boolean): TextMatcherInternalModel.this.type

    Sets whether to skip augmentation for matcher patterns.

    Sets whether to skip augmentation for matcher patterns.

    value

    If true, matcher patterns won't be augmented with lemmatization, stemming, etc. If false, matcher patterns will be augmented if the corresponding features are enabled.

    returns

    This instance with the updated configuration.

    Definition Classes
    TextMatcherInternalParams
  157. def setSkipSourceTextAugmentation(value: Boolean): TextMatcherInternalModel.this.type

    Sets whether to skip augmentation for source text.

    Sets whether to skip augmentation for source text.

    value

    If true, source text won't be augmented with lemmatization, stemming, etc. If false, source text will be augmented if the corresponding features are enabled.

    returns

    This instance with the updated configuration.

    Definition Classes
    TextMatcherInternalParams
  158. def setStopWords(value: ArrayList[String]): TextMatcherInternalModel.this.type
    Definition Classes
    TextMatcherInternalParams
  159. def setStopWords(v: Array[String]): TextMatcherInternalModel.this.type

    Sets the list of stop words to be used in text processing.

    Sets the list of stop words to be used in text processing.

    v

    Array of strings representing the stop words.

    returns

    This instance with the updated stop words setting.

    Definition Classes
    TextMatcherInternalParams
  160. val skipMatcherAugmentation: BooleanParam

    A Boolean parameter that controls whether to skip augmentation (lemmatization, stemming, etc.) for matcher patterns.

    A Boolean parameter that controls whether to skip augmentation (lemmatization, stemming, etc.) for matcher patterns.

    When set to true, the matcher patterns won't be augmented with lemmatization, stemming, stopword removal, etc., even if those features are enabled. This applies only to entities/patterns being matched, not the source text.

    Default value is false, meaning matcher patterns will be augmented if the corresponding features are enabled.

    Definition Classes
    TextMatcherInternalParams
  161. val skipSourceTextAugmentation: BooleanParam

    A Boolean parameter that controls whether to skip augmentation (lemmatization, stemming, etc.) for the source text.

    A Boolean parameter that controls whether to skip augmentation (lemmatization, stemming, etc.) for the source text.

    When set to true, the source text won't be augmented with lemmatization, stemming, stopword removal, etc., even if those features are enabled. This applies only to the source text being analyzed, not the matcher patterns.

    Default value is false, meaning source text will be augmented if the corresponding features are enabled.

    Definition Classes
    TextMatcherInternalParams
  162. val stopWords: StringArrayParam

    A parameter representing the list of stop words to be filtered out during text processing.

    A parameter representing the list of stop words to be filtered out during text processing.

    By default, it is set to the English stop words provided by Spark ML.

    Definition Classes
    TextMatcherInternalParams
  163. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  164. def toString(): String
    Definition Classes
    Identifiable → AnyRef → Any
  165. final def transform(dataset: Dataset[_]): DataFrame
    Definition Classes
    AnnotatorModel → Transformer
  166. def transform(dataset: Dataset[_], paramMap: ParamMap): DataFrame
    Definition Classes
    Transformer
    Annotations
    @Since( "2.0.0" )
  167. def transform(dataset: Dataset[_], firstParamPair: ParamPair[_], otherParamPairs: ParamPair[_]*): DataFrame
    Definition Classes
    Transformer
    Annotations
    @Since( "2.0.0" ) @varargs()
  168. final def transformSchema(schema: StructType): StructType
    Definition Classes
    RawAnnotator → PipelineStage
  169. def transformSchema(schema: StructType, logging: Boolean): StructType
    Attributes
    protected
    Definition Classes
    PipelineStage
    Annotations
    @DeveloperApi()
  170. val uid: String
    Definition Classes
    TextMatcherInternalModel → Identifiable
  171. def validate(schema: StructType): Boolean
    Attributes
    protected
    Definition Classes
    RawAnnotator
  172. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  173. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  174. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  175. def wrapColumnMetadata(col: Column): Column
    Attributes
    protected
    Definition Classes
    RawAnnotator
  176. def write: MLWriter
    Definition Classes
    ParamsAndFeaturesWritable → DefaultParamsWritable → MLWritable

Inherited from CheckLicense

Inherited from TextMatcherInternalParams

Inherited from HasSimpleAnnotate[TextMatcherInternalModel]

Inherited from AnnotatorModel[TextMatcherInternalModel]

Inherited from CanBeLazy

Inherited from RawAnnotator[TextMatcherInternalModel]

Inherited from HasOutputAnnotationCol

Inherited from HasInputAnnotationCols

Inherited from HasOutputAnnotatorType

Inherited from ParamsAndFeaturesWritable

Inherited from HasFeatures

Inherited from DefaultParamsWritable

Inherited from MLWritable

Inherited from Model[TextMatcherInternalModel]

Inherited from Transformer

Inherited from PipelineStage

Inherited from Logging

Inherited from Params

Inherited from Serializable

Inherited from Serializable

Inherited from Identifiable

Inherited from AnyRef

Inherited from Any

Parameters

A list of (hyper-)parameter keys this annotator can take. Users can set and get the parameter values through setters and getters, respectively.

Annotator types

Required input and expected output annotator types

Members

Parameter setters

Parameter getters