Fix Thai being skipped from language detection (#13989)
Thai does not separate words by spaces, so I figured out it should be in 'reliable characters regexp' that denotes languages that do the same. Related #13891.
This commit is contained in:
parent
f7bdfec5bb
commit
f5cea4d2ea
|
@ -4,7 +4,7 @@ class LanguageDetector
|
|||
include Singleton
|
||||
|
||||
WORDS_THRESHOLD = 4
|
||||
RELIABLE_CHARACTERS_RE = /[\p{Hebrew}\p{Arabic}\p{Syriac}\p{Thaana}\p{Nko}\p{Han}\p{Katakana}\p{Hiragana}\p{Hangul}]+/m
|
||||
RELIABLE_CHARACTERS_RE = /[\p{Hebrew}\p{Arabic}\p{Syriac}\p{Thaana}\p{Nko}\p{Han}\p{Katakana}\p{Hiragana}\p{Hangul}\p{Thai}]+/m
|
||||
|
||||
def initialize
|
||||
@identifier = CLD3::NNetLanguageIdentifier.new(1, 2048)
|
||||
|
|
Loading…
Reference in New Issue