[Elasticsearch] Synonym expand or contract 알아보기
Elastic/Elasticsearch 2016. 2. 26. 15:00synonym 은 아주 유용한 기능 입니다. 하지만 이 기능을 사용하기에 앞서 index time 시 장단점과 search time 시 장단점을 잘 이해하고 사용하시길 추천 드립니다.
관련 내용은 elastic.co에 "The Definitive Guide" 에서 가져왔습니다.
[원문 링크]
[원문 Snippet]
Simple Expansionedit
With simple expansion, any of the listed synonyms is expanded into all of the listed synonyms:
"jump,hop,leap"Expansion can be applied either at index time or at query time.
Each has advantages (⬆)︎ and disadvantages (⬇)︎.
When to use which comes down to performance versus flexibility.
| Index time | Query time | |
|---|---|---|
Index size | ⬇︎ Bigger index because all synonyms must be indexed. | ⬆︎ Normal. |
Relevance | ⬇︎ All synonyms will have the same IDF (see What Is Relevance?), meaning that more commonly used words will have the same weight as less commonly used words. | ⬆︎ The IDF for each synonym will be correct. |
Performance | ⬆︎ A query needs to find only the single term specified in the query string. | ⬇︎ A query for a single term is rewritten to look up all synonyms, |
Flexibility | ⬇︎ The synonym rules can’t be changed for existing documents. For the new rules to have effect, existing documents have to be reindexed. | ⬆︎ Synonym rules can be updated without reindexing documents. |
Simple Contractionedit
Simple contraction maps a group of synonyms on the left side to a single value on the right side:
"leap,hop => jump"It must be applied both at index time and at query time, to ensure that query terms are mapped to the same single value that exists in the index.
This approach has some advantages and some disadvantages compared to the simple expansion approach:
- Index size
- ⬆︎ The index size is normal, as only a single term is indexed.
- Relevance
- ⬇︎ The IDF for all terms is the same, so you can’t distinguish between more commonly used words and less commonly used words.
- Performance
- ⬆︎ A query needs to find only the single term that appears in the index.
- Flexibility
⬆︎ New synonyms can be added to the left side of the rule and applied at query time.
For instance, imagine that we wanted to add the wordboundto the rule specified previously.
The following rule would work for queries that containboundor for newly added documents that containbound:"leap,hop,bound => jump"But we could expand the effect to also take into account existing documents that contain
boundby writing the rule as follows:"leap,hop,bound => jump,bound"When you reindex your documents, you could revert to the previous rule to gain the performance benefit of querying only a single term.
Genre Expansionedit
Genre expansion is quite different from simple contraction or expansion.
Instead of treating all synonyms as equal, genre expansion widens the meaning of a term to be more generic. Take these rules, for example:
"cat => cat,pet",
"kitten => kitten,cat,pet",
"dog => dog,pet"
"puppy => puppy,dog,pet"By applying genre expansion at index time:
- A query for
kittenwould find just documents about kittens. - A query for
catwould find documents abouts kittens and cats. - A query for
petwould find documents about kittens, cats, puppies, dogs, or pets.
Alternatively, by applying genre expansion at query time,
a query for kitten would be expanded to return documents that mention kittens, cats, or pets specifically.
문서에서 보시면 index time 보다 search time 에 적용하는게 더 이점이 있는 것으로 나옵니다.
그런데 simple expansion 과 contraction 을 적절히 사용한다고 하면 검색 성능이나 품질을 풍성하게 할 수 있지 않을까 생각 합니다.