'elastic'에 해당되는 글 17건

  1. 2017.06.08 [Elasticsearch] 각 노드별 meta 정보 저장 관련.
  2. 2017.03.09 [Elasticsearch] TransportClient on 5.x
  3. 2017.01.24 [검색추천] Apache mahout + Elastic Stack 을 이용한 기본 추천
  4. 2017.01.12 [Logstash] 간헐적으로 logstash 가 죽습니다.
  5. 2017.01.02 [Elasticsearch] Range Query From, To 포함 여부.
  6. 2016.12.06 [Elasticsearch] _cat nodes/indices help
  7. 2016.11.24 [Elasticsearch] Lucene Arirang Analyzer Plugin for Elasticsearch 5.0.1
  8. 2016.11.21 [Kibana] Date Histogram Advanced DSL
  9. 2016.11.02 [Elasticsearch] Elasticsearch 5.0 GA 살펴보기 - 2
  10. 2016.11.02 [Elasticsearch] Elasticsearch 5.0 GA 살펴보기 - 1

[Elasticsearch] 각 노드별 meta 정보 저장 관련.

Elastic/Elasticsearch 2017.06.08 10:12

master, data 노드에는 기본적으로 모든 index 들의 settings, mappings 정보를 가지고 있습니다.

search  노드만 index 에 대한 meta 정보를 가지고 있지 않습니다.

어떻게 보면 당연한 건데 확인 하지 않으면 실수 할 수 있는 내용이라 그냥 기록해 봅니다.


master 노드는 모드 node 와 index 에 대한 관리를 하기 때문에 당연히 정보를 가지고 있어야 합니다.

data 노드는 물리적인 index, shard 를 저장하고 있기 때문에 역시 정보를 가지고 있을 수 밖에 없습니다.

search 노드는 coordinator 역할과 질의 결과에 대한 merge 등의 역할을 하고 있어 물리적으로 정보를 가지고 있지는 않지만 필요 시 master node 로 정보를 요청 할 수는 있습니다.

저작자 표시 비영리 변경 금지
신고
크리에이티브 커먼즈 라이선스
Creative Commons License
Trackback 0 : Comment 0

[Elasticsearch] TransportClient on 5.x

Elastic/Elasticsearch 2017.03.09 11:51

elasticsearch 2.4 에서 사용하던 java api 중 TransportClinet 사용 방법이 바뀌어서 작성 합니다.

변경된 내용에 대해서는 elasticsearch 공식 홈페이지에 자세히 나와 있습니다.


[참고문서]

https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/_maven_repository.html

https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/transport-client.html


[코드 변경]


2.x)

settings = settingsBuilder()

  .put("cluster.name", cluster)

  .put("client.transport.sniff", true)

  .put("network.tcp.blocking", false)           // tcp non-blocking mode

  .put("client.transport.ping_timeout", "10s")

  .build();


5.x)

settings = builder()

  .put("cluster.name", cluster)

  .put("client.transport.sniff", true)

  .put("network.tcp.blocking", false)           // tcp non-blocking mode

  .put("client.transport.ping_timeout", "10s")

  .build();


2.x)

TransportClient client = TransportClient.builder().settings(settings).build();


5.x)

TransportClient client = new PreBuiltTransportClient(settings);


여기서 주의 하실 점은 참고문서에 있지만  transport 가 분리 되었기 때문에 별도로 dependency 구성을 해주셔야 합니다.


Maven Dependency 추가)

<dependency>

  <groupId>org.elasticsearch.client</groupId>

  <artifactId>transport</artifactId>

  <version>${elasticsearch.version}</version>

</dependency>


별 내용은 아니지만 혹시라도 삽질 하시는 분들이 계실 수 있어 작성해 봤습니다.

저작자 표시 비영리 변경 금지
신고
크리에이티브 커먼즈 라이선스
Creative Commons License
Trackback 0 : Comment 0

[검색추천] Apache mahout + Elastic Stack 을 이용한 기본 추천

Elastic/Elasticsearch 2017.01.24 11:47

Elastic Stack 과 Apache mahout 을 이용한 추천 데이터 생성을 다뤄 볼까 합니다.

기본적으로는 Elastic Stack 만 가지고도 cohort 분석을 통해 추천 데이터 마트 구성이 가능 한데요.

추천 데이터에 대한 품질을 좀 더 좋게 하기 위해 Apache mahout 을 활용해 보도록 하겠습니다.


여기서 다루는 내용은 누구나 쉽게 접근 할 수 있도록 Hello World! 수준만 기술 합니다.


[Elastic Stack]

https://www.elastic.co/products


[Apache mahout]

https://mahout.apache.org/


위 두 솔루션은 모두 오픈소스 이며 예제 코드가 해당 소스에 잘 만들어져 있어 누구나 쉽게 활용이 가능합니다.


Step 1)

Elasticsearch + Logstash + Kibana 를 이용해 로그를 수집하고 추천 할 raw data 를 생성 합니다.


User item click log -> Logstash collect -> Elasticsearch store -> Kibana visualize -> CSV download


여기서 수집한 데이터 중 추출 데이터는 user id + item id + click count 입니다.

아래는 Kibana QueryDSL 예제 입니다.

{

  "size": 0,

  "query": {

    "filtered": {

      "query": {

        "query_string": {

          "query": "cp:CLK AND id:[0 TO *]",

          "analyze_wildcard": true

        }

      },

      "filter": {

        "bool": {

          "must": [

            {

              "range": {

                "time": {

                  "gte": 1485010800000,

                  "lte": 1485097199999,

                  "format": "epoch_millis"

                }

              }

            }

          ],

          "must_not": []

        }

      }

    }

  },

  "aggs": {

    "2": {

      "terms": {

        "field": "user_id",

        "size": 30000,

        "order": {

          "_count": "desc"

        }

      },

      "aggs": {

        "3": {

          "terms": {

            "field": "item_id",

            "size": 10,

            "order": {

              "_count": "desc"

            }

          }

        }

      }

    }

  }

}


Step 2)

Apache mahout 에서 사용할 recommender 는 UserBasedRecommender 입니다.

샘플 코드에도 나와 있지만 dataset.csv 파일은 아래와 같은 형식 입니다.

- Creating a User-Based Recommender in 5 minutes


1,10,1.0
1,11,2.0
1,12,5.0
1,13,5.0

형식) userId,itemId,ratingValue


Step1 에서 위와 같은 형식을 맞추기 위해 user_id, item_id, click_count 를 생성 하였습니다.

이 데이터를 기반으로 UserBasedRecommender 를 돌려 보도록 하겠습니다.


Step 3)

아래 보시면 샘플 코드가 잘 나와 있습니다.

https://github.com/apache/mahout/tree/master/examples/src/main/java/org/apache/mahout


Main class 하나 만드셔서 Step2 에 나와 있는 코드로 돌려 보시면 됩니다.

저는 UserBasedRecommender 를 implements 해서 별도로 구현했습니다.

이건 누구나 쉽게 하실 수 있는 부분이기 때문에 examples 에 나와 있는 BookCrossingRecommender 클래스등을 참고 하시면 됩니다.


UserBasedRecommenderRunner runner = new UserBasedRecommenderRunner();

Recommender recommender = runner.buildRecommender();


// 710039번 유저에 대한 추천 아이템 3개

List<RecommendedItem> recommendations = recommender.recommend(710039, 3);


for (RecommendedItem recommendation : recommendations) {

    LOG.debug("추천 아이템 : {}", recommendation);

}


[실행 로그]

11:39:31.527 [main] INFO  o.a.m.c.t.i.model.file.FileDataModel - Creating FileDataModel for file /git/prototype/data/user-to-item.csv

11:39:31.626 [main] INFO  o.a.m.c.t.i.model.file.FileDataModel - Reading file info...

11:39:31.765 [main] INFO  o.a.m.c.t.i.model.file.FileDataModel - Read lines: 63675

11:39:31.896 [main] INFO  o.a.m.c.t.i.model.GenericDataModel - Processed 10000 users

11:39:31.911 [main] INFO  o.a.m.c.t.i.model.GenericDataModel - Processed 19124 users

11:39:31.949 [main] DEBUG o.a.m.c.t.i.r.GenericUserBasedRecommender - Recommending items for user ID '710039'

11:39:31.965 [main] DEBUG o.a.m.c.t.i.r.GenericUserBasedRecommender - Recommendations are: [RecommendedItem[item:35222, value:4.0], RecommendedItem[item:12260, value:4.0], RecommendedItem[item:12223, value:1.5]]

11:39:31.966 [main] DEBUG o.h.p.mahout.meme.MemeProductRunner - 추천 아이템 : RecommendedItem[item:35222, value:4.0]

11:39:31.966 [main] DEBUG o.h.p.mahout.meme.MemeProductRunner - 추천 아이템 : RecommendedItem[item:12260, value:4.0]

11:39:31.967 [main] DEBUG o.h.p.mahout.meme.MemeProductRunner - 추천 아이템 : RecommendedItem[item:12223, value:1.5]


[Recommender]

similarity = new PearsonCorrelationSimilarity(dataModel);


// 이웃한 N명의 사용자 데이터로 추천 데이터 생성

// UserNeighborhood neighborhood = new NearestNUserNeighborhood(2, similarity, dataModel, 0.2);


// 특정 값이나 임계치를 넘는 모든 사용자의 데이터로 추천 데이터 생성, samplingrate : user sampling rate 10%

// UserNeighborhood neighborhood = new ThresholdUserNeighborhood(0.1, similarity, dataModel, 0.1);


UserNeighborhood neighborhood = new ThresholdUserNeighborhood(0.2, similarity, dataModel, 1.0);

recommender = new GenericUserBasedRecommender(dataModel, neighborhood, similarity);


- 데이터 크기가 너무 작아 ThresholdUserNeighborhood 를 이용하였습니다.


이와 같이 검색 클릭 로그를 기반으로 CF를 돌려 추천 데이터를 만드는 아주 간단한 방법을 알아봤습니다.

만든 추천 데이터에 대한 평가도 가능 합니다.

역시 examples 에 xxxxxxEvaluator 클래스들을 참고하셔서 구현해 보시면 됩니다.


저작자 표시 비영리 변경 금지
신고
크리에이티브 커먼즈 라이선스
Creative Commons License
Trackback 0 : Comment 0

[Logstash] 간헐적으로 logstash 가 죽습니다.

Elastic/Logstash 2017.01.12 11:35

aws 에 ec2 인스턴스를 이용해서 logstash 를 다양한 용도로 활용하고 있습니다.

그렇다 보니 시스템 자원을 최대한 뽑아서 사용을 하고 있어서 간헐적으로 logstash 가 죽는 문제가 발생을 했었습니다.


하나의 인스턴스에 logstash daemon 을 4개 띄워서 사용중에 있습니다.

그 이외 elasticsearch, kibana 등도 함께 띄워서 사용을 하고 있구요.


모든 데몬이 실행중일 경우 시스템 memory 자원은 1GB 정도 남게 되며 7GB 정도를 사용하게 됩니다.

처음 부터 7GB를 잡고 실행 되지는 않지만 JVM 특성상 full gc 돌기 전에는 조금씩 증가 하다 7GB 까지 도달하게 되고 좀 다양한 상황에 맞춰 700MB 이하로 까지 free memory 가 남기도 합니다.


반드시 free memory 가 1GB 미만 이면 죽는 것은 아니지만 죽을 확율이 높아 지는 것으로 보입니다.

실제 운영을 하고 분석을 해보니 그렇습니다.


혹시 운영 하시는 분들은 참고하시라고 글 올려 봤습니다.


[참고문서]

https://www.elastic.co/guide/en/logstash/2.4/performance-troubleshooting.html


[발췌문구]

Leave at least 1GB free for the OS and other processes.


저작자 표시 비영리 변경 금지
신고
크리에이티브 커먼즈 라이선스
Creative Commons License
Trackback 0 : Comment 0

[Elasticsearch] Range Query From, To 포함 여부.

Elastic/Elasticsearch 2017.01.02 12:44

도대체 왜 맨날 잊어버리는지 모르겠지만, 기억력 회복을 위해 기록해 봅니다.


Range query  사용 시 from, to, gt, gte, lt, lte parameter 를 사용 합니다.

RangeQueryBuilder.java 소스코드를 보면 아래와 같이 정의가 되어 있습니다.


private final String name;
private Object from;
private Object to;
private String timeZone;
private boolean includeLower = true;
private boolean includeUpper = true;
private float boost = -1;
private String queryName;
private String format;


기본적으로 lower, upper 값을 포함하게 되어 있습니다.

그러므로, from, to 는 값을 포함 하게 됩니다.

MySQL 에서 제공하고 있는 BETWEEN min AND max 도 min 과 max 값을 포함 하고 있는 것 처럼 동일 합니다.


저작자 표시 비영리 변경 금지
신고
크리에이티브 커먼즈 라이선스
Creative Commons License
Trackback 0 : Comment 0

[Elasticsearch] _cat nodes/indices help

Elastic/Elasticsearch 2016.12.06 12:18

그냥 볼일이 있어서 올려봅니다.

nodes 랑 indices 만.


[참고문서]

https://www.elastic.co/guide/en/elasticsearch/reference/current/cat-nodes.html

id                               | id,nodeId                          | unique node id                                                                                                   
pid                              | p                                  | process id                                                                                                       
host                             | h                                  | host name                                                                                                        
ip                               | i                                  | ip address                                                                                                       
port                             | po                                 | bound transport port                                                                                             
version                          | v                                  | es version                                                                                                       
build                            | b                                  | es build hash                                                                                                    
jdk                              | j                                  | jdk version                                                                                                      
disk.avail                       | d,disk,diskAvail                   | available disk space                                                                                             
heap.current                     | hc,heapCurrent                     | used heap                                                                                                        
heap.percent                     | hp,heapPercent                     | used heap ratio                                                                                                  
heap.max                         | hm,heapMax                         | max configured heap                                                                                              
ram.current                      | rc,ramCurrent                      | used machine memory                                                                                              
ram.percent                      | rp,ramPercent                      | used machine memory ratio                                                                                        
ram.max                          | rm,ramMax                          | total machine memory                                                                                             
file_desc.current                | fdc,fileDescriptorCurrent          | used file descriptors                                                                                            
file_desc.percent                | fdp,fileDescriptorPercent          | used file descriptor ratio                                                                                       
file_desc.max                    | fdm,fileDescriptorMax              | max file descriptors                                                                                             
cpu                              | cpu                                | recent cpu usage                                                                                                 
load                             | l                                  | most recent load avg                                                                                             
uptime                           | u                                  | node uptime                                                                                                      
node.role                        | r,role,dc,nodeRole                 | d:data node, c:client node                                                                                       
master                           | m                                  | m:master-eligible, *:current master                                                                              
name                             | n                                  | node name                                                                                                        
completion.size                  | cs,completionSize                  | size of completion                                                                                               
fielddata.memory_size            | fm,fielddataMemory                 | used fielddata cache                                                                                             
fielddata.evictions              | fe,fielddataEvictions              | fielddata evictions                                                                                              
query_cache.memory_size          | qcm,queryCacheMemory               | used query cache                                                                                                 
query_cache.evictions            | qce,queryCacheEvictions            | query cache evictions                                                                                            
request_cache.memory_size        | rcm,requestCacheMemory             | used request cache                                                                                               
request_cache.evictions          | rce,requestCacheEvictions          | request cache evictions                                                                                          
request_cache.hit_count          | rchc,requestCacheHitCount          | request cache hit counts                                                                                         
request_cache.miss_count         | rcmc,requestCacheMissCount         | request cache miss counts                                                                                        
flush.total                      | ft,flushTotal                      | number of flushes                                                                                                
flush.total_time                 | ftt,flushTotalTime                 | time spent in flush                                                                                              
get.current                      | gc,getCurrent                      | number of current get ops                                                                                        
get.time                         | gti,getTime                        | time spent in get                                                                                                
get.total                        | gto,getTotal                       | number of get ops                                                                                                
get.exists_time                  | geti,getExistsTime                 | time spent in successful gets                                                                                    
get.exists_total                 | geto,getExistsTotal                | number of successful gets                                                                                        
get.missing_time                 | gmti,getMissingTime                | time spent in failed gets                                                                                        
get.missing_total                | gmto,getMissingTotal               | number of failed gets                                                                                            
indexing.delete_current          | idc,indexingDeleteCurrent          | number of current deletions                                                                                      
indexing.delete_time             | idti,indexingDeleteTime            | time spent in deletions                                                                                          
indexing.delete_total            | idto,indexingDeleteTotal           | number of delete ops                                                                                             
indexing.index_current           | iic,indexingIndexCurrent           | number of current indexing ops                                                                                   
indexing.index_time              | iiti,indexingIndexTime             | time spent in indexing                                                                                           
indexing.index_total             | iito,indexingIndexTotal            | number of indexing ops                                                                                           
indexing.index_failed            | iif,indexingIndexFailed            | number of failed indexing ops                                                                                    
merges.current                   | mc,mergesCurrent                   | number of current merges                                                                                         
merges.current_docs              | mcd,mergesCurrentDocs              | number of current merging docs                                                                                   
merges.current_size              | mcs,mergesCurrentSize              | size of current merges                                                                                           
merges.total                     | mt,mergesTotal                     | number of completed merge ops                                                                                    
merges.total_docs                | mtd,mergesTotalDocs                | docs merged                                                                                                      
merges.total_size                | mts,mergesTotalSize                | size merged                                                                                                      
merges.total_time                | mtt,mergesTotalTime                | time spent in merges                                                                                             
percolate.current                | pc,percolateCurrent                | number of current percolations                                                                                   
percolate.memory_size            | pm,percolateMemory                 | memory used by percolations                                                                                      
percolate.queries                | pq,percolateQueries                | number of registered percolation queries                                                                         
percolate.time                   | pti,percolateTime                  | time spent percolating                                                                                           
percolate.total                  | pto,percolateTotal                 | total percolations                                                                                               
refresh.total                    | rto,refreshTotal                   | total refreshes                                                                                                  
refresh.time                     | rti,refreshTime                    | time spent in refreshes                                                                                          
script.compilations              | scrcc,scriptCompilations           | script compilations                                                                                              
script.cache_evictions           | scrce,scriptCacheEvictions         | script cache evictions                                                                                           
search.fetch_current             | sfc,searchFetchCurrent             | current fetch phase ops                                                                                          
search.fetch_time                | sfti,searchFetchTime               | time spent in fetch phase                                                                                        
search.fetch_total               | sfto,searchFetchTotal              | total fetch ops                                                                                                  
search.open_contexts             | so,searchOpenContexts              | open search contexts                                                                                             
search.query_current             | sqc,searchQueryCurrent             | current query phase ops                                                                                          
search.query_time                | sqti,searchQueryTime               | time spent in query phase                                                                                        
search.query_total               | sqto,searchQueryTotal              | total query phase ops                                                                                            
search.scroll_current            | scc,searchScrollCurrent            | open scroll contexts                                                                                             
search.scroll_time               | scti,searchScrollTime              | time scroll contexts held open                                                                                   
search.scroll_total              | scto,searchScrollTotal             | completed scroll contexts                                                                                        
segments.count                   | sc,segmentsCount                   | number of segments                                                                                               
segments.memory                  | sm,segmentsMemory                  | memory used by segments                                                                                          
segments.index_writer_memory     | siwm,segmentsIndexWriterMemory     | memory used by index writer                                                                                      
segments.index_writer_max_memory | siwmx,segmentsIndexWriterMaxMemory | maximum memory index writer may use before it must write buffered documents to a new segment                     
segments.version_map_memory      | svmm,segmentsVersionMapMemory      | memory used by version map                                                                                       
segments.fixed_bitset_memory     | sfbm,fixedBitsetMemory             | memory used by fixed bit sets for nested object field types and type filters for types referred in _parent fields
suggest.current                  | suc,suggestCurrent                 | number of current suggest ops                                                                                    
suggest.time                     | suti,suggestTime                   | time spend in suggest                                                                                            
suggest.total                    | suto,suggestTotal                  | number of suggest ops    


[참고문서]

https://www.elastic.co/guide/en/elasticsearch/reference/current/cat-indices.html

health                               | h                                  | current health status                                                                                            
status                               | s                                  | open/close status                                                                                                
index                                | i,idx                              | index name                                                                                                       
pri                                  | p,shards.primary,shardsPrimary     | number of primary shards                                                                                         
rep                                  | r,shards.replica,shardsReplica     | number of replica shards                                                                                         
docs.count                           | dc,docsCount                       | available docs                                                                                                   
docs.deleted                         | dd,docsDeleted                     | deleted docs                                                                                                     
creation.date                        | cd                                 | index creation date (millisecond value)                                                                          
creation.date.string                 | cds                                | index creation date (as string)                                                                                  
store.size                           | ss,storeSize                       | store size of primaries & replicas                                                                               
pri.store.size                       |                                    | store size of primaries                                                                                          
completion.size                      | cs,completionSize                  | size of completion                                                                                               
pri.completion.size                  |                                    | size of completion                                                                                               
fielddata.memory_size                | fm,fielddataMemory                 | used fielddata cache                                                                                             
pri.fielddata.memory_size            |                                    | used fielddata cache                                                                                             
fielddata.evictions                  | fe,fielddataEvictions              | fielddata evictions                                                                                              
pri.fielddata.evictions              |                                    | fielddata evictions                                                                                              
query_cache.memory_size              | qcm,queryCacheMemory               | used query cache                                                                                                 
pri.query_cache.memory_size          |                                    | used query cache                                                                                                 
query_cache.evictions                | qce,queryCacheEvictions            | query cache evictions                                                                                            
pri.query_cache.evictions            |                                    | query cache evictions                                                                                            
request_cache.memory_size            | rcm,requestCacheMemory             | used request cache                                                                                               
pri.request_cache.memory_size        |                                    | used request cache                                                                                               
request_cache.evictions              | rce,requestCacheEvictions          | request cache evictions                                                                                          
pri.request_cache.evictions          |                                    | request cache evictions                                                                                          
request_cache.hit_count              | rchc,requestCacheHitCount          | request cache hit count                                                                                          
pri.request_cache.hit_count          |                                    | request cache hit count                                                                                          
request_cache.miss_count             | rcmc,requestCacheMissCount         | request cache miss count                                                                                         
pri.request_cache.miss_count         |                                    | request cache miss count                                                                                         
flush.total                          | ft,flushTotal                      | number of flushes                                                                                                
pri.flush.total                      |                                    | number of flushes                                                                                                
flush.total_time                     | ftt,flushTotalTime                 | time spent in flush                                                                                              
pri.flush.total_time                 |                                    | time spent in flush                                                                                              
get.current                          | gc,getCurrent                      | number of current get ops                                                                                        
pri.get.current                      |                                    | number of current get ops                                                                                        
get.time                             | gti,getTime                        | time spent in get                                                                                                
pri.get.time                         |                                    | time spent in get                                                                                                
get.total                            | gto,getTotal                       | number of get ops                                                                                                
pri.get.total                        |                                    | number of get ops                                                                                                
get.exists_time                      | geti,getExistsTime                 | time spent in successful gets                                                                                    
pri.get.exists_time                  |                                    | time spent in successful gets                                                                                    
get.exists_total                     | geto,getExistsTotal                | number of successful gets                                                                                        
pri.get.exists_total                 |                                    | number of successful gets                                                                                        
get.missing_time                     | gmti,getMissingTime                | time spent in failed gets                                                                                        
pri.get.missing_time                 |                                    | time spent in failed gets                                                                                        
get.missing_total                    | gmto,getMissingTotal               | number of failed gets                                                                                            
pri.get.missing_total                |                                    | number of failed gets                                                                                            
indexing.delete_current              | idc,indexingDeleteCurrent          | number of current deletions                                                                                      
pri.indexing.delete_current          |                                    | number of current deletions                                                                                      
indexing.delete_time                 | idti,indexingDeleteTime            | time spent in deletions                                                                                          
pri.indexing.delete_time             |                                    | time spent in deletions                                                                                          
indexing.delete_total                | idto,indexingDeleteTotal           | number of delete ops                                                                                             
pri.indexing.delete_total            |                                    | number of delete ops                                                                                             
indexing.index_current               | iic,indexingIndexCurrent           | number of current indexing ops                                                                                   
pri.indexing.index_current           |                                    | number of current indexing ops                                                                                   
indexing.index_time                  | iiti,indexingIndexTime             | time spent in indexing                                                                                           
pri.indexing.index_time              |                                    | time spent in indexing                                                                                           
indexing.index_total                 | iito,indexingIndexTotal            | number of indexing ops                                                                                           
pri.indexing.index_total             |                                    | number of indexing ops                                                                                           
indexing.index_failed                | iif,indexingIndexFailed            | number of failed indexing ops                                                                                    
pri.indexing.index_failed            |                                    | number of failed indexing ops                                                                                    
merges.current                       | mc,mergesCurrent                   | number of current merges                                                                                         
pri.merges.current                   |                                    | number of current merges                                                                                         
merges.current_docs                  | mcd,mergesCurrentDocs              | number of current merging docs                                                                                   
pri.merges.current_docs              |                                    | number of current merging docs                                                                                   
merges.current_size                  | mcs,mergesCurrentSize              | size of current merges                                                                                           
pri.merges.current_size              |                                    | size of current merges                                                                                           
merges.total                         | mt,mergesTotal                     | number of completed merge ops                                                                                    
pri.merges.total                     |                                    | number of completed merge ops                                                                                    
merges.total_docs                    | mtd,mergesTotalDocs                | docs merged                                                                                                      
pri.merges.total_docs                |                                    | docs merged                                                                                                      
merges.total_size                    | mts,mergesTotalSize                | size merged                                                                                                      
pri.merges.total_size                |                                    | size merged                                                                                                      
merges.total_time                    | mtt,mergesTotalTime                | time spent in merges                                                                                             
pri.merges.total_time                |                                    | time spent in merges                                                                                             
percolate.current                    | pc,percolateCurrent                | number of current percolations                                                                                   
pri.percolate.current                |                                    | number of current percolations                                                                                   
percolate.memory_size                | pm,percolateMemory                 | memory used by percolations                                                                                      
pri.percolate.memory_size            |                                    | memory used by percolations                                                                                      
percolate.queries                    | pq,percolateQueries                | number of registered percolation queries                                                                         
pri.percolate.queries                |                                    | number of registered percolation queries                                                                         
percolate.time                       | pti,percolateTime                  | time spent percolating                                                                                           
pri.percolate.time                   |                                    | time spent percolating                                                                                           
percolate.total                      | pto,percolateTotal                 | total percolations                                                                                               
pri.percolate.total                  |                                    | total percolations                                                                                               
refresh.total                        | rto,refreshTotal                   | total refreshes                                                                                                  
pri.refresh.total                    |                                    | total refreshes                                                                                                  
refresh.time                         | rti,refreshTime                    | time spent in refreshes                                                                                          
pri.refresh.time                     |                                    | time spent in refreshes                                                                                          
search.fetch_current                 | sfc,searchFetchCurrent             | current fetch phase ops                                                                                          
pri.search.fetch_current             |                                    | current fetch phase ops                                                                                          
search.fetch_time                    | sfti,searchFetchTime               | time spent in fetch phase                                                                                        
pri.search.fetch_time                |                                    | time spent in fetch phase                                                                                        
search.fetch_total                   | sfto,searchFetchTotal              | total fetch ops                                                                                                  
pri.search.fetch_total               |                                    | total fetch ops                                                                                                  
search.open_contexts                 | so,searchOpenContexts              | open search contexts                                                                                             
pri.search.open_contexts             |                                    | open search contexts                                                                                             
search.query_current                 | sqc,searchQueryCurrent             | current query phase ops                                                                                          
pri.search.query_current             |                                    | current query phase ops                                                                                          
search.query_time                    | sqti,searchQueryTime               | time spent in query phase                                                                                        
pri.search.query_time                |                                    | time spent in query phase                                                                                        
search.query_total                   | sqto,searchQueryTotal              | total query phase ops                                                                                            
pri.search.query_total               |                                    | total query phase ops                                                                                            
search.scroll_current                | scc,searchScrollCurrent            | open scroll contexts                                                                                             
pri.search.scroll_current            |                                    | open scroll contexts                                                                                             
search.scroll_time                   | scti,searchScrollTime              | time scroll contexts held open                                                                                   
pri.search.scroll_time               |                                    | time scroll contexts held open                                                                                   
search.scroll_total                  | scto,searchScrollTotal             | completed scroll contexts                                                                                        
pri.search.scroll_total              |                                    | completed scroll contexts                                                                                        
segments.count                       | sc,segmentsCount                   | number of segments                                                                                               
pri.segments.count                   |                                    | number of segments                                                                                               
segments.memory                      | sm,segmentsMemory                  | memory used by segments                                                                                          
pri.segments.memory                  |                                    | memory used by segments                                                                                          
segments.index_writer_memory         | siwm,segmentsIndexWriterMemory     | memory used by index writer                                                                                      
pri.segments.index_writer_memory     |                                    | memory used by index writer                                                                                      
segments.index_writer_max_memory     | siwmx,segmentsIndexWriterMaxMemory | maximum memory index writer may use before it must write buffered documents to a new segment                     
pri.segments.index_writer_max_memory |                                    | maximum memory index writer may use before it must write buffered documents to a new segment                     
segments.version_map_memory          | svmm,segmentsVersionMapMemory      | memory used by version map                                                                                       
pri.segments.version_map_memory      |                                    | memory used by version map                                                                                       
segments.fixed_bitset_memory         | sfbm,fixedBitsetMemory             | memory used by fixed bit sets for nested object field types and type filters for types referred in _parent fields
pri.segments.fixed_bitset_memory     |                                    | memory used by fixed bit sets for nested object field types and type filters for types referred in _parent fields
warmer.current                       | wc,warmerCurrent                   | current warmer ops                                                                                               
pri.warmer.current                   |                                    | current warmer ops                                                                                               
warmer.total                         | wto,warmerTotal                    | total warmer ops                                                                                                 
pri.warmer.total                     |                                    | total warmer ops                                                                                                 
warmer.total_time                    | wtt,warmerTotalTime                | time spent in warmers                                                                                            
pri.warmer.total_time                |                                    | time spent in warmers                                                                                            
suggest.current                      | suc,suggestCurrent                 | number of current suggest ops                                                                                    
pri.suggest.current                  |                                    | number of current suggest ops                                                                                    
suggest.time                         | suti,suggestTime                   | time spend in suggest                                                                                            
pri.suggest.time                     |                                    | time spend in suggest                                                                                            
suggest.total                        | suto,suggestTotal                  | number of suggest ops                                                                                            
pri.suggest.total                    |                                    | number of suggest ops                                                                                            
memory.total                         | tm,memoryTotal                     | total used memory                                                                                                
pri.memory.total                     |                                    | total user memory  


저작자 표시 비영리 변경 금지
신고
크리에이티브 커먼즈 라이선스
Creative Commons License
Trackback 0 : Comment 0

[Elasticsearch] Lucene Arirang Analyzer Plugin for Elasticsearch 5.0.1

Elastic/Elasticsearch 2016.11.24 19:02

우선 빌드한 플러그인 zip 파일 먼저 공유 합니다.

나중에 작업한 내용에 대해서는 github 에 올리도록 하겠습니다.

요즘 프로젝트며 운영 업무가 너무 많아서 이것도 겨우 겨우 시간 내서 작업 했내요.


elasticsearch-analysis-arirang-5.0.1.zip


설치 방법)

$ bin/elasticsearch-plugin install --verbose file:///elasticsearch-analysis-arirang/target/elasticsearch-analysis-arirang-5.0.1.zip


설치 로그)

-> Downloading file:///elasticsearch-analysis-arirang-5.0.1.zip

Retrieving zip from file:///elasticsearch-analysis-arirang-5.0.1.zip

[=================================================] 100%

- Plugin information:

Name: analysis-arirang

Description: Arirang plugin

Version: 5.0.1

 * Classname: org.elasticsearch.plugin.analysis.arirang.AnalysisArirangPlugin

-> Installed analysis-arirang


Elasticsearch 실행 로그)

$ bin/elasticsearch

[2016-11-24T18:49:09,922][INFO ][o.e.n.Node               ] [] initializing ...

[2016-11-24T18:49:10,083][INFO ][o.e.e.NodeEnvironment    ] [aDGu2B9] using [1] data paths, mounts [[/ (/dev/disk1)]], net usable_space [733.1gb], net total_space [930.3gb], spins? [unknown], types [hfs]

[2016-11-24T18:49:10,084][INFO ][o.e.e.NodeEnvironment    ] [aDGu2B9] heap size [1.9gb], compressed ordinary object pointers [true]

[2016-11-24T18:49:10,085][INFO ][o.e.n.Node               ] [aDGu2B9] node name [aDGu2B9] derived from node ID; set [node.name] to override

[2016-11-24T18:49:10,087][INFO ][o.e.n.Node               ] [aDGu2B9] version[5.0.1], pid[56878], build[080bb47/2016-11-11T22:08:49.812Z], OS[Mac OS X/10.12.1/x86_64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_72/25.72-b15]

[2016-11-24T18:49:11,335][INFO ][o.e.p.PluginsService     ] [aDGu2B9] loaded module [aggs-matrix-stats]

[2016-11-24T18:49:11,335][INFO ][o.e.p.PluginsService     ] [aDGu2B9] loaded module [ingest-common]

[2016-11-24T18:49:11,335][INFO ][o.e.p.PluginsService     ] [aDGu2B9] loaded module [lang-expression]

[2016-11-24T18:49:11,335][INFO ][o.e.p.PluginsService     ] [aDGu2B9] loaded module [lang-groovy]

[2016-11-24T18:49:11,335][INFO ][o.e.p.PluginsService     ] [aDGu2B9] loaded module [lang-mustache]

[2016-11-24T18:49:11,336][INFO ][o.e.p.PluginsService     ] [aDGu2B9] loaded module [lang-painless]

[2016-11-24T18:49:11,336][INFO ][o.e.p.PluginsService     ] [aDGu2B9] loaded module [percolator]

[2016-11-24T18:49:11,336][INFO ][o.e.p.PluginsService     ] [aDGu2B9] loaded module [reindex]

[2016-11-24T18:49:11,336][INFO ][o.e.p.PluginsService     ] [aDGu2B9] loaded module [transport-netty3]

[2016-11-24T18:49:11,336][INFO ][o.e.p.PluginsService     ] [aDGu2B9] loaded module [transport-netty4]

[2016-11-24T18:49:11,336][INFO ][o.e.p.PluginsService     ] [aDGu2B9] loaded plugin [analysis-arirang]

[2016-11-24T18:49:14,151][INFO ][o.e.n.Node               ] [aDGu2B9] initialized

[2016-11-24T18:49:14,151][INFO ][o.e.n.Node               ] [aDGu2B9] starting ...

[2016-11-24T18:49:14,377][INFO ][o.e.t.TransportService   ] [aDGu2B9] publish_address {127.0.0.1:9300}, bound_addresses {[fe80::1]:9300}, {[::1]:9300}, {127.0.0.1:9300}

[2016-11-24T18:49:17,511][INFO ][o.e.c.s.ClusterService   ] [aDGu2B9] new_master {aDGu2B9}{aDGu2B9mQ8KkWCe3fnqeMw}{_y9RzyKGSvqYAFcv99HBXg}{127.0.0.1}{127.0.0.1:9300}, reason: zen-disco-elected-as-master ([0] nodes joined)

[2016-11-24T18:49:17,584][INFO ][o.e.g.GatewayService     ] [aDGu2B9] recovered [0] indices into cluster_state

[2016-11-24T18:49:17,588][INFO ][o.e.h.HttpServer         ] [aDGu2B9] publish_address {127.0.0.1:9200}, bound_addresses {[fe80::1]:9200}, {[::1]:9200}, {127.0.0.1:9200}

[2016-11-24T18:49:17,588][INFO ][o.e.n.Node               ] [aDGu2B9] started


한글형태소분석 실행)

$ curl -X POST -H "Cache-Control: no-cache" -H "Postman-Token: 6d392d83-5816-71ad-556b-5cd6f92af634" -d '{

  "analyzer" : "arirang_analyzer",

  "text" : "[한국] 엘라스틱서치 사용자 그룹의 HENRY 입니다."

}' "http://localhost:9200/_analyze"


형태소분석 결과)

{

  "tokens": [

    {

      "token": "[",

      "start_offset": 0,

      "end_offset": 1,

      "type": "symbol",

      "position": 0

    },

    {

      "token": "한국",

      "start_offset": 1,

      "end_offset": 3,

      "type": "korean",

      "position": 1

    },

    {

      "token": "]",

      "start_offset": 3,

      "end_offset": 4,

      "type": "symbol",

      "position": 2

    },

    {

      "token": "엘라스틱서치",

      "start_offset": 5,

      "end_offset": 11,

      "type": "korean",

      "position": 3

    },

    {

      "token": "엘라",

      "start_offset": 5,

      "end_offset": 7,

      "type": "korean",

      "position": 3

    },

    {

      "token": "스틱",

      "start_offset": 7,

      "end_offset": 9,

      "type": "korean",

      "position": 4

    },

    {

      "token": "서치",

      "start_offset": 9,

      "end_offset": 11,

      "type": "korean",

      "position": 5

    },

    {

      "token": "사용자",

      "start_offset": 12,

      "end_offset": 15,

      "type": "korean",

      "position": 6

    },

    {

      "token": "그룹",

      "start_offset": 16,

      "end_offset": 18,

      "type": "korean",

      "position": 7

    },

    {

      "token": "henry",

      "start_offset": 20,

      "end_offset": 25,

      "type": "word",

      "position": 8

    },

    {

      "token": "입니다",

      "start_offset": 26,

      "end_offset": 29,

      "type": "korean",

      "position": 9

    }

  ]

}


저작자 표시 비영리 변경 금지
신고
크리에이티브 커먼즈 라이선스
Creative Commons License
Trackback 0 : Comment 0

[Kibana] Date Histogram Advanced DSL

Elastic/Kibana 2016.11.21 10:34

Kibana 를 사용하면서 많이 사용하는 것중 하나가 바로 Date Histogram Aggregation 입니다.

기본 결과에 대한 정렬은 count 에 대한 desc 으로 결과가 전달 되는데요.

날짜에 대한 desc 정렬을 하고 싶은 경우 아래와 같은 스크립트를 추가 하시면 됩니다.


제가 많이 사용하는 스크립트라 매번 타이핑 하기 귀찮아서 올려 봅니다.


{

"field":"time",

"interval":"1w",

  "order": {

    "_key": "desc"

  }

}


- field 값에 time 을 넣은 이유는 @timestamp 를 사용하기 않기 때문 입니다. :)

- order 에 사용 가능한 field 는 

Date Histogram 에서는 _key, _count

Terms 에서는 _term, _count

입니다.


저작자 표시 비영리 변경 금지
신고
크리에이티브 커먼즈 라이선스
Creative Commons License
Trackback 0 : Comment 0

[Elasticsearch] Elasticsearch 5.0 GA 살펴보기 - 2

Elastic/Elasticsearch 2016.11.02 17:49

site plugin head 설치를 해보려 합니다.

설치 방법이 달라졌습니다.


[사전준비]

Elasticsearch 5.0 GA 살펴보기 - 1 에서 클러스터 구성

그냥 bin/elasticsearch, bin/elasticsearch 두 번 하시면 됩니다.


[설치 명령어]

https://www.elastic.co/guide/en/elasticsearch/plugins/current/installation.html#_core_elasticsearch_plugins


$ bin/elasticsearch-plugin install [plugin-name]


[Head Plugins]

https://github.com/mobz/elasticsearch-head#running-with-built-in-server


설치 방법은 위 문서에 잘 나와 있습니다.

아래는 제가 그냥 실행한 명령어를 나열 하였습니다.


$ git clone git://github.com/mobz/elasticsearch-head.git

$ cd elasticsearch-head/

$ npm install

$ sudo npm install -g grunt-cli

$ grunt server


[Head 접속]

$ open http://localhost:9100

- cross origin  정책으로 인해서 초기 접속이 되지 않습니다.

- elasticsearch.yml 에 아래 내용 추가해 주시고 재실행 하시면 됩니다.


[Cross Origin 정책수정]

$ vi config/elasticsearch.yml


http.cors.enabled: true

http.cors.allow-origin: "*"




노드명이 많이 아쉽내요 ㅠ.ㅠ

저작자 표시 비영리 변경 금지
신고
크리에이티브 커먼즈 라이선스
Creative Commons License
Trackback 0 : Comment 0

[Elasticsearch] Elasticsearch 5.0 GA 살펴보기 - 1

Elastic/Elasticsearch 2016.11.02 17:11
내년에 업그레이드를 해야 하기 때문에 한번 살펴 보고자 합니다.

오늘은 가볍게 다운 받고 압축 풀고 기본 구조가 어떻게 변경이 되었나 한번 살펴 보려 합니다.


[다운로드]

https://www.elastic.co/downloads/elasticsearch


$ wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.0.0.tar.gz


[압축해제]

$ tar -xvzf elasticsearch-5.0.0.tar.gz

$ ln -s elasticsearch-5.0.0 elasticsearch


[바로실행]

$ bin/elasticsearch


[2016-11-02T16:52:22,051][INFO ][o.e.n.Node               ] [] initializing ...

[2016-11-02T16:52:22,246][INFO ][o.e.e.NodeEnvironment    ] [DiStpli] using [1] data paths, mounts [[/ (/dev/disk1)]], net usable_space [734.4gb], net total_space [930.3gb], spins? [unknown], types [hfs]

[2016-11-02T16:52:22,247][INFO ][o.e.e.NodeEnvironment    ] [DiStpli] heap size [1.9gb], compressed ordinary object pointers [true]

[2016-11-02T16:52:22,248][INFO ][o.e.n.Node               ] [DiStpli] node name [DiStpli] derived from node ID; set [node.name] to override

[2016-11-02T16:52:22,268][INFO ][o.e.n.Node               ] [DiStpli] version[5.0.0], pid[14095], build[253032b/2016-10-26T04:37:51.531Z], OS[Mac OS X/10.12.1/x86_64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_72/25.72-b15]

[2016-11-02T16:52:23,487][INFO ][o.e.p.PluginsService     ] [DiStpli] loaded module [aggs-matrix-stats]

[2016-11-02T16:52:23,487][INFO ][o.e.p.PluginsService     ] [DiStpli] loaded module [ingest-common]

[2016-11-02T16:52:23,488][INFO ][o.e.p.PluginsService     ] [DiStpli] loaded module [lang-expression]

[2016-11-02T16:52:23,488][INFO ][o.e.p.PluginsService     ] [DiStpli] loaded module [lang-groovy]

[2016-11-02T16:52:23,488][INFO ][o.e.p.PluginsService     ] [DiStpli] loaded module [lang-mustache]

[2016-11-02T16:52:23,488][INFO ][o.e.p.PluginsService     ] [DiStpli] loaded module [lang-painless]

[2016-11-02T16:52:23,488][INFO ][o.e.p.PluginsService     ] [DiStpli] loaded module [percolator]

[2016-11-02T16:52:23,488][INFO ][o.e.p.PluginsService     ] [DiStpli] loaded module [reindex]

[2016-11-02T16:52:23,488][INFO ][o.e.p.PluginsService     ] [DiStpli] loaded module [transport-netty3]

[2016-11-02T16:52:23,488][INFO ][o.e.p.PluginsService     ] [DiStpli] loaded module [transport-netty4]

[2016-11-02T16:52:23,489][INFO ][o.e.p.PluginsService     ] [DiStpli] no plugins loaded

[2016-11-02T16:52:27,399][INFO ][o.e.n.Node               ] [DiStpli] initialized

[2016-11-02T16:52:27,399][INFO ][o.e.n.Node               ] [DiStpli] starting ...

[2016-11-02T16:52:27,809][INFO ][o.e.t.TransportService   ] [DiStpli] publish_address {127.0.0.1:9300}, bound_addresses {[fe80::1]:9300}, {[::1]:9300}, {127.0.0.1:9300}

[2016-11-02T16:52:30,900][INFO ][o.e.c.s.ClusterService   ] [DiStpli] new_master {DiStpli}{DiStpli8S4-rE5QKCPP_Lw}{bwHQPV_UTl6NSPCfx8IE3w}{127.0.0.1}{127.0.0.1:9300}, reason: zen-disco-elected-as-master ([0] nodes joined)

[2016-11-02T16:52:30,974][INFO ][o.e.h.HttpServer         ] [DiStpli] publish_address {127.0.0.1:9200}, bound_addresses {[fe80::1]:9200}, {[::1]:9200}, {127.0.0.1:9200}

[2016-11-02T16:52:30,974][INFO ][o.e.n.Node               ] [DiStpli] started

[2016-11-02T16:52:30,975][INFO ][o.e.g.GatewayService     ] [DiStpli] recovered [0] indices into cluster_state


[접속]

http://localhost:9200


{
  "name" : "DiStpli",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "eA4iUJ4BSoOxmoDbLjFgoQ",
  "version" : {
    "number" : "5.0.0",
    "build_hash" : "253032b",
    "build_date" : "2016-10-26T04:37:51.531Z",
    "build_snapshot" : false,
    "lucene_version" : "6.2.0"
  },
  "tagline" : "You Know, for Search"
}



[디렉토리구조]

$ ls -al

total 56

drwxr-xr-x  12  staff    408 11  2 16:52 .

drwxr-xr-x  34  staff   1156 11  2 16:45 ..

-rw-r--r--   1  staff  11358 10 26 13:35 LICENSE.txt

-rw-r--r--   1  staff    150 10 26 13:35 NOTICE.txt

-rw-r--r--   1  staff   9108 10 26 13:35 README.textile

drwxr-xr-x  15  staff    510 10 26 13:40 bin

drwxr-xr-x   6  staff    204 11  2 16:52 config

drwxr-xr-x   3  staff    102 11  2 16:52 data

drwxr-xr-x  36  staff   1224 10 26 13:40 lib

drwxr-xr-x   6  staff    204 11  2 16:52 logs

drwxr-xr-x  12  staff    408 10 26 13:40 modules

drwxr-xr-x   2  staff     68 11  2 16:52 plugins


- 실행 후 모습 입니다.


[디렉토리 bin]

- 실행 파일들이 위치해 있습니다.


[디렉토리 config]

- elasticsearch.yml 은 그대로 존재 합니다.

- logging.yml 은 log4j2.properties 로 변경 되었습니다.

- jvm.options 가 추가 되었습니다.

기존에 실행 시 설정에 필요했던 jvm 옵션을 스크립트에 추가 하기 위해 커스텀하게 사용했는데요. 이 옵션이 생겨서 좀 더 유연해 졌내요.

- scripts 디렉토리가 생겼습니다.

잘 아시겠지만 각종 script 관련 코드를 이곳에 저장하고 읽어 들일 수 있도록 된 것 같습니다.


[디렉토리 data]

- 예전 그대로 같지만 조금 달라졌습니다.

즉, 예전에는 data/클러스터명/nodes/0 뭐 이런 식이였는데요.

지금은 data/nodes/0 으로 생성이 되내요.

클러스터명이 사라졌내요. (여전히 default cluster_name 은 elasticsearch 입니다.)


[디렉토리 lib]

- 바뀐건 종속 라이브러리가 바뀌었내요.


[디렉토리 modules]

- elastic 에서 제공하는 추가 modules 들이 들어 있습니다.


[디렉토리 plugins]

- 바뀐건 아마도 plugin 만드는 방법이 바뀌었을 것 같습니다.


[초간단 클러스터 구성 테스트]

- 기본적으로 elasticsearch 는 bin/elasticsearch 를 여러번 실행 하면 클러스터로 묶어 줍니다.

단, 인스턴스를 분리 하셔야 합니다.


$ bin/elasticsearch

[2016-11-02T17:21:04,689][INFO ][o.e.n.Node               ] [] initializing ...

[2016-11-02T17:21:04,770][INFO ][o.e.e.NodeEnvironment    ] [npjHq7P] using [1] data paths, mounts [[/ (/dev/disk1)]], net usable_space [734.4gb], net total_space [930.3gb], spins? [unknown], types [hfs]

[2016-11-02T17:21:04,771][INFO ][o.e.e.NodeEnvironment    ] [npjHq7P] heap size [1.9gb], compressed ordinary object pointers [true]

[2016-11-02T17:21:04,772][INFO ][o.e.n.Node               ] [npjHq7P] node name [npjHq7P] derived from node ID; set [node.name] to override

[2016-11-02T17:21:04,773][INFO ][o.e.n.Node               ] [npjHq7P] version[5.0.0], pid[14680], build[253032b/2016-10-26T04:37:51.531Z], OS[Mac OS X/10.12.1/x86_64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_72/25.72-b15]

[2016-11-02T17:21:05,375][INFO ][o.e.p.PluginsService     ] [npjHq7P] loaded module [aggs-matrix-stats]

[2016-11-02T17:21:05,375][INFO ][o.e.p.PluginsService     ] [npjHq7P] loaded module [ingest-common]

[2016-11-02T17:21:05,376][INFO ][o.e.p.PluginsService     ] [npjHq7P] loaded module [lang-expression]

[2016-11-02T17:21:05,376][INFO ][o.e.p.PluginsService     ] [npjHq7P] loaded module [lang-groovy]

[2016-11-02T17:21:05,376][INFO ][o.e.p.PluginsService     ] [npjHq7P] loaded module [lang-mustache]

[2016-11-02T17:21:05,376][INFO ][o.e.p.PluginsService     ] [npjHq7P] loaded module [lang-painless]

[2016-11-02T17:21:05,376][INFO ][o.e.p.PluginsService     ] [npjHq7P] loaded module [percolator]

[2016-11-02T17:21:05,376][INFO ][o.e.p.PluginsService     ] [npjHq7P] loaded module [reindex]

[2016-11-02T17:21:05,376][INFO ][o.e.p.PluginsService     ] [npjHq7P] loaded module [transport-netty3]

[2016-11-02T17:21:05,376][INFO ][o.e.p.PluginsService     ] [npjHq7P] loaded module [transport-netty4]

[2016-11-02T17:21:05,377][INFO ][o.e.p.PluginsService     ] [npjHq7P] no plugins loaded

[2016-11-02T17:21:06,764][INFO ][o.e.n.Node               ] [npjHq7P] initialized

[2016-11-02T17:21:06,765][INFO ][o.e.n.Node               ] [npjHq7P] starting ...

[2016-11-02T17:21:06,895][INFO ][o.e.t.TransportService   ] [npjHq7P] publish_address {127.0.0.1:9301}, bound_addresses {[fe80::1]:9301}, {[::1]:9301}, {127.0.0.1:9301}

[2016-11-02T17:21:09,996][INFO ][o.e.c.s.ClusterService   ] [npjHq7P] detected_master {DiStpli}{DiStpli8S4-rE5QKCPP_Lw}{oJ9y-SxJTYex3ZNBmRIpoQ}{127.0.0.1}{127.0.0.1:9300}, added {{DiStpli}{DiStpli8S4-rE5QKCPP_Lw}{oJ9y-SxJTYex3ZNBmRIpoQ}{127.0.0.1}{127.0.0.1:9300},}, reason: zen-disco-receive(from master [master {DiStpli}{DiStpli8S4-rE5QKCPP_Lw}{oJ9y-SxJTYex3ZNBmRIpoQ}{127.0.0.1}{127.0.0.1:9300} committed version [7]])

[2016-11-02T17:21:10,030][INFO ][o.e.h.HttpServer         ] [npjHq7P] publish_address {127.0.0.1:9201}, bound_addresses {[fe80::1]:9201}, {[::1]:9201}, {127.0.0.1:9201}

[2016-11-02T17:21:10,031][INFO ][o.e.n.Node               ] [npjHq7P] started



$ bin/elasticsearch

[2016-11-02T17:19:34,869][INFO ][o.e.n.Node               ] [] initializing ...

[2016-11-02T17:19:35,074][INFO ][o.e.e.NodeEnvironment    ] [DiStpli] using [1] data paths, mounts [[/ (/dev/disk1)]], net usable_space [734.4gb], net total_space [930.3gb], spins? [unknown], types [hfs]

[2016-11-02T17:19:35,074][INFO ][o.e.e.NodeEnvironment    ] [DiStpli] heap size [1.9gb], compressed ordinary object pointers [true]

[2016-11-02T17:19:35,076][INFO ][o.e.n.Node               ] [DiStpli] node name [DiStpli] derived from node ID; set [node.name] to override

[2016-11-02T17:19:35,079][INFO ][o.e.n.Node               ] [DiStpli] version[5.0.0], pid[14588], build[253032b/2016-10-26T04:37:51.531Z], OS[Mac OS X/10.12.1/x86_64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_72/25.72-b15]

[2016-11-02T17:19:37,110][INFO ][o.e.p.PluginsService     ] [DiStpli] loaded module [aggs-matrix-stats]

[2016-11-02T17:19:37,111][INFO ][o.e.p.PluginsService     ] [DiStpli] loaded module [ingest-common]

[2016-11-02T17:19:37,111][INFO ][o.e.p.PluginsService     ] [DiStpli] loaded module [lang-expression]

[2016-11-02T17:19:37,111][INFO ][o.e.p.PluginsService     ] [DiStpli] loaded module [lang-groovy]

[2016-11-02T17:19:37,111][INFO ][o.e.p.PluginsService     ] [DiStpli] loaded module [lang-mustache]

[2016-11-02T17:19:37,111][INFO ][o.e.p.PluginsService     ] [DiStpli] loaded module [lang-painless]

[2016-11-02T17:19:37,111][INFO ][o.e.p.PluginsService     ] [DiStpli] loaded module [percolator]

[2016-11-02T17:19:37,111][INFO ][o.e.p.PluginsService     ] [DiStpli] loaded module [reindex]

[2016-11-02T17:19:37,111][INFO ][o.e.p.PluginsService     ] [DiStpli] loaded module [transport-netty3]

[2016-11-02T17:19:37,111][INFO ][o.e.p.PluginsService     ] [DiStpli] loaded module [transport-netty4]

[2016-11-02T17:19:37,111][INFO ][o.e.p.PluginsService     ] [DiStpli] no plugins loaded

[2016-11-02T17:19:40,144][INFO ][o.e.n.Node               ] [DiStpli] initialized

[2016-11-02T17:19:40,145][INFO ][o.e.n.Node               ] [DiStpli] starting ...

[2016-11-02T17:19:40,356][INFO ][o.e.t.TransportService   ] [DiStpli] publish_address {127.0.0.1:9300}, bound_addresses {[fe80::1]:9300}, {[::1]:9300}, {127.0.0.1:9300}



[2016-11-02T17:19:43,625][INFO ][o.e.c.s.ClusterService   ] [DiStpli] new_master {DiStpli}{DiStpli8S4-rE5QKCPP_Lw}{oJ9y-SxJTYex3ZNBmRIpoQ}{127.0.0.1}{127.0.0.1:9300}, reason: zen-disco-elected-as-master ([0] nodes joined)

[2016-11-02T17:19:43,742][INFO ][o.e.g.GatewayService     ] [DiStpli] recovered [0] indices into cluster_state

[2016-11-02T17:19:43,750][INFO ][o.e.h.HttpServer         ] [DiStpli] publish_address {127.0.0.1:9200}, bound_addresses {[fe80::1]:9200}, {[::1]:9200}, {127.0.0.1:9200}

[2016-11-02T17:19:43,750][INFO ][o.e.n.Node               ] [DiStpli] started

[2016-11-02T17:21:09,966][INFO ][o.e.c.s.ClusterService   ] [DiStpli] added {{npjHq7P}{npjHq7PlTqqiB2bhXkIxqQ}{IhQPGv_rTsyd7a1DbXFMwA}{127.0.0.1}{127.0.0.1:9301},}, reason: zen-disco-node-join[{npjHq7P}{npjHq7PlTqqiB2bhXkIxqQ}{IhQPGv_rTsyd7a1DbXFMwA}{127.0.0.1}{127.0.0.1:9301}]

[2016-11-02T17:21:10,007][WARN ][o.e.d.z.ElectMasterService] [DiStpli] value for setting "discovery.zen.minimum_master_nodes" is too low. This can result in data loss! Please set it to at least a quorum of master-eligible nodes (current value: [-1], total number of master-eligible nodes used for publishing in this round: [2])

- 친절하게 quorum 구성 하라고 메시지도 보여 주내요.

저작자 표시 비영리 변경 금지
신고
크리에이티브 커먼즈 라이선스
Creative Commons License
Trackback 0 : Comment 0

티스토리 툴바