In Elasticsearch, this lightweight process of writing and opening a new segment is called a refresh. By default, every shard is refreshed automatically once every second. This is why we say that Elasticsearch has near real-time search: document changes are not visible to search immediately, but will become visible within one second.

The action of performing a commit and truncating the translog is known in Elasticsearch as a flush. Shards are flushed automatically every 30 minutes, or when the translog becomes too big. See thetranslog documentation for settings that can be used to control these thresholds.

The optimize API is best described as the forced merge API. It forces a shard to be merged down to the number of segments specified in the max_num_segments parameter. The intention is to reduce the number of segments (usually to 1) in order to speed up search performance.

new OptimizeRequest()

    .indices(indice)

    .flush(true)

    .onlyExpungeDeletes(false)

    .refresh(true)

    .waitForMerge(true)

    .operationThreading(BroadcastOperationThreading.THREAD_PER_SHARD)

    .maxNumSegments(1)

curl -XPOST 'http://localhost:9200/indice/_optimize?only_expunge_deletes=false&max_num_segments=1&refresh=true&flush=true&wait_for_merge=true'

curl -XPOST 'http://localhost:9200/indice/_close'

curl -XPOST 'http://localhost:9200/indice/_open'

client.admin()

    .indices()

    .close(new CloseIndexRequest("indice"));

client.admin()

    .indices()

    .close(new OpenIndexRequest("indice"));

http://www.elasticsearch.org/guide/reference/api/admin-indices-optimize/

http://www.elasticsearch.org/guide/reference/api/admin-indices-open-close/

본 문서는 개인적인 테스트와 elasticsearch.org 그리고 community 등을 참고해서 작성된 것이며,

정보 교환이 목적입니다.

잘못된 부분에 대해서는 지적 부탁 드립니다.

(예시 코드는 성능 및 보안 검증이 되지 않았습니다.)

response = builder.setIndices("blog")

.setMaxNumSegments(1) // full optimize 를 위해서는 1, 기본은 설정을 하지 않으면 simply checking 을 

.setOnlyExpungeDeletes(false) // 기본 false

.setWaitForMerge(true)

.setOperationThreading(BroadcastOperationThreading.THREAD_PER_SHARD)

.execute()

.actionGet();

log.debug("{}", response.getSuccessfulShards());

http://blog.bugsense.com/post/35580279634/indexing-bigdata-with-elasticsearch

http://www.slideshare.net/kucrafal/scaling-massive-elastic-search-clusters-rafa-ku-sematext

Page 27.

    index.cache.field.max_size

    index.cache.field.expire

    index.cache.field.type: soft

Page 30.

    JVM Optimization

    -XX:+UseParNewGC

    -XX:+UseConcMarkSweepGC

    -XX:+CMSParallelRemarkEnabled

https://gist.github.com/deverton/2970285

JVM Swap 방지

    bootstrap.mlockall: true

http://stackoverflow.com/questions/13757398/elasticsearch-bulk-indexing-gets-slower-over-time-with-constant-number-of-indexe    

    Set the ES_HEAP_SIZE environment variable so that the JVM uses the same value for minimum and maximum memory. Configuring the JVM to have different minimum and maximum values means that each time the JVM needs additional memory (up to the maximum), it will block the Java process to allocate it. Combined with the old Java version, this explains the pauses that our nodes exhibited when introduced to higher load and continuous memory allocation when they were opened up to public searches. The elasticsearch team recommends a setting of 50% of system RAM.    

http://www.elasticsearch.org/guide/reference/index-modules/store.html

        using  index.store.type: mmapfs

http://jprante.github.com/2012/11/28/Elasticsearch-Java-Virtual-Machine-settings-explained.html