'java'에 해당되는 글 21건

  1. 2018.04.03 [Elasticsearch] JSON 스트링으로 질의요청
  2. 2017.12.22 [GC] 관련 링크 모음
  3. 2017.12.12 [Java] Executors 로 간단 multithread 테스트
  4. 2017.11.24 [Java] Excutors 에서 제공하는 ExecutorService
  5. 2017.11.24 [Java] java.util.concurrent 관련 예제
  6. 2017.04.18 [Java] Array Value 존재 유무 확인하기
  7. 2017.01.16 [Intellij] intellij-java-google-style.xml 적용하기
  8. 2017.01.09 [Gradle] Gradle project 맛보기.
  9. 2016.12.19 [웹 크롤러] crawler4j 링크
  10. 2016.11.09 [Java] String[] to List, List dedup. List to String comma

[Elasticsearch] JSON 스트링으로 질의요청

Elastic/Elasticsearch 2018.04.03 13:41

예전에도 JSON 스트링을 가지고 setSource() 함수로 질의 했었는데요.

필요하신 분들이 계신것 같아 쉬운 샘플 코드 올려 봅니다.


[Source Code]

package org.elasticsearch.action.search;

import org.elasticsearch.client.Client;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.index.query.TermQueryBuilder;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.elasticsearch.transport.client.PreBuiltTransportClient;

public class HelloSearchSourceBuilder {

public static void main (String[] args) {
Client client;
client = new PreBuiltTransportClient(Settings.EMPTY);

SearchRequestBuilder searchRequestBuilder = client.prepareSearch();
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
TermQueryBuilder termQueryBuilder = QueryBuilders.termQuery("field1","value1");

System.out.println(termQueryBuilder.toString());

searchSourceBuilder.query(termQueryBuilder).from(0).size(10);

System.out.println(searchSourceBuilder.toString());

searchRequestBuilder.setSource(searchSourceBuilder);

System.out.println(searchRequestBuilder.toString());
}
}


[Output]

[2018-04-03T13:36:38,466][INFO ][o.e.p.PluginsService     ] [_client_] no modules loaded

[2018-04-03T13:36:38,470][INFO ][o.e.p.PluginsService     ] [_client_] loaded plugin [org.elasticsearch.index.reindex.ReindexPlugin]

[2018-04-03T13:36:38,470][INFO ][o.e.p.PluginsService     ] [_client_] loaded plugin [org.elasticsearch.join.ParentJoinPlugin]

[2018-04-03T13:36:38,470][INFO ][o.e.p.PluginsService     ] [_client_] loaded plugin [org.elasticsearch.percolator.PercolatorPlugin]

[2018-04-03T13:36:38,470][INFO ][o.e.p.PluginsService     ] [_client_] loaded plugin [org.elasticsearch.script.mustache.MustachePlugin]

[2018-04-03T13:36:38,470][INFO ][o.e.p.PluginsService     ] [_client_] loaded plugin [org.elasticsearch.transport.Netty4Plugin]

{

  "term" : {

    "field1" : {

      "value" : "value1",

      "boost" : 1.0

    }

  }

}

{"from":0,"size":10,"query":{"term":{"field1":{"value":"value1","boost":1.0}}}}

{"from":0,"size":10,"query":{"term":{"field1":{"value":"value1","boost":1.0}}}}


Process finished with exit code 0



Trackback 0 : Comment 0

[GC] 관련 링크 모음

ITWeb/개발일반 2017.12.22 12:44

[GC tuning 글 모음]

https://blogs.apache.org/hbase/entry/tuning_g1gc_for_your_hbase


https://logonjava.blogspot.kr/2015/08/java-g1-gc-full-gc.html

http://product.hubspot.com/blog/g1gc-fundamentals-lessons-from-taming-garbage-collection

http://xmlandmore.blogspot.com/2014/09/g1-gc-tuning-mixed-garbage-collections.html


https://docs.oracle.com/javase/8/docs/technotes/guides/vm/gctuning/

http://www.oracle.com/technetwork/articles/java/g1gc-1984535.html

http://www.oracle.com/technetwork/java/javase/tech/vmoptions-jsp-140102.html#G1Options


http://d2.naver.com/helloworld/1329

http://d2.naver.com/helloworld/4717

http://d2.naver.com/helloworld/6043

http://d2.naver.com/helloworld/37111

http://d2.naver.com/helloworld/329631


tags : g1gc, gc, java, JVM, tuning
Trackback 0 : Comment 0

[Java] Executors 로 간단 multithread 테스트

ITWeb/개발일반 2017.12.12 11:54

이전 글의 참고 문서들을 먼저 보시면 좋습니다.

http://jjeong.tistory.com/1296

package hello.executors;

import java.util.HashSet;
import java.util.List;
import java.util.Set;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;

public class HelloExecutors {

public static class HelloCallableThread implements Callable<Integer> {
int input;

public HelloCallableThread(int input) {
this.input = input;
}

@Override
public Integer call() throws Exception {
return input + 1;
}
}

public static void main(String[] args) throws Exception {
ExecutorService executorService = Executors.newFixedThreadPool(4);
Set<Callable<Integer>> callables = new HashSet<Callable<Integer>>();

callables.add(new HelloCallableThread(1));
callables.add(new HelloCallableThread(2));
callables.add(new HelloCallableThread(3));
callables.add(new HelloCallableThread(4));

List<Future<Integer>> futures = executorService.invokeAll(callables);

for(Future<Integer> future : futures){
System.out.println("future.get = " + future.get());
}

executorService.shutdown();
executorService.awaitTermination(1, TimeUnit.SECONDS);
}
}


Trackback 0 : Comment 0

[Java] Excutors 에서 제공하는 ExecutorService

ITWeb/개발일반 2017.11.24 15:36

구글링 하기 귀찮아서 소소 코드에 있는 주석이랑 코드 가져 왔습니다.

/**
* Creates a thread pool that reuses a fixed number of threads
* operating off a shared unbounded queue. At any point, at most
* {@code nThreads} threads will be active processing tasks.
* If additional tasks are submitted when all threads are active,
* they will wait in the queue until a thread is available.
* If any thread terminates due to a failure during execution
* prior to shutdown, a new one will take its place if needed to
* execute subsequent tasks. The threads in the pool will exist
* until it is explicitly {@link ExecutorService#shutdown shutdown}.
*
* @param nThreads the number of threads in the pool
* @return the newly created thread pool
* @throws IllegalArgumentException if {@code nThreads <= 0}
*/
public static ExecutorService newFixedThreadPool(int nThreads) {
return new ThreadPoolExecutor(nThreads, nThreads,
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<Runnable>());
}
/**
* Creates a thread pool that maintains enough threads to support
* the given parallelism level, and may use multiple queues to
* reduce contention. The parallelism level corresponds to the
* maximum number of threads actively engaged in, or available to
* engage in, task processing. The actual number of threads may
* grow and shrink dynamically. A work-stealing pool makes no
* guarantees about the order in which submitted tasks are
* executed.
*
* @param parallelism the targeted parallelism level
* @return the newly created thread pool
* @throws IllegalArgumentException if {@code parallelism <= 0}
* @since 1.8
*/
public static ExecutorService newWorkStealingPool(int parallelism) {
return new ForkJoinPool
(parallelism,
ForkJoinPool.defaultForkJoinWorkerThreadFactory,
null, true);
}
/**
* Creates a work-stealing thread pool using all
* {@link Runtime#availableProcessors available processors}
* as its target parallelism level.
* @return the newly created thread pool
* @see #newWorkStealingPool(int)
* @since 1.8
*/
public static ExecutorService newWorkStealingPool() {
return new ForkJoinPool
(Runtime.getRuntime().availableProcessors(),
ForkJoinPool.defaultForkJoinWorkerThreadFactory,
null, true);
}
/**
* Creates a thread pool that reuses a fixed number of threads
* operating off a shared unbounded queue, using the provided
* ThreadFactory to create new threads when needed. At any point,
* at most {@code nThreads} threads will be active processing
* tasks. If additional tasks are submitted when all threads are
* active, they will wait in the queue until a thread is
* available. If any thread terminates due to a failure during
* execution prior to shutdown, a new one will take its place if
* needed to execute subsequent tasks. The threads in the pool will
* exist until it is explicitly {@link ExecutorService#shutdown
* shutdown}.
*
* @param nThreads the number of threads in the pool
* @param threadFactory the factory to use when creating new threads
* @return the newly created thread pool
* @throws NullPointerException if threadFactory is null
* @throws IllegalArgumentException if {@code nThreads <= 0}
*/
public static ExecutorService newFixedThreadPool(int nThreads, ThreadFactory threadFactory) {
return new ThreadPoolExecutor(nThreads, nThreads,
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<Runnable>(),
threadFactory);
}
/**
* Creates an Executor that uses a single worker thread operating
* off an unbounded queue. (Note however that if this single
* thread terminates due to a failure during execution prior to
* shutdown, a new one will take its place if needed to execute
* subsequent tasks.) Tasks are guaranteed to execute
* sequentially, and no more than one task will be active at any
* given time. Unlike the otherwise equivalent
* {@code newFixedThreadPool(1)} the returned executor is
* guaranteed not to be reconfigurable to use additional threads.
*
* @return the newly created single-threaded Executor
*/
public static ExecutorService newSingleThreadExecutor() {
return new FinalizableDelegatedExecutorService
(new ThreadPoolExecutor(1, 1,
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<Runnable>()));
}
/**
* Creates an Executor that uses a single worker thread operating
* off an unbounded queue, and uses the provided ThreadFactory to
* create a new thread when needed. Unlike the otherwise
* equivalent {@code newFixedThreadPool(1, threadFactory)} the
* returned executor is guaranteed not to be reconfigurable to use
* additional threads.
*
* @param threadFactory the factory to use when creating new
* threads
*
* @return the newly created single-threaded Executor
* @throws NullPointerException if threadFactory is null
*/
public static ExecutorService newSingleThreadExecutor(ThreadFactory threadFactory) {
return new FinalizableDelegatedExecutorService
(new ThreadPoolExecutor(1, 1,
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<Runnable>(),
threadFactory));
}


/**
* Creates a thread pool that creates new threads as needed, but
* will reuse previously constructed threads when they are
* available. These pools will typically improve the performance
* of programs that execute many short-lived asynchronous tasks.
* Calls to {@code execute} will reuse previously constructed
* threads if available. If no existing thread is available, a new
* thread will be created and added to the pool. Threads that have
* not been used for sixty seconds are terminated and removed from
* the cache. Thus, a pool that remains idle for long enough will
* not consume any resources. Note that pools with similar
* properties but different details (for example, timeout parameters)
* may be created using {@link ThreadPoolExecutor} constructors.
*
* @return the newly created thread pool
*/
public static ExecutorService newCachedThreadPool() {
return new ThreadPoolExecutor(0, Integer.MAX_VALUE,
60L, TimeUnit.SECONDS,
new SynchronousQueue<Runnable>());
}
/**
* Creates a thread pool that creates new threads as needed, but
* will reuse previously constructed threads when they are
* available, and uses the provided
* ThreadFactory to create new threads when needed.
* @param threadFactory the factory to use when creating new threads
* @return the newly created thread pool
* @throws NullPointerException if threadFactory is null
*/
public static ExecutorService newCachedThreadPool(ThreadFactory threadFactory) {
return new ThreadPoolExecutor(0, Integer.MAX_VALUE,
60L, TimeUnit.SECONDS,
new SynchronousQueue<Runnable>(),
threadFactory);
}
/**
* Creates a single-threaded executor that can schedule commands
* to run after a given delay, or to execute periodically.
* (Note however that if this single
* thread terminates due to a failure during execution prior to
* shutdown, a new one will take its place if needed to execute
* subsequent tasks.) Tasks are guaranteed to execute
* sequentially, and no more than one task will be active at any
* given time. Unlike the otherwise equivalent
* {@code newScheduledThreadPool(1)} the returned executor is
* guaranteed not to be reconfigurable to use additional threads.
* @return the newly created scheduled executor
*/
public static ScheduledExecutorService newSingleThreadScheduledExecutor() {
return new DelegatedScheduledExecutorService
(new ScheduledThreadPoolExecutor(1));
}
/**
* Creates a single-threaded executor that can schedule commands
* to run after a given delay, or to execute periodically. (Note
* however that if this single thread terminates due to a failure
* during execution prior to shutdown, a new one will take its
* place if needed to execute subsequent tasks.) Tasks are
* guaranteed to execute sequentially, and no more than one task
* will be active at any given time. Unlike the otherwise
* equivalent {@code newScheduledThreadPool(1, threadFactory)}
* the returned executor is guaranteed not to be reconfigurable to
* use additional threads.
* @param threadFactory the factory to use when creating new
* threads
* @return a newly created scheduled executor
* @throws NullPointerException if threadFactory is null
*/
public static ScheduledExecutorService newSingleThreadScheduledExecutor( ThreadFactory threadFactory) {
return new DelegatedScheduledExecutorService
(new ScheduledThreadPoolExecutor(1, threadFactory));
}
/**
* Creates a thread pool that can schedule commands to run after a
* given delay, or to execute periodically.
* @param corePoolSize the number of threads to keep in the pool,
* even if they are idle
* @return a newly created scheduled thread pool
* @throws IllegalArgumentException if {@code corePoolSize < 0}
*/
public static ScheduledExecutorService newScheduledThreadPool(int corePoolSize) {
return new ScheduledThreadPoolExecutor(corePoolSize);
}


/**
* Creates a thread pool that can schedule commands to run after a
* given delay, or to execute periodically.
* @param corePoolSize the number of threads to keep in the pool,
* even if they are idle
* @param threadFactory the factory to use when the executor
* creates a new thread
* @return a newly created scheduled thread pool
* @throws IllegalArgumentException if {@code corePoolSize < 0}
* @throws NullPointerException if threadFactory is null
*/
public static ScheduledExecutorService newScheduledThreadPool(
int corePoolSize, ThreadFactory threadFactory) {
return new ScheduledThreadPoolExecutor(corePoolSize, threadFactory);
}
/**
* Returns an object that delegates all defined {@link
* ExecutorService} methods to the given executor, but not any
* other methods that might otherwise be accessible using
* casts. This provides a way to safely "freeze" configuration and
* disallow tuning of a given concrete implementation.
* @param executor the underlying implementation
* @return an {@code ExecutorService} instance
* @throws NullPointerException if executor null
*/
public static ExecutorService unconfigurableExecutorService(ExecutorService executor) {
if (executor == null)
throw new NullPointerException();
return new DelegatedExecutorService(executor);
}
/**
* Returns an object that delegates all defined {@link
* ScheduledExecutorService} methods to the given executor, but
* not any other methods that might otherwise be accessible using
* casts. This provides a way to safely "freeze" configuration and
* disallow tuning of a given concrete implementation.
* @param executor the underlying implementation
* @return a {@code ScheduledExecutorService} instance
* @throws NullPointerException if executor null
*/
public static ScheduledExecutorService unconfigurableScheduledExecutorService( ScheduledExecutorService executor) {
if (executor == null)
throw new NullPointerException();
return new DelegatedScheduledExecutorService(executor);
}


  • newFixedThreadPool
    • 정해준 크기 만큼의 쓰레드를 생성하고 재사용 합니다. 명시적으로 shutdown() 하지 않는 한 쓰레드 중 하나가 종료 되면 다시 생성을 하게 됩니다.


  • newWorkStealingPool
    • 작업 순서에 대한 보장은 하지 않습니다, parallelism 수준에 따라 쓰레드를 충분히 지원 하지만 다중큐를 사용하는 것이 좋습니다. 쓰레드의 크기는 동적으로 늘었다 줄었다 합니다.


  • newSingleThreadExecutor
    • 쓰레드를 하나만 생성해서 사용합니다. 만약 종료 되면 다시 쓰레드가 생성이 되며 작업에 대한 연속성을 보장해 줍니다.


  • newCachedThreadPool
    • 필요한 만큼 쓰레드를 생성 하게 됩니다. 하지만 60초 동안 사용되지 않으면 풀에서 제거 됩니다.
    • 60초가 기본 설정 값 이며, 생성된 쓰레드는 재사용 됩니다.


  • newSingleThreadScheduledExecutor
    • 스케쥴링이 가능한 하나의 쓰레드를 생성 합니다. 스케쥴 기능을 빼고는 newSingleThreadExecutor 와 비슷 하다고 보시면 됩니다.


  • newScheduledThreadPool
    • 스케쥴링이 가능한 쓰레드 풀을 생성 합니다. 쓰레드가 idle 상태에 있더라도 종료 되거나 소멸 되지 않고 풀에 그대로 남아 있습니다.


Trackback 0 : Comment 0

[Java] java.util.concurrent 관련 예제

ITWeb/개발일반 2017.11.24 15:03

구글링 하기 귀찮을 땐.

http://tutorials.jenkov.com/java-util-concurrent/index.html

http://winterbe.com/posts/2015/04/07/java8-concurrency-tutorial-thread-executor-examples/


그리고 java 8 관련해서는 위에 링크 중 아래 블로그 쥔장이 잘 정리해 둔것 같내요.

http://winterbe.com/java/

Trackback 0 : Comment 0

[Java] Array Value 존재 유무 확인하기

ITWeb/개발일반 2017.04.18 10:38

생각 나지 않을때가 많아서 퍼왔습니다.


Originanl Source)

http://www.programcreek.com/2014/04/check-if-array-contains-a-value-java/


public static boolean useList(String[] arr, String targetValue) {

  return Arrays.asList(arr).contains(targetValue);

}


public static boolean useSet(String[] arr, String targetValue) {

  Set<String> set = new HashSet<String>(Arrays.asList(arr));

  return set.contains(targetValue);

}


public static boolean useLoop(String[] arr, String targetValue) {

  for(String s: arr){

    if(s.equals(targetValue))

      return true;

  }

  return false;

}


public static boolean useArraysBinarySearch(String[] arr, String targetValue) { 

  int a =  Arrays.binarySearch(arr, targetValue);

  if(a > 0)

    return true;

  else

    return false;

}


binary search 가 제일 성능이 좋습니다.

단, 위 코드에서 빠진게 있다면 Arrays.sort(arr) 를 먼저 하고 하셔야 된다는 것입니다.


Arrays.sort(arr);

int a = Arrays.binarySearch(arr, targetValue);

Trackback 0 : Comment 0

[Intellij] intellij-java-google-style.xml 적용하기

ITWeb/개발일반 2017.01.16 15:42

각자 필요한 코드 스타일은 아래서 다운로드 받으시면 됩니다.


[구글 코드 스타일]

https://github.com/google/styleguide


[Intellij 적용하기 - mac]

# 저는 intellij 2016.3 사용중입니다.

# 다운로드 받으신 intellij-java-google-style.xml 파일을 아래 경로로 복사해서 넣습니다.


$ cd ~/Library/Preferences/IntelliJIdea2016.3/codestyles


[Intellij 에서 import 하기]

Preferences -> Editor -> Code Style -> Manage Button -> Import


# 그러나 파일을 먼저 복사해 넣었기 때문에 import 하지 않으셔도 scheme 이 정상적으로 나옵니다.

# Scheme 을 GoogleStyle 로 변경 하시면 끝납니다.


Trackback 0 : Comment 0

[Gradle] Gradle project 맛보기.

ITWeb/개발일반 2017.01.09 13:46

Intellij 를 이용한 gradle project 를 생성하는 아주 초보적인 내용입니다.

개인적인 생각으로는 single project 이고 maven 에 익숙하시면 그냥 maven project 로 개발 하시는게 편하실 수 있습니다.


Step 0)

- Gradle 설치가 되어 있어야 합니다.

- Java 설치가 되어 있어야 합니다.

- Path 설정이 되어 있어야 합니다.


Step 1)

New -> Project -> Gradle 선택을 하시면 됩니다.


Step 2)

maven project 생성 할 때와 동일하게 GroupId, ArtifactId, Version 정보를 등록 하면 됩니다.


Next 버튼을 누르고 누르고 나면 기본 Gradle Project 가 생성이 됩니다.


Step 3) build.gradle & settings.gradle

기본적인 빌드 환경과 정보들을 작성 하게 됩니다.

maven 의 pom.xml 과 비슷하다고 보시면 될 것 같습니다.


Step 4) module 추가하기

해당 프로젝트에서 new -> module 하시면 됩니다.

동일하게 gradle project 으로 해서 추가했습니다.


Step 5) 기본 java application directory 생성

src/main/java

src/main/resources

src/test/java

src/test/resources


Step 6) Java main class 생성

public class HelloWorld {

public static void main (String[] args) {
System.out.println("Hello World!!");
}
}


Step 7) Build & Run

$ gradle build


$ java -jar ./helloworld/build/libs/helloworld-1.0-SNAPSHOT.jar HelloWorld

./helloworld/build/libs/helloworld-1.0-SNAPSHOT.jar에 기본 Manifest 속성이 없습니다.


# manifest 속성을 선언 안해서 그렇습니다.
해당 프로젝트의 build.gradle 에 선언 하면 됩니다.
excutable jar 생성 하는 거랑 같은 거라고 보시면 됩니다.
jar {
manifest {
attributes 'Main-Class': 'org.jjeong93.hello.HelloWorld'
}
}

$ java -jar ./helloworld/build/libs/helloworld-1.0-SNAPSHOT.jar HelloWorld

Hello World!!


일반적인 Java application 만드는 걸 예제로 보여 드렸습니다.
저는 기존 maven project 를 gradle multi project 로 마이그레이션 하려고 합니다.
dependency 설정 하는 번거로움이 좀 있기는 하지만 마이그레이션 방법을 제공 하고 있으니 참고 하면 될 것 같습니다.

[참고문서]


tags : gradle, java, maven
Trackback 0 : Comment 0

[웹 크롤러] crawler4j 링크

ITWeb/개발일반 2016.12.19 13:17

웹 크롤러가 많이 있습니다.

그 중에서 그냥 java 로 쉽게 구현 할 수 있는 crawler4j 링크 공유해 봅니다.


https://github.com/yasserg/crawler4j


샘플 코드는 그냥 위 링크에 올라와 있는 코드 그대로 테스트 해보시면 동작 잘 합니다.

robots.txt 에러가 발생 하는 부분이 있을 경우 아래 설정 추가해 주시면 됩니다.


RobotstxtConfig robotstxtConfig = new RobotstxtConfig();
robotstxtConfig.setEnabled(false);


다른 크롤러가 궁금하신 분들은 아래 구글링 링크 참고하세요.

https://www.google.co.kr/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#newwindow=1&q=web+crawler+open+source


Trackback 0 : Comment 0

[Java] String[] to List, List dedup. List to String comma

ITWeb/개발일반 2016.11.09 12:42

이젠 뭘 해도 기억력이 따라오질 못합니다. ㅠ.ㅠ

복습을 위해서.


Stirng[] to List)

String[] keywords = {"기어", "s3", "기어"};

List<String> lists = Arrays.asList(keywords);


List DeDup)

HashSet<String> set = new HashSet<>(lists);

List<String> result = new ArrayList<>(set);


List to String comma)

StringUtils.join(result, ',');


tags : dedup, java, list, string[]
Trackback 0 : Comment 0