'ITWeb/개발일반'에 해당되는 글 490건

  1. 2017.11.24 [Java] Excutors 에서 제공하는 ExecutorService
  2. 2017.11.24 [Java] java.util.concurrent 관련 예제
  3. 2017.11.22 [Bash] for loop range and if condition
  4. 2017.11.18 [Spark] Spark installation on osx
  5. 2017.11.02 [Gradle] Dynamic version cache 설정하기
  6. 2017.10.31 [Groovy] foreach..break 테스트
  7. 2017.10.27 [Git] Merge request 전 commit log 정리
  8. 2017.10.27 [Gradle] Dependency filename 구하기
  9. 2017.10.27 [Git] Commit log 수정하기
  10. 2017.10.25 [Gradle] Use latest version on dependency jar.

[Java] Excutors 에서 제공하는 ExecutorService

ITWeb/개발일반 2017. 11. 24. 15:36

구글링 하기 귀찮아서 소소 코드에 있는 주석이랑 코드 가져 왔습니다.

/**
* Creates a thread pool that reuses a fixed number of threads
* operating off a shared unbounded queue. At any point, at most
* {@code nThreads} threads will be active processing tasks.
* If additional tasks are submitted when all threads are active,
* they will wait in the queue until a thread is available.
* If any thread terminates due to a failure during execution
* prior to shutdown, a new one will take its place if needed to
* execute subsequent tasks. The threads in the pool will exist
* until it is explicitly {@link ExecutorService#shutdown shutdown}.
*
* @param nThreads the number of threads in the pool
* @return the newly created thread pool
* @throws IllegalArgumentException if {@code nThreads <= 0}
*/
public static ExecutorService newFixedThreadPool(int nThreads) {
return new ThreadPoolExecutor(nThreads, nThreads,
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<Runnable>());
}
/**
* Creates a thread pool that maintains enough threads to support
* the given parallelism level, and may use multiple queues to
* reduce contention. The parallelism level corresponds to the
* maximum number of threads actively engaged in, or available to
* engage in, task processing. The actual number of threads may
* grow and shrink dynamically. A work-stealing pool makes no
* guarantees about the order in which submitted tasks are
* executed.
*
* @param parallelism the targeted parallelism level
* @return the newly created thread pool
* @throws IllegalArgumentException if {@code parallelism <= 0}
* @since 1.8
*/
public static ExecutorService newWorkStealingPool(int parallelism) {
return new ForkJoinPool
(parallelism,
ForkJoinPool.defaultForkJoinWorkerThreadFactory,
null, true);
}
/**
* Creates a work-stealing thread pool using all
* {@link Runtime#availableProcessors available processors}
* as its target parallelism level.
* @return the newly created thread pool
* @see #newWorkStealingPool(int)
* @since 1.8
*/
public static ExecutorService newWorkStealingPool() {
return new ForkJoinPool
(Runtime.getRuntime().availableProcessors(),
ForkJoinPool.defaultForkJoinWorkerThreadFactory,
null, true);
}
/**
* Creates a thread pool that reuses a fixed number of threads
* operating off a shared unbounded queue, using the provided
* ThreadFactory to create new threads when needed. At any point,
* at most {@code nThreads} threads will be active processing
* tasks. If additional tasks are submitted when all threads are
* active, they will wait in the queue until a thread is
* available. If any thread terminates due to a failure during
* execution prior to shutdown, a new one will take its place if
* needed to execute subsequent tasks. The threads in the pool will
* exist until it is explicitly {@link ExecutorService#shutdown
* shutdown}.
*
* @param nThreads the number of threads in the pool
* @param threadFactory the factory to use when creating new threads
* @return the newly created thread pool
* @throws NullPointerException if threadFactory is null
* @throws IllegalArgumentException if {@code nThreads <= 0}
*/
public static ExecutorService newFixedThreadPool(int nThreads, ThreadFactory threadFactory) {
return new ThreadPoolExecutor(nThreads, nThreads,
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<Runnable>(),
threadFactory);
}
/**
* Creates an Executor that uses a single worker thread operating
* off an unbounded queue. (Note however that if this single
* thread terminates due to a failure during execution prior to
* shutdown, a new one will take its place if needed to execute
* subsequent tasks.) Tasks are guaranteed to execute
* sequentially, and no more than one task will be active at any
* given time. Unlike the otherwise equivalent
* {@code newFixedThreadPool(1)} the returned executor is
* guaranteed not to be reconfigurable to use additional threads.
*
* @return the newly created single-threaded Executor
*/
public static ExecutorService newSingleThreadExecutor() {
return new FinalizableDelegatedExecutorService
(new ThreadPoolExecutor(1, 1,
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<Runnable>()));
}
/**
* Creates an Executor that uses a single worker thread operating
* off an unbounded queue, and uses the provided ThreadFactory to
* create a new thread when needed. Unlike the otherwise
* equivalent {@code newFixedThreadPool(1, threadFactory)} the
* returned executor is guaranteed not to be reconfigurable to use
* additional threads.
*
* @param threadFactory the factory to use when creating new
* threads
*
* @return the newly created single-threaded Executor
* @throws NullPointerException if threadFactory is null
*/
public static ExecutorService newSingleThreadExecutor(ThreadFactory threadFactory) {
return new FinalizableDelegatedExecutorService
(new ThreadPoolExecutor(1, 1,
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<Runnable>(),
threadFactory));
}


/**
* Creates a thread pool that creates new threads as needed, but
* will reuse previously constructed threads when they are
* available. These pools will typically improve the performance
* of programs that execute many short-lived asynchronous tasks.
* Calls to {@code execute} will reuse previously constructed
* threads if available. If no existing thread is available, a new
* thread will be created and added to the pool. Threads that have
* not been used for sixty seconds are terminated and removed from
* the cache. Thus, a pool that remains idle for long enough will
* not consume any resources. Note that pools with similar
* properties but different details (for example, timeout parameters)
* may be created using {@link ThreadPoolExecutor} constructors.
*
* @return the newly created thread pool
*/
public static ExecutorService newCachedThreadPool() {
return new ThreadPoolExecutor(0, Integer.MAX_VALUE,
60L, TimeUnit.SECONDS,
new SynchronousQueue<Runnable>());
}
/**
* Creates a thread pool that creates new threads as needed, but
* will reuse previously constructed threads when they are
* available, and uses the provided
* ThreadFactory to create new threads when needed.
* @param threadFactory the factory to use when creating new threads
* @return the newly created thread pool
* @throws NullPointerException if threadFactory is null
*/
public static ExecutorService newCachedThreadPool(ThreadFactory threadFactory) {
return new ThreadPoolExecutor(0, Integer.MAX_VALUE,
60L, TimeUnit.SECONDS,
new SynchronousQueue<Runnable>(),
threadFactory);
}
/**
* Creates a single-threaded executor that can schedule commands
* to run after a given delay, or to execute periodically.
* (Note however that if this single
* thread terminates due to a failure during execution prior to
* shutdown, a new one will take its place if needed to execute
* subsequent tasks.) Tasks are guaranteed to execute
* sequentially, and no more than one task will be active at any
* given time. Unlike the otherwise equivalent
* {@code newScheduledThreadPool(1)} the returned executor is
* guaranteed not to be reconfigurable to use additional threads.
* @return the newly created scheduled executor
*/
public static ScheduledExecutorService newSingleThreadScheduledExecutor() {
return new DelegatedScheduledExecutorService
(new ScheduledThreadPoolExecutor(1));
}
/**
* Creates a single-threaded executor that can schedule commands
* to run after a given delay, or to execute periodically. (Note
* however that if this single thread terminates due to a failure
* during execution prior to shutdown, a new one will take its
* place if needed to execute subsequent tasks.) Tasks are
* guaranteed to execute sequentially, and no more than one task
* will be active at any given time. Unlike the otherwise
* equivalent {@code newScheduledThreadPool(1, threadFactory)}
* the returned executor is guaranteed not to be reconfigurable to
* use additional threads.
* @param threadFactory the factory to use when creating new
* threads
* @return a newly created scheduled executor
* @throws NullPointerException if threadFactory is null
*/
public static ScheduledExecutorService newSingleThreadScheduledExecutor( ThreadFactory threadFactory) {
return new DelegatedScheduledExecutorService
(new ScheduledThreadPoolExecutor(1, threadFactory));
}
/**
* Creates a thread pool that can schedule commands to run after a
* given delay, or to execute periodically.
* @param corePoolSize the number of threads to keep in the pool,
* even if they are idle
* @return a newly created scheduled thread pool
* @throws IllegalArgumentException if {@code corePoolSize < 0}
*/
public static ScheduledExecutorService newScheduledThreadPool(int corePoolSize) {
return new ScheduledThreadPoolExecutor(corePoolSize);
}


/**
* Creates a thread pool that can schedule commands to run after a
* given delay, or to execute periodically.
* @param corePoolSize the number of threads to keep in the pool,
* even if they are idle
* @param threadFactory the factory to use when the executor
* creates a new thread
* @return a newly created scheduled thread pool
* @throws IllegalArgumentException if {@code corePoolSize < 0}
* @throws NullPointerException if threadFactory is null
*/
public static ScheduledExecutorService newScheduledThreadPool(
int corePoolSize, ThreadFactory threadFactory) {
return new ScheduledThreadPoolExecutor(corePoolSize, threadFactory);
}
/**
* Returns an object that delegates all defined {@link
* ExecutorService} methods to the given executor, but not any
* other methods that might otherwise be accessible using
* casts. This provides a way to safely "freeze" configuration and
* disallow tuning of a given concrete implementation.
* @param executor the underlying implementation
* @return an {@code ExecutorService} instance
* @throws NullPointerException if executor null
*/
public static ExecutorService unconfigurableExecutorService(ExecutorService executor) {
if (executor == null)
throw new NullPointerException();
return new DelegatedExecutorService(executor);
}
/**
* Returns an object that delegates all defined {@link
* ScheduledExecutorService} methods to the given executor, but
* not any other methods that might otherwise be accessible using
* casts. This provides a way to safely "freeze" configuration and
* disallow tuning of a given concrete implementation.
* @param executor the underlying implementation
* @return a {@code ScheduledExecutorService} instance
* @throws NullPointerException if executor null
*/
public static ScheduledExecutorService unconfigurableScheduledExecutorService( ScheduledExecutorService executor) {
if (executor == null)
throw new NullPointerException();
return new DelegatedScheduledExecutorService(executor);
}


  • newFixedThreadPool
    • 정해준 크기 만큼의 쓰레드를 생성하고 재사용 합니다. 명시적으로 shutdown() 하지 않는 한 쓰레드 중 하나가 종료 되면 다시 생성을 하게 됩니다.


  • newWorkStealingPool
    • 작업 순서에 대한 보장은 하지 않습니다, parallelism 수준에 따라 쓰레드를 충분히 지원 하지만 다중큐를 사용하는 것이 좋습니다. 쓰레드의 크기는 동적으로 늘었다 줄었다 합니다.


  • newSingleThreadExecutor
    • 쓰레드를 하나만 생성해서 사용합니다. 만약 종료 되면 다시 쓰레드가 생성이 되며 작업에 대한 연속성을 보장해 줍니다.


  • newCachedThreadPool
    • 필요한 만큼 쓰레드를 생성 하게 됩니다. 하지만 60초 동안 사용되지 않으면 풀에서 제거 됩니다.
    • 60초가 기본 설정 값 이며, 생성된 쓰레드는 재사용 됩니다.


  • newSingleThreadScheduledExecutor
    • 스케쥴링이 가능한 하나의 쓰레드를 생성 합니다. 스케쥴 기능을 빼고는 newSingleThreadExecutor 와 비슷 하다고 보시면 됩니다.


  • newScheduledThreadPool
    • 스케쥴링이 가능한 쓰레드 풀을 생성 합니다. 쓰레드가 idle 상태에 있더라도 종료 되거나 소멸 되지 않고 풀에 그대로 남아 있습니다.


:

[Java] java.util.concurrent 관련 예제

ITWeb/개발일반 2017. 11. 24. 15:03

구글링 하기 귀찮을 땐.

http://tutorials.jenkov.com/java-util-concurrent/index.html

http://winterbe.com/posts/2015/04/07/java8-concurrency-tutorial-thread-executor-examples/


그리고 java 8 관련해서는 위에 링크 중 아래 블로그 쥔장이 잘 정리해 둔것 같내요.

http://winterbe.com/java/

:

[Bash] for loop range and if condition

ITWeb/개발일반 2017. 11. 22. 11:16

이런 간단한것도 매번 생각이 나지 않아서 기록해 봅니다.


$ for i in {1..10}

do

s3cmd get s3://part-$i

if [ $i -gt 9 ]

then

break

fi

done


:

[Spark] Spark installation on osx

ITWeb/개발일반 2017. 11. 18. 21:54

기본 설치 입니다.

제가 brew install 보다는 직접 binary 받아서 설치 하는걸 더 선호해서 올려 봅니다.

[Spark installation on osx]

Ref. https://isaacchanghau.github.io/2017/06/28/Spark-Installation-on-Mac-OS-X/


1. scala

https://www.scala-lang.org/download/


$ tar -xvzf scala-2.12.4.tgz

$ vi .bash_profile

export SCALA_HOME=/Users/henry/Work/apps/scala-2.12.4

PATH=$SCALA_HOME/bin:$PATH

2. spark

https://spark.apache.org/downloads.html


$ tar -xvzf spark-2.2.0-bin-hadoop2.7.tgz

$ vi .bash_profile

export SPARK_HOME=/Users/henry/Work/apps/spark-2.2.0-bin-hadoop2.7

PATH=$SPARK_HOME/bin:$PATH

$ cd /Users/henry/Work/apps/spark-2.2.0-bin-hadoop2.7

$ cp spark-env.sh.template spark-env.sh

$ vim spark-env.sh

export SCALA_HOME=/Users/henry/Work/apps/scala-2.12.4

export SPARK_MASTER_IP=localhost

export SPARK_WORKER_MEMORY=1g

$ spark-shell


scala 와 spark 이 설치가 되어야 합니다.

보시면 아시겠지만 .bash_profile 에 환경 설정 해주시고 실행하시면 됩니다.

PATH=$SCALA_HOME/bin:$SPARK_HOME/bin:$PATH

$ source .bash_profile

$ spark-shell

Spark context Web UI available at http://127.0.0.1:4040

Spark context available as 'sc' (master = local[*], app id = local-1511009262380).

Spark session available as 'spark'.

Welcome to

      ____              __

     / __/__  ___ _____/ /__

    _\ \/ _ \/ _ `/ __/  '_/

   /___/ .__/\_,_/_/ /_/\_\   version 2.2.0

      /_/


Using Scala version 2.11.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_72)

Type in expressions to have them evaluated.

Type :help for more information.


scala>


:

[Gradle] Dynamic version cache 설정하기

ITWeb/개발일반 2017. 11. 2. 21:45

지난 글과 함께 보시면 좋습니다.


[지난 글]

[Gradle] Dependency filename 구하기

[Gradle] Use latest version on dependency jar.


[참고문서]

https://docs.gradle.org/current/dsl/org.gradle.api.artifacts.ResolutionStrategy.html


[설정]

configurations {

dependencyFn

}


configurations.dependencyFn {

resolutionStrategy.cacheDynamicVersionsFor 10, 'minutes'

}


dependencies {

dependencyFn group: "org.apaceh.lucene.analysis.arirang", name: "arirang-dictionary", \

version: "+", changing: true

}


:

[Groovy] foreach..break 테스트

ITWeb/개발일반 2017. 10. 31. 16:03

[foreach.groovy]

def list = ["A", "B", "C", "D"]


list.each {

  println it


  if ( it.indexOf("B") > -1 ) return true

}

println "---------------"

list.any {

  println it


  if ( it.indexOf("B") > -1 ) return true

}


[Result]

A

B

C

D

---------------

A

B


:

[Git] Merge request 전 commit log 정리

ITWeb/개발일반 2017. 10. 27. 17:19

master 브랜치를 가지고 있고 work 브랜치와 merge request 브랜치를 가지고 정리해 보겠습니다.


1. master 브랜치에서 work 브랜치를 하나 만듭니다.

2. work 브랜치에서 열심히 개발을 합니다.

3. master 브랜치에서 merge request 용 브랜치를 하나 만듭니다.

4. merge request checkout 상태에서 work 브랜치를 merge 합니다.

$ git merge WORK_BRANCH --squash

5. 특별히 conflict 난게 없으면 commit 합니다.

$ git commit -a -m 'simple commit message'

6. commit 이 잘 되었으면 push 합니다.

$ git push origin MERGE_REQ_BRANCH

7. 마지막으로 merge request 를 보내시면 됩니다.


:

[Gradle] Dependency filename 구하기

ITWeb/개발일반 2017. 10. 27. 09:26

차분히 Gradle DSL 을 읽어 보는 것도 좋습니다.

[참고링크]

https://docs.gradle.org/current/dsl/index.html


Gradle 의 configurations 설정을 통해서 구해 보겠습니다.

FileTree 를 이용해서도 할 수 있습니다.

[build.gradle]

configurations {

dependencyFn

}


dependencies {

dependencyFn group: "org.apaceh.lucene.analysis.arirang", name: "arirang-dictionary", \

version: "+", changing: true

}


task requiredResourceFilename {

configurations.dependencyFn.files.each {

if ( it.getName().indexOf("arirang-dictionary") > -1 ) {

arirangDicFilename = it.getName().tokenize('-').last()


println arirangDicFilename

}

}

}


:

[Git] Commit log 수정하기

ITWeb/개발일반 2017. 10. 27. 09:04

가끔 사용합니다.

# 먼저 git log 부터 확인 합니다.

$ git log


# head 부터 2개의 commit 로그를 보여 줍니다.

# 여기서 수정하고 싶은 log 에 pick 을 edit 로 수정 합니다.

$ git rebase -i HEAD~2


# 저장하고 나왔으면 아래 명령어를 차례로 실행 합니다.

$ git commit --amend

$ git commit --continue


# 이건 head commit log 를 바로 수정 할때 사용합니다.

$ git commit --amend


:

[Gradle] Use latest version on dependency jar.

ITWeb/개발일반 2017. 10. 25. 17:36

Gradle dependency jar 에 대한 최신 정보 반영을 위해서는 아래와 같이 수정 하면 됩니다.


ASIS)

compile "com.google.api.grpc:proto-google-common-protos:0.1.9"

drivers group: "com.google.api.grpc", name: "cproto-google-common-protos", version: "0.1.9", changing: true


TOBE)

compile "com.google.api.grpc:proto-google-common-protos:+"

drivers group: "com.google.api.grpc", name: "cproto-google-common-protos", version: "+", changing: true


항상 최신 버전을 사용하고 싶으실때 쓰시면 좋을 듯 합니다.


참고문서)

https://docs.gradle.org/2.11/userguide/dependency_management.html#sec:dependency_resolution

23.7. How dependency resolution works

https://docs.gradle.org/2.11/userguide/artifact_dependencies_tutorial.html#N1059B

7.2. Declaring your dependencies


: