'Scala'에 해당되는 글 1건

  1. 2017.11.18 [Spark] Spark installation on osx

[Spark] Spark installation on osx

ITWeb/개발일반 2017. 11. 18. 21:54

기본 설치 입니다.

제가 brew install 보다는 직접 binary 받아서 설치 하는걸 더 선호해서 올려 봅니다.

[Spark installation on osx]

Ref. https://isaacchanghau.github.io/2017/06/28/Spark-Installation-on-Mac-OS-X/


1. scala

https://www.scala-lang.org/download/


$ tar -xvzf scala-2.12.4.tgz

$ vi .bash_profile

export SCALA_HOME=/Users/henry/Work/apps/scala-2.12.4

PATH=$SCALA_HOME/bin:$PATH

2. spark

https://spark.apache.org/downloads.html


$ tar -xvzf spark-2.2.0-bin-hadoop2.7.tgz

$ vi .bash_profile

export SPARK_HOME=/Users/henry/Work/apps/spark-2.2.0-bin-hadoop2.7

PATH=$SPARK_HOME/bin:$PATH

$ cd /Users/henry/Work/apps/spark-2.2.0-bin-hadoop2.7

$ cp spark-env.sh.template spark-env.sh

$ vim spark-env.sh

export SCALA_HOME=/Users/henry/Work/apps/scala-2.12.4

export SPARK_MASTER_IP=localhost

export SPARK_WORKER_MEMORY=1g

$ spark-shell


scala 와 spark 이 설치가 되어야 합니다.

보시면 아시겠지만 .bash_profile 에 환경 설정 해주시고 실행하시면 됩니다.

PATH=$SCALA_HOME/bin:$SPARK_HOME/bin:$PATH

$ source .bash_profile

$ spark-shell

Spark context Web UI available at http://127.0.0.1:4040

Spark context available as 'sc' (master = local[*], app id = local-1511009262380).

Spark session available as 'spark'.

Welcome to

      ____              __

     / __/__  ___ _____/ /__

    _\ \/ _ \/ _ `/ __/  '_/

   /___/ .__/\_,_/_/ /_/\_\   version 2.2.0

      /_/


Using Scala version 2.11.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_72)

Type in expressions to have them evaluated.

Type :help for more information.


scala>


: