Cloudera CDH/CDP 및 Hadoop EcoSystem, Semantic IoT등의 개발/운영 기술을 정리합니다. gooper@gooper.com로 문의 주세요.
Setting up RStudio Server to run with Apache Spark
https://markobigdata.com/2016/03/27/setting-up-rstudio-server-to-run-with-apache-spark/
Zeppelin supports Spark, PySpark, Spark R, Spark SQL with dependency loader.
Python is supported with Matplotlib, Conda, Pandas SQL and PySpark integrations.
Jupyter kernels
Here is a list of available kernels. If you are writing your own kernel, feel free to add it to the table!
| Name | Jupyter/IPython Version | Language(s) Version | 3rd party dependencies | Example Notebooks | Notes |
|---|---|---|---|---|---|
| Jupyter 4.0 | Fortran 2008/2015 | GFortran >= 7.1, , MPICH >= 3.2 | Demo, Binder demo | Docker image | |
| Jupyter >=4.0 | Pyspark (Python 2 & 3), Spark (Scala), SparkR (R) | Notebooks, | This kernels are implemented via the magics machinery of the ipython kernel to use Spark via Livy | ||
| Jupyter 4.0 | python >= 3.3 | SAS 9.4 or higher | |||
| Jupyter 4.0 | python 2.7, >= 3.3 | pyzmq | |||
| julia >= 0.3 | |||||
| IHaskell | ghc >= 7.6 | ||||
| ruby >= 2.1 | |||||
| nodejs >= 0.10 | |||||
| coffeescript >= 1.7 | |||||
| Jupyter 4.0 | C# 4.0+ | scriptcs | |||
| IRKernel | IPython 3.0 | R 3.2 | rzmq | ||
| SageMath | Jupyter 4 | Any | many | ||
| pari_jupyter | Jupyter 4 | PARI/GP >= 2.9 | |||
| IPython 2.0 | F# | Features | |||
| Jupyter >= 4, JupyterLab | Go >= 1.8 | ZeroMQ (4.x) | Example | Docker image | |
| Jupyter 4.1, JupyterLab | Go >= 1.6 | ZeroMQ (4.x) | examples | Deprecated, use gophernotes | |
| Jupyter 4, JupyterLab, nteract | Go >= 1.9 | ZeroMQ 4.x.x | examples | docker image | |
| Go >= 1.4 | |||||
| Scala | |||||
| Jupyter-scala | IPython>=3.0 | Scala>=2.10 | example | ||
| IPython 2.3 | Erlang | rebar | |||
| ITorch | IPython >= 2.2 and <= 5.x | Torch 7 (LuaJIT) | |||
| Jupyter < 6.0 | Elixir < 1.5 | Erlang OTP <= 19.3, Rebar | example | IElixir Notebook in Docker | |
| Jupyter >= 4.0 | Erlang 19 or 20, Elixir 1.4 or 1.5, LFE 1.2 | Erlang, (optional) Elixir | |||
| IPython >= 1 | Aldor | ||||
| IPython >= 1.1 | OCaml >= 4.01 | opam | |||
| Jupyter >= 4.0 | OCaml >= 4.02 | opam | Example | ||
| IPython >= 3 | Forth | ||||
| IPython 6/Jupyter 5 | Forth | Example | python debugger in FORTH syntax | ||
| IPerl | Perl 5 | ||||
| Perl6 | Jupyter >= 4 | Perl 6.c | zeromq 4 | ||
| Perl 6 | |||||
| Jupyter-Perl6 | Jupyter | Perl 6.C | Rakudo Perl 6 | ||
| IPHP | IPython >= 2 | PHP >= 5.4 | composer | DEPRECATED, use Jupyter-PHP | |
| Jupyter-PHP | Jupyter 4.0 | PHP >= 7.0.0 | composer, php-zmq | ||
| Jupyter | Octave | Example | MetaKernel | ||
| Jupyter | Scilab | Example | MetaKernel | ||
| Jupyter | Matlab | pymatbridge | Example | MetaKernel | |
| IPython >= 3 | bash | Wrapper | |||
| IPython >= 3 | Windows | Wrapper, Based on Bash Kernel | |||
| CloJupyter | Jupyter | Clojure >= 1.7 | |||
| Jupyter | Clojure | Abandoned as of 2017-02-12 | |||
| Jupyter>=4.0 | Clojure 1.8 | , Java>=7 | Java based JSR223 compliant | ||
| Jupyter | Hy | Tutorial | treats Hy as Python pre-processor | ||
| Jupyter | Hy | Tutorial | based on MetaKernel (magics, shell, parallel, etc.) | ||
| IPython >= 3 | redis | Wrapper | |||
| jove | io.js | ||||
| jp-babel | Jupyter | Babel | |||
| ICalico | IPython >= 2 | multiple | Index | ||
| IMathics | Mathics | ||||
| Wolfram Mathematica | Wolfram Mathematica(R), Metakernel | MetaKernel | |||
| Lua | |||||
| Purescript | |||||
| Lua | Fork of Lua Kernel | ||||
| Calysto Scheme | Scheme | Reference Guide | MetaKernel | ||
| Calysto Processing | Processing.js >= 2 | MetaKernel | |||
| idl_kernel | IDL | IDL seem to have a built-in kernelstarting with version 8.5 | |||
| Mochi Kernel | Mochi | ||||
| Lua (used in Splash) | Lua | ||||
| Apache Toree (formerly Spark Kernel) | Jupyter | Scala, Python, R | Spark >= 1.5 | Example | |
| Skulpt Python Kernel | Skulpt Python | Examples | MetaKernel | ||
| MetaKernel Bash | bash | MetaKernel | |||
| MetaKernel Python | python | MetaKernel | |||
| IVisual | VPython | Ball-in-Box | |||
| IBrainfuck | Brainfuck | Demo | Wrapper | ||
| KDB+/Q Kernel (IKdbQ) | IPython >= 3.1 | Q | qzmq, qcrypt | ||
| KDB+/Q Kernel (KdbQ Kernel) | Jupyter | Q | |||
| ICryptol | Cryptol | CVC4 | |||
| cling | Jupyter 4 | C++ | Example | ||
| Xonsh | Xonsh | Example | MetaKernel | ||
| Prolog | Prolog | MetaKernel | |||
| cl-jupyter | Jupyter | Common Lisp | Quicklisp | About | |
| Maxima-Jupyter | Jupyter | Maxima | Quicklisp | ||
| Calysto LC3 | Assembly Language for the Little Computer 3 | ||||
| Yacas | YACAS | ||||
| IJython | Jython 2.7 | ||||
| ROOT | Jupyter | C++/python | ROOT >= 6.05 | ||
| Gnuplot Kernel | Gnuplot | Example | MetaKernel | ||
| Tcl | Jupyter | Tcl 8.5 | Based on Bash Kernel | ||
| J | Jupyter | J 805 | Examples | ||
| Jython | Jupyter>=4.0 | Jython>=2.7.0 | Java>=7 | Java based JSR223 compliant | |
| C | Jupyter | C | gcc | ||
| TaQL | Jupyter | TaQL | python-casacore | TaQL tutorial | |
| Coconut | Jupyter | Coconut | |||
| SPARQL | Jupyter 4 | Python 2.7 or >=3.4 | rdflib, SPARQLWrapper | Examples | Optional GraphVizdependency |
| AIML chatbot | Jupyter 4 | Python 2.7 | pyAIML | Examples | |
| IArm | Jupyter 4 | ARMv6 THUMB | Examples | Based off of the ARM Cortex M0+ CPU | |
| SoS | Jupyter 4 | Python >=3.4 | Examples | Workflow system, Multi-Kernel support | |
| jupyter-nodejs | Jupyter, iPython 3.x | NodeJS, Babel, Clojurescript | Examples | ||
| Pike | IPython >= 3 | Pike >= 7.8 | Wrapper, Based on Bash Kernel | ||
| ITypeScript | Typescript >= 2.0 | Node.js >= 0.10.0 | |||
| imatlab | ipykernel >= 4.1 | MATLAB >= 2016b | |||
| jupyter-kotlin | Jupyter | Kotlin 1.1-M04 EAP | Java >= 8 | ||
| jupyter_kernel_singular | Jupyter | Singular 4.1.0 | Demo | Optional PySingular for better performance, surf for images, details | |
| spylon-kernel | ipykernel >=4.5 | python >= 3.5, scala >= 2.11 | Apache Spark >=2.0 | Example | MetaKernel |
| mit-scheme-kernel | Jupyter 4.0 | MIT Scheme 9.2 | |||
| elm-kernel | Jupyter | Examples | |||
| SciJava Jupyter Kernel | Jupyter 4.3.0 | Java + 9 scripting languages | Java | Examples | |
| Isbt | Jupyter 4.3.0 | sbt >= 1.0.0 | sbt | example | |
| BeakerX | Groovy, Java, Scala, Clojure, Kotlin, SQL | example | docker image | ||
| MicroPython | Jupyter | ESP8266/ESP32 | USB or Webrepl | developer notebooks | relies on the micro-controller's paste-mode |
| IJava | Jupyter | Java 9 | Java JDK >= 9 | Binder online demo | Based on the new JShell tool |
| Guile | Jupyter 5.2 | Guile 2.0 | guile-json, openssl |