메뉴 건너뛰기

Cloudera, BigData, Semantic IoT, Hadoop, NoSQL

Cloudera CDH/CDP 및 Hadoop EcoSystem, Semantic IoT등의 개발/운영 기술을 정리합니다. gooper@gooper.com로 문의 주세요.


hive-site.xml에서 아래 3개의 <value>에서 ms를 뺀다.


 <property>

    <name>hive.hmshandler.retry.interval</name>

    <value>2000ms</value>

    <description>

      Expects a time value with unit (d/day, h/hour, m/min, s/sec, ms/msec, us/usec, ns/nsec), which is msec if not specified.

      The time between HMSHandler retry attempts on failure.

    </description>

  </property>


  <property>

    <name>hive.llap.am.liveness.connection.sleep.between.retries.ms</name>

    <value>2000ms</value>

    <description>

      Expects a time value with unit (d/day, h/hour, m/min, s/sec, ms/msec, us/usec, ns/nsec), which is msec if not specified.

      Sleep duration while waiting to retry connection failures to the AM from the daemon for

      the general keep-alive thread (milliseconds).

    </description>

  </property>



  <property>

    <name>hive.llap.task.communicator.connection.sleep.between.retries.ms</name>

    <value>2000ms</value>

    <description>

      Expects a time value with unit (d/day, h/hour, m/min, s/sec, ms/msec, us/usec, ns/nsec), which is msec if not specified.

      Sleep duration (in milliseconds) to wait before retrying on error when obtaining a

      connection to LLAP daemon from Tez AM.

    </description>

  </property>




---------------------------------오류내용-----------------------------------------

16/06/09 13:57:38 INFO SparkDeploySchedulerBackend: SchedulerBackend is ready for scheduling beginning after reached minRegisteredResourcesRatio: 0.0

SET spark.sql.hive.version=0.13.1

16/06/09 13:57:39 ERROR log: Got exception: java.lang.NumberFormatException For input string: "2000ms"

java.lang.NumberFormatException: For input string: "2000ms"

        at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)

        at java.lang.Integer.parseInt(Integer.java:580)

        at java.lang.Integer.parseInt(Integer.java:615)

        at org.apache.hadoop.conf.Configuration.getInt(Configuration.java:1134)

        at org.apache.hadoop.hive.conf.HiveConf.getIntVar(HiveConf.java:1211)

        at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:87)

        at com.sun.proxy.$Proxy6.get_all_databases(Unknown Source)

        at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getAllDatabases(HiveMetaStoreClient.java:837)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:497)

        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89)

        at com.sun.proxy.$Proxy7.getAllDatabases(Unknown Source)

        at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1098)

        at org.apache.hadoop.hive.ql.exec.FunctionRegistry.getFunctionNames(FunctionRegistry.java:671)

        at org.apache.hadoop.hive.ql.exec.FunctionRegistry.getFunctionNames(FunctionRegistry.java:662)

        at org.apache.hadoop.hive.cli.CliDriver.getCommandCompletor(CliDriver.java:540)

        at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver$.main(SparkSQLCLIDriver.scala:168)

        at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.main(SparkSQLCLIDriver.scala)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:497)

        at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:569)

        at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:166)

        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:189)

        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:110)

        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

16/06/09 13:57:39 ERROR log: Converting exception to MetaException

16/06/09 13:57:39 ERROR FunctionRegistry: org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Got exception: java.lang.NumberFormatException For input string: "2000ms")

spark-sql> 16/06/09 13:57:39 INFO SparkDeploySchedulerBackend: Registered executor: Actor[akka.tcp://sparkExecutor@sda2:47136/user/Executor#-853352827] with ID 2

16/06/09 13:57:39 INFO SparkDeploySchedulerBackend: Registered executor: Actor[akka.tcp://sparkExecutor@so2:35213/user/Executor#-894168911] with ID 0

16/06/09 13:57:39 INFO BlockManagerMasterActor: Registering block manager sda2:58779 with 265.1 MB RAM, BlockManagerId(2, sda2, 58779)

16/06/09 13:57:39 INFO BlockManagerMasterActor: Registering block manager so2:33469 with 265.1 MB RAM, BlockManagerId(0, so2, 33469)

16/06/09 13:57:41 INFO SparkDeploySchedulerBackend: Registered executor: Actor[akka.tcp://sparkExecutor@so-db2:35159/user/Executor#-473750792] with ID 1

16/06/09 13:57:41 INFO BlockManagerMasterActor: Registering block manager so-db2:47990 with 265.1 MB RAM, BlockManagerId(1, so-db2, 47990)

번호 제목 날짜 조회 수
201 cloudera에서 spark-shell를 실행했을때 default master는 spark.master=yarn-client임 2018.06.20 429
200 CDH에서 Sentry 개념및 설정 file 2018.06.21 582
199 하둡기반 데이타 모델링(6편) 2018.06.27 289
198 resouce manager에 dr.who가 아닌 다른 사용자로 로그인 하기 2018.06.28 1487
197 upsert구현방법(년-월-일 파티션을 기준으로) 및 테스트 script file 2018.07.03 1617
196 sentry설정후 beeline으로 hive2server에 접속하여 admin계정에 admin권한 부여하기 2018.07.03 489
195 spark 2.3.0을 설치하가 위해서 parcel에 다음 url을 입력한다. 2018.07.15 316
194 [postgresql 9.x] PostgreSQL Replication 구축하기 2018.07.17 324
193 HiveServer2인증을 PAM을 이용하도록 설정하는 방법 2018.07.21 475
192 anaconda3(v5.4)를 이용하여 tensorflow설치후 ipython프로그램을 실행하여 import할때 오류발생시 조치 2018.07.27 289
191 anaconda3 (v5.2) 설치및 머신러닝 관련 라이브러리 설치 절차 2018.07.27 675
190 conda를 이용한 jupyterhub(v0.9)및 jupyter설치 (v4.4.0) 2018.07.30 714
189 RHEL 7.4에 zeppelin 0.7.4 설치 2018.07.31 482
188 Last transaction was partial에 따른 Unable to load database on disk오류 발생시 조치사항 2018.08.03 4099
187 sqoop으로 mariadb에 접근해서 hive 테이블로 자동으로 생성하기 2018.08.03 819
186 컬럼및 라인의 구분자를 지정하여 sqoop으로 데이타를 가져오고 hive테이블을 생성하는 명령문 2018.08.03 694
185 sentry설정 방법및 활성화시 설정이 필요한 파일및 설정값, 계정생성 방법 2018.08.16 891
184 postgresql-9.4에서 FATAL: remaining connection slots are reserved for non-replication superuser connections가 나올때 조치 2018.08.16 1085
183 oracle to hive data type정리표 2018.08.22 939
182 ubuntu 커널 업그레이드 방법 2018.09.02 1486
위로