메뉴 건너뛰기

Cloudera, BigData, Semantic IoT, Hadoop, NoSQL

Cloudera CDH/CDP 및 Hadoop EcoSystem, Semantic IoT등의 개발/운영 기술을 정리합니다. gooper@gooper.com로 문의 주세요.


./spark-sql 실행시 mysql driver jar파일도 classpath에 등록했는데도 아래오 같은 오류가 발생시

hive-site.xml의 아래 항목을 찾아서 <value>0s</value>를 <value>0</value>로 변경해준다.

 <property>

    <name>hive.metastore.client.socket.lifetime</name>

    <value>0s</value>

    <description>

      Expects a time value with unit (d/day, h/hour, m/min, s/sec, ms/msec, us/usec, ns/nsec), which is sec if not specified.

      MetaStore Client socket lifetime in seconds. After this time is exceeded, client

      reconnects on the next MetaStore operation. A value of 0s means the connection

      has an infinite lifetime.

    </description>

  </property>

--->

 <property>

    <name>hive.metastore.event.clean.freq</name>

    <value>0</value>

    <description>

      Expects a time value with unit (d/day, h/hour, m/min, s/sec, ms/msec, us/usec, ns/nsec), which is sec if not specified.

      Frequency at which timer task runs to purge expired events in metastore.

    </description>

  </property>

  <property>

    <name>hive.metastore.event.expiry.duration</name>

    <value>0</value>

    <description>

      Expects a time value with unit (d/day, h/hour, m/min, s/sec, ms/msec, us/usec, ns/nsec), which is sec if not specified.

      Duration after which events expire from events table

    </description>

  </property>




---------------------------------오류내용--------------------------------------

16/06/09 13:37:46 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

16/06/09 13:37:46 INFO HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore

16/06/09 13:37:46 INFO ObjectStore: ObjectStore, initialize called

16/06/09 13:37:46 INFO Persistence: Property datanucleus.schema.validateColumns unknown - will be ignored

16/06/09 13:37:46 INFO Persistence: Property datanucleus.schema.validateConstraints unknown - will be ignored

16/06/09 13:37:46 INFO Persistence: Property hive.metastore.integral.jdo.pushdown unknown - will be ignored

16/06/09 13:37:46 INFO Persistence: Property datanucleus.cache.level2 unknown - will be ignored

16/06/09 13:37:46 INFO Persistence: Property datanucleus.schema.autoCreateAll unknown - will be ignored

16/06/09 13:37:46 INFO Persistence: Property datanucleus.schema.validateTables unknown - will be ignored

16/06/09 13:37:46 WARN Connection: BoneCP specified but not present in CLASSPATH (or one of dependencies)

16/06/09 13:37:46 WARN Connection: BoneCP specified but not present in CLASSPATH (or one of dependencies)

16/06/09 13:37:46 INFO ObjectStore: Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"

16/06/09 13:37:47 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table.

16/06/09 13:37:47 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table.

16/06/09 13:37:47 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table.

16/06/09 13:37:47 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table.

16/06/09 13:37:47 INFO Query: Reading in results for query "org.datanucleus.store.rdbms.query.SQLQuery@0" since the connection used is closing

16/06/09 13:37:47 INFO ObjectStore: Initialized ObjectStore

16/06/09 13:37:47 INFO HiveMetaStore: Added admin role in metastore

16/06/09 13:37:47 INFO HiveMetaStore: Added public role in metastore

16/06/09 13:37:47 INFO HiveMetaStore: No user is added in admin role, since config is empty

Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient

        at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:346)

        at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver$.main(SparkSQLCLIDriver.scala:101)

        at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.main(SparkSQLCLIDriver.scala)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:497)

        at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:569)

        at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:166)

        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:189)

        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:110)

        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient

        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1412)

        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:62)

        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:72)

        at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2453)

        at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2465)

        at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:340)

        ... 11 more

Caused by: java.lang.reflect.InvocationTargetException

        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)

        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

        at java.lang.reflect.Constructor.newInstance(Constructor.java:422)

        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1410)

        ... 16 more

Caused by: java.lang.NumberFormatException: For input string: "0s"

        at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)

        at java.lang.Long.parseLong(Long.java:589)

        at java.lang.Long.parseLong(Long.java:631)

        at org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1187)

        at org.apache.hadoop.hive.conf.HiveConf.getLongVar(HiveConf.java:1229)

        at org.apache.hadoop.hive.conf.HiveConf.getLongVar(HiveConf.java:1242)

        at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:429)

        at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.<init>(HiveMetaStore.java:356)

        at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:54)

        at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:59)

        at org.apache.hadoop.hive.metastore.HiveMetaStore.newHMSHandler(HiveMetaStore.java:4944)

        at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:171)

        ... 21 more

번호 제목 날짜 조회 수
750 [Shellscript]Impala view의 실제 참조 테이블 추출용 shellscript파일 2025.03.22 1001
749 엑셀에서 K ,M, G ,T 단위를 숫자로 변환 하는 수식 2025.04.09 1262
748 beeline을 이용한 impala JDBC 테스트 방법(Kerberos 설정된 상태임) 2024.11.29 1539
747 외부에서 ImpalaJDBC42.jar를 통해서 Impala에 접속시 sessions정보 2024.11.26 1624
746 test333 2017.05.01 1834
745 http://blog.naver.com... 2017.06.23 1838
744 Failed to resolve 'acme-v02.api.letsencrypt.org' ([Errno -3] Temporary failure in name resolution)" 2024.11.27 1914
743 eclipse 3.1 단축키 정리파일 2017.01.02 2058
742 5건의 triple data를 이용하여 특정 작업 폴더에서 작업하는 방법/절차 2016.06.16 2079
741 [vi] test.nq파일에서 특정문자열(예, <>)을 찾아서 포함되는 라인을 삭제한 동일한 이름의 파일을 만드는 방법 2017.01.25 2079
740 Windows에서 sbt개발환경 구축 방법(링크) 2016.06.02 2087
739 [EncryptionZone]User:testuser not allowed to do "DECRYPT_EEK" on 'testkey' 2023.06.29 2097
738 외부 jar파일을 만들려고하는jar파일의 package로 포함하는 방법 2016.08.10 2105
737 java스레드 덤프 분석하기 file 2016.11.03 2114
736 restaurant-controller,에서 등록 예시 2022.04.30 2123
735 DataSetCreator.py 실행시 파일을 찾을 수 없는 오류 2016.05.27 2129
734 실시간 쿼리 변환 모니터링(팩트내 필드값의 변경사항을 실시간으로 추적함)하는 테스트 java 프로그램 file 2016.07.21 2129
733 [oracle]10자리 timestamp값을 날짜로 변환하는 방법 2022.04.14 2164
732 하둡기반 데이타 모델링(6편) 2018.06.27 2177
731 [메모리 덤프파일 분석] 2017.03.31 2242
위로