메뉴 건너뛰기

Cloudera, BigData, Semantic IoT, Hadoop, NoSQL

Cloudera CDH/CDP 및 Hadoop EcoSystem, Semantic IoT등의 개발/운영 기술을 정리합니다. gooper@gooper.com로 문의 주세요.


./hadoop-daemon.sh start namenode로 namenode기동시 EditLog의 custerId, namespaceId가 달라서 오류가 발생하면
"bin/hadoop namenode -recover"를 실행하면 문제를 해결해준다.

recover를 실행하면 아래의 짊문을 하는데 s를 선택한다.
Enter 'c' to continue, skipping the bad section in the log
Enter 's' to stop reading the edit log here, abandoning any later edits
Enter 'q' to quit without saving
Enter 'a' to always select the first choice in the future without prompting. (c/s/q/a)



-------------------------오류내용-----------------
16/09/24 10:57:03 INFO namenode.FSImageFormatPBINode: Loading 1 INodes.
16/09/24 10:57:03 INFO namenode.FSImageFormatProtobuf: Loaded FSImage in 0 seconds.
16/09/24 10:57:03 INFO namenode.FSImage: Loaded image for txid 0 from /engine/bigdata/hadoop/dfs/namenode/current/fsimage_                                              0000000000000000000
16/09/24 10:57:03 INFO namenode.FSImage: Reading org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream@830730                                              f8 expecting start txid #1
16/09/24 10:57:03 INFO namenode.FSImage: Start loading edits file http://IUDGTMP03:8480/getJournal?jid=mycluster&segmentTx                                              Id=1&storageInfo=-63%3A1812369513%3A0%3ACID-732a108d-6a42-44e4-9b2a-313750cc4c35, http://IUDGTMP04:8480/getJournal?jid=myc                                              luster&segmentTxId=1&storageInfo=-63%3A1812369513%3A0%3ACID-732a108d-6a42-44e4-9b2a-313750cc4c35
16/09/24 10:57:03 INFO namenode.EditLogInputStream: Fast-forwarding stream 'http://IUDGTMP03:8480/getJournal?jid=mycluster                                              &segmentTxId=1&storageInfo=-63%3A1812369513%3A0%3ACID-732a108d-6a42-44e4-9b2a-313750cc4c35, http://IUDGTMP04:8480/getJourn                                              al?jid=mycluster&segmentTxId=1&storageInfo=-63%3A1812369513%3A0%3ACID-732a108d-6a42-44e4-9b2a-313750cc4c35' to transaction                                               ID 1
16/09/24 10:57:03 INFO namenode.EditLogInputStream: Fast-forwarding stream 'http://IUDGTMP03:8480/getJournal?jid=mycluster                                              &segmentTxId=1&storageInfo=-63%3A1812369513%3A0%3ACID-732a108d-6a42-44e4-9b2a-313750cc4c35' to transaction ID 1
16/09/24 10:57:03 ERROR namenode.EditLogInputStream: caught exception initializing http://IUDGTMP03:8480/getJournal?jid=my                                              cluster&segmentTxId=1&storageInfo=-63%3A1812369513%3A0%3ACID-732a108d-6a42-44e4-9b2a-313750cc4c35
org.apache.hadoop.hdfs.server.namenode.TransferFsImage$HttpGetFailedException: Fetch of http://IUDGTMP03:8480/getJournal?j                                              id=mycluster&segmentTxId=1&storageInfo=-63%3A1812369513%3A0%3ACID-732a108d-6a42-44e4-9b2a-313750cc4c35 failed with status                                               code 403
Response message:
This node has namespaceId '671989181 and clusterId 'CID-8395f825-0009-4668-862b-3a9e8d184b54' but the requesting node expe                                              cted '1812369513' and 'CID-732a108d-6a42-44e4-9b2a-313750cc4c35'
        at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog$1.run(EditLogFileInputStream.java:468)
        at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog$1.run(EditLogFileInputStream.java:456)
        at java.security.AccessController.doPrivileged(AccessController.java:377)
        at javax.security.auth.Subject.doAs(Subject.java:572)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
        at org.apache.hadoop.security.SecurityUtil.doAsUser(SecurityUtil.java:448)
        at org.apache.hadoop.security.SecurityUtil.doAsCurrentUser(SecurityUtil.java:442)
        at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog.getInputStream(EditLogFileInputStream.java                                              :455)
        at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream.init(EditLogFileInputStream.java:141)
        at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream.nextOpImpl(EditLogFileInputStream.java:192)
        at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream.nextOp(EditLogFileInputStream.java:250)
        at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
        at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.skipUntil(EditLogInputStream.java:151)
        at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:178)
        at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
        at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.skipUntil(EditLogInputStream.java:151)
        at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:178)
        at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
        at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:190)
        at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:143)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:837)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:692)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:294)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:975)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:681)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.doRecovery(NameNode.java:1382)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1472)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1554)
16/09/24 10:57:03 ERROR namenode.EditLogInputStream: Got error reading edit log input stream http://IUDGTMP03:8480/getJour                                              nal?jid=mycluster&segmentTxId=1&storageInfo=-63%3A1812369513%3A0%3ACID-732a108d-6a42-44e4-9b2a-313750cc4c35; failing over                                               to edit log http://IUDGTMP04:8480/getJournal?jid=mycluster&segmentTxId=1&storageInfo=-63%3A1812369513%3A0%3ACID-732a108d-6                                              a42-44e4-9b2a-313750cc4c35
org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream$PrematureEOFException: got premature end-of-file at txi                                              d 0; expected file to go up to 2
        at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:194)
        at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
        at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.skipUntil(EditLogInputStream.java:151)
        at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:178)
        at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
        at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:190)
        at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:143)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:837)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:692)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:294)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:975)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:681)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.doRecovery(NameNode.java:1382)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1472)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1554)
16/09/24 10:57:03 INFO namenode.EditLogInputStream: Fast-forwarding stream 'http://IUDGTMP04:8480/getJournal?jid=mycluster                                              &segmentTxId=1&storageInfo=-63%3A1812369513%3A0%3ACID-732a108d-6a42-44e4-9b2a-313750cc4c35' to transaction ID 1
16/09/24 10:57:03 ERROR namenode.EditLogInputStream: caught exception initializing http://IUDGTMP04:8480/getJournal?jid=my                                              cluster&segmentTxId=1&storageInfo=-63%3A1812369513%3A0%3ACID-732a108d-6a42-44e4-9b2a-313750cc4c35
org.apache.hadoop.hdfs.server.namenode.TransferFsImage$HttpGetFailedException: Fetch of http://IUDGTMP04:8480/getJournal?j                                              id=mycluster&segmentTxId=1&storageInfo=-63%3A1812369513%3A0%3ACID-732a108d-6a42-44e4-9b2a-313750cc4c35 failed with status                                               code 403
Response message:
This node has namespaceId '671989181 and clusterId 'CID-8395f825-0009-4668-862b-3a9e8d184b54' but the requesting node expe                                              cted '1812369513' and 'CID-732a108d-6a42-44e4-9b2a-313750cc4c35'
        at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog$1.run(EditLogFileInputStream.java:468)
        at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog$1.run(EditLogFileInputStream.java:456)
        at java.security.AccessController.doPrivileged(AccessController.java:377)
        at javax.security.auth.Subject.doAs(Subject.java:572)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
        at org.apache.hadoop.security.SecurityUtil.doAsUser(SecurityUtil.java:448)
        at org.apache.hadoop.security.SecurityUtil.doAsCurrentUser(SecurityUtil.java:442)
        at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog.getInputStream(EditLogFileInputStream.java                                              :455)
        at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream.init(EditLogFileInputStream.java:141)
        at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream.nextOpImpl(EditLogFileInputStream.java:192)
        at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream.nextOp(EditLogFileInputStream.java:250)
        at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
        at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.skipUntil(EditLogInputStream.java:151)
        at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:178)
        at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
        at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.skipUntil(EditLogInputStream.java:151)
        at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:178)
        at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
        at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:190)
        at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:143)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:837)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:692)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:294)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:975)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:681)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.doRecovery(NameNode.java:1382)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1472)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1554)
16/09/24 10:57:03 ERROR namenode.FSImage: Error replaying edit log at offset 0.  Expected transaction ID was 1
org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream$PrematureEOFException: got premature end-of-file at txi                                              d 0; expected file to go up to 2
        at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:194)
        at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
        at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.skipUntil(EditLogInputStream.java:151)
        at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:178)
        at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
        at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:190)
        at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:143)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:837)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:692)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:294)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:975)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:681)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.doRecovery(NameNode.java:1382)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1472)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1554)
16/09/24 10:57:03 ERROR namenode.MetaRecoveryContext: We failed to read txId 1

Enter 'c' to continue, skipping the bad section in the log
Enter 's' to stop reading the edit log here, abandoning any later edits
Enter 'q' to quit without saving
Enter 'a' to always select the first choice in the future without prompting. (c/s/q/a)
s
16/09/24 10:57:52 WARN namenode.MetaRecoveryContext: Stopped reading edit log at 0/-1
16/09/24 10:57:52 INFO namenode.FSImage: Edits file http://IUDGTMP03:8480/getJournal?jid=mycluster&segmentTxId=1&storageInfo=-63%3A1812369513%3A0%3ACI                  D-732a108d-6a42-44e4-9b2a-313750cc4c35, http://IUDGTMP04:8480/getJournal?jid=mycluster&segmentTxId=1&storageInfo=-63%3A1812369513%3A0%3ACID-732a108d-6                  a42-44e4-9b2a-313750cc4c35 of size -1 edits # 0 loaded in 49 seconds
16/09/24 10:57:52 INFO namenode.FSImage: Reading org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream@fc20cc3e expecting start txid #3
16/09/24 10:57:52 INFO namenode.FSImage: Start loading edits file http://IUDGTMP04:8480/getJournal?jid=mycluster&segmentTxId=3&storageInfo=-63%3A18123                  69513%3A0%3ACID-732a108d-6a42-44e4-9b2a-313750cc4c35, http://IUDGTMP03:8480/getJournal?jid=mycluster&segmentTxId=3&storageInfo=-63%3A1812369513%3A0%3A                  CID-732a108d-6a42-44e4-9b2a-313750cc4c35
16/09/24 10:57:52 INFO namenode.EditLogInputStream: Fast-forwarding stream 'http://IUDGTMP04:8480/getJournal?jid=mycluster&segmentTxId=3&storageInfo=-                  63%3A1812369513%3A0%3ACID-732a108d-6a42-44e4-9b2a-313750cc4c35, http://IUDGTMP03:8480/getJournal?jid=mycluster&segmentTxId=3&storageInfo=-63%3A1812369                  513%3A0%3ACID-732a108d-6a42-44e4-9b2a-313750cc4c35' to transaction ID 1
16/09/24 10:57:52 INFO namenode.EditLogInputStream: Fast-forwarding stream 'http://IUDGTMP04:8480/getJournal?jid=mycluster&segmentTxId=3&storageInfo=-                  63%3A1812369513%3A0%3ACID-732a108d-6a42-44e4-9b2a-313750cc4c35' to transaction ID 1
16/09/24 10:57:52 ERROR namenode.EditLogInputStream: caught exception initializing http://IUDGTMP04:8480/getJournal?jid=mycluster&segmentTxId=3&storag                  eInfo=-63%3A1812369513%3A0%3ACID-732a108d-6a42-44e4-9b2a-313750cc4c35
org.apache.hadoop.hdfs.server.namenode.TransferFsImage$HttpGetFailedException: Fetch of http://IUDGTMP04:8480/getJournal?jid=mycluster&segmentTxId=3&s                  torageInfo=-63%3A1812369513%3A0%3ACID-732a108d-6a42-44e4-9b2a-313750cc4c35 failed with status code 403
Response message:
This node has namespaceId '671989181 and clusterId 'CID-8395f825-0009-4668-862b-3a9e8d184b54' but the requesting node expected '1812369513' and 'CID-7                  32a108d-6a42-44e4-9b2a-313750cc4c35'
        at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog$1.run(EditLogFileInputStream.java:468)
        at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog$1.run(EditLogFileInputStream.java:456)
        at java.security.AccessController.doPrivileged(AccessController.java:377)
        at javax.security.auth.Subject.doAs(Subject.java:572)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
        at org.apache.hadoop.security.SecurityUtil.doAsUser(SecurityUtil.java:448)
        at org.apache.hadoop.security.SecurityUtil.doAsCurrentUser(SecurityUtil.java:442)
        at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog.getInputStream(EditLogFileInputStream.java:455)
        at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream.init(EditLogFileInputStream.java:141)
        at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream.nextOpImpl(EditLogFileInputStream.java:192)
        at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream.nextOp(EditLogFileInputStream.java:250)
        at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
        at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.skipUntil(EditLogInputStream.java:151)
        at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:178)
        at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
        at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.skipUntil(EditLogInputStream.java:151)
        at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:178)
        at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
        at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:190)
        at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:143)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:837)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:692)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:294)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:975)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:681)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.doRecovery(NameNode.java:1382)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1472)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1554)
16/09/24 10:57:52 ERROR namenode.EditLogInputStream: Got error reading edit log input stream http://IUDGTMP04:8480/getJournal?jid=mycluster&segmentTxI                  d=3&storageInfo=-63%3A1812369513%3A0%3ACID-732a108d-6a42-44e4-9b2a-313750cc4c35; failing over to edit log http://IUDGTMP03:8480/getJournal?jid=myclust                  er&segmentTxId=3&storageInfo=-63%3A1812369513%3A0%3ACID-732a108d-6a42-44e4-9b2a-313750cc4c35
org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream$PrematureEOFException: got premature end-of-file at txid 0; expected file to go up                   to 127
        at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:194)
        at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
        at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.skipUntil(EditLogInputStream.java:151)
        at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:178)
        at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
        at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:190)
        at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:143)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:837)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:692)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:294)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:975)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:681)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.doRecovery(NameNode.java:1382)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1472)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1554)
16/09/24 10:57:52 INFO namenode.EditLogInputStream: Fast-forwarding stream 'http://IUDGTMP03:8480/getJournal?jid=mycluster&segmentTxId=3&storageInfo=-                  63%3A1812369513%3A0%3ACID-732a108d-6a42-44e4-9b2a-313750cc4c35' to transaction ID 1
16/09/24 10:57:52 ERROR namenode.EditLogInputStream: caught exception initializing http://IUDGTMP03:8480/getJournal?jid=mycluster&segmentTxId=3&storag                  eInfo=-63%3A1812369513%3A0%3ACID-732a108d-6a42-44e4-9b2a-313750cc4c35
org.apache.hadoop.hdfs.server.namenode.TransferFsImage$HttpGetFailedException: Fetch of http://IUDGTMP03:8480/getJournal?jid=mycluster&segmentTxId=3&s                  torageInfo=-63%3A1812369513%3A0%3ACID-732a108d-6a42-44e4-9b2a-313750cc4c35 failed with status code 403
Response message:
This node has namespaceId '671989181 and clusterId 'CID-8395f825-0009-4668-862b-3a9e8d184b54' but the requesting node expected '1812369513' and 'CID-7                  32a108d-6a42-44e4-9b2a-313750cc4c35'
        at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog$1.run(EditLogFileInputStream.java:468)
        at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog$1.run(EditLogFileInputStream.java:456)
        at java.security.AccessController.doPrivileged(AccessController.java:377)
        at javax.security.auth.Subject.doAs(Subject.java:572)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
        at org.apache.hadoop.security.SecurityUtil.doAsUser(SecurityUtil.java:448)
        at org.apache.hadoop.security.SecurityUtil.doAsCurrentUser(SecurityUtil.java:442)
        at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog.getInputStream(EditLogFileInputStream.java:455)
        at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream.init(EditLogFileInputStream.java:141)
        at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream.nextOpImpl(EditLogFileInputStream.java:192)
        at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream.nextOp(EditLogFileInputStream.java:250)
        at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
        at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.skipUntil(EditLogInputStream.java:151)
        at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:178)
        at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
        at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.skipUntil(EditLogInputStream.java:151)
        at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:178)
        at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
        at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:190)
        at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:143)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:837)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:692)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:294)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:975)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:681)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.doRecovery(NameNode.java:1382)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1472)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1554)
16/09/24 10:57:52 ERROR namenode.FSImage: Error replaying edit log at offset 0.  Expected transaction ID was 3
org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream$PrematureEOFException: got premature end-of-file at txid 0; expected file to go up                   to 127
        at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:194)
        at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
        at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.skipUntil(EditLogInputStream.java:151)
        at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:178)
        at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
        at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:190)
        at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:143)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:837)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:692)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:294)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:975)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:681)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.doRecovery(NameNode.java:1382)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1472)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1554)
16/09/24 10:57:52 ERROR namenode.MetaRecoveryContext: We failed to read txId 3

Enter 'c' to continue, skipping the bad section in the log
Enter 's' to stop reading the edit log here, abandoning any later edits
Enter 'q' to quit without saving
Enter 'a' to always select the first choice in the future without prompting. (c/s/q/a)
s
16/09/24 10:57:54 WARN namenode.MetaRecoveryContext: Stopped reading edit log at 0/-1
16/09/24 10:57:54 INFO namenode.FSImage: Edits file http://IUDGTMP04:8480/getJournal?jid=mycluster&segmentTxId=3&storageInfo=-63%3A1812369513%3A0%3ACI                  D-732a108d-6a42-44e4-9b2a-313750cc4c35, http://IUDGTMP03:8480/getJournal?jid=mycluster&segmentTxId=3&storageInfo=-63%3A1812369513%3A0%3ACID-732a108d-6                  a42-44e4-9b2a-313750cc4c35 of size -1 edits # 0 loaded in 1 seconds
16/09/24 10:57:54 INFO namenode.FSNamesystem: Need to save fs image? false (staleImage=true, haEnabled=true, isRollingUpgrade=false)
16/09/24 10:57:54 INFO namenode.NameCache: initialized with 0 entries 0 lookups
16/09/24 10:57:54 INFO namenode.FSNamesystem: Finished loading FSImage in 52039 msecs
16/09/24 10:57:54 INFO namenode.FSImage: Save namespace ...
16/09/24 10:57:54 INFO namenode.NNStorageRetentionManager: Going to retain 2 images with txid >= 0
16/09/24 10:57:55 INFO namenode.MetaRecoveryContext: RECOVERY COMPLETE
16/09/24 10:57:55 INFO namenode.FSNamesystem: Stopping services started for active state
16/09/24 10:57:55 INFO namenode.FSNamesystem: Stopping services started for standby state
16/09/24 10:57:55 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at IUDGTMP02/XXX.XXX.XXX.32
************************************************************/

번호 제목 날짜 조회 수
47 hadoop에서 yarn jar ..를 이용하여 appliction을 실행하여 정상적(?)으로 수행되었으나 yarn UI의 어플리케이션 목록에 나타나지 않는 문제 2017.05.02 217
46 hadoop에서 yarn jar ..를 이용하여 appliction을 실행하여 정상적으로 수행되었으나 yarn UI의 어플리케이션 목록에 나타나지 않는 문제 2017.05.02 227
45 hadoop에서 yarn jar ..를 이용하여 appliction을 실행하여 정상적으로 수행되었으나 yarn UI의 어플리케이션 목록에 나타나지 않는 문제 2017.05.02 156
44 hadoop에서 yarn jar ..를 이용하여 appliction을 실행하여 정상적으로 수행되었으나 yarn UI의 어플리케이션 목록에 나타나지 않는 문제 2017.05.02 287
43 Cleaning up the staging area file시 'cannot access' 혹은 'Directory is not writable' 발생시 조치사항 2017.05.02 451
42 Ubuntu 16.04 LTS에 4대에 Hadoop 2.8.0설치 2017.05.01 1130
41 Container killed by the ApplicationMaster. Container killed on request. Exit code is 143 Container exited with a non-zero exit code 143 TaskAttempt killed because it ran on unusable node 오류시 조치방법 2017.04.06 1003
40 서버중 slave,worker,regionserver만 재기동해야 할때 필요한 기동스크립트및 사용방법 2017.02.03 787
39 HDFS상의 /tmp폴더에 Permission denied오류가 발생시 조치사항 2017.01.25 490
38 Mountable HDFS on CentOS 6.x(hadoop 2.7.2의 nfs기능을 이용) 2016.11.24 826
» ./hadoop-daemon.sh start namenode로 namenode기동시 EditLog의 custerId, namespaceId가 달라서 발생하는 오류 해결방법 2016.09.24 289
36 hadoop 어플리케이션을 사용하는 사용자 변경시 바꿔줘야 하는 부분 2016.09.23 818
35 format된 namenode를 다른 서버에서 다시 format했을때 오류내용 2016.09.22 326
34 AIX 7.1에 Hadoop설치(정리중#2) 2016.09.20 375
33 AIX 7.1에 Hadoop설치(정리중) 2016.09.12 889
32 hadoop클러스터를 구성하던 서버중 HA를 담당하는 서버의 hostname등이 변경되어 문제가 발생했을때 조치사항 2016.07.29 447
31 Journal Storage Directory /data/hadoop/journal/data/mycluster not formatted 오류시 조치사항 2016.07.29 1699
30 Job이 끝난 log을 볼수 있도록 설정하기 2016.05.30 1034
29 Hadoop 완벽 가이드 정리된 링크 2016.04.19 830
28 Incompatible clusterIDs오류 원인및 해결방법 2016.04.01 645
위로