broker에 메세지 전송을 요청했을때 처음에는 sda3(파티션1번)에 접속했으나 접속이 되지 않자 sda2로 자동 전환접속하여 메세지 전송을 처리완료함

이때의 topic정보를 보면 leader가 -1인데 이는 해당 노드의 kafka 브로커가 down된 상태임을 나타내므로 "./kafka-server-start.sh ../config/server-3.properties &"등의 명령으로 broker데몬을 기동시켜주고 topic정보를 확인하면 leader가 정상적으로 표시되며

아래와 같은 메세지는 나타나지 않는다.

---------------오류내용----------

-bash-4.1$ $HOME/spark/bin/spark-submit --master spark://sda1:7077,sda2:7077 --deploy-mode client --class com.gooper.icbms.sda.test.kafka.onem2m.content_instance.AvroOneM2MEmitter --jars sda-client-assembly-2.0.jar sda-client-assembly-2.0.jar

16/08/12 09:44:05 INFO VerifiableProperties: Verifying properties

16/08/12 09:44:05 INFO VerifiableProperties: Property metadata.broker.list is overridden to sda1:9092,sda2:9092,sda3:9092

16/08/12 09:44:05 INFO VerifiableProperties: Property partitioner.class is overridden to kafka.producer.DefaultPartitioner

16/08/12 09:44:05 INFO VerifiableProperties: Property request.required.acks is overridden to 1

16/08/12 09:44:05 INFO VerifiableProperties: Property serializer.class is overridden to kafka.serializer.DefaultEncoder

Send start (ContentInstance)......................

16/08/12 09:44:06 INFO ClientUtils$: Fetching metadata from broker BrokerEndPoint(2,sda3,9092) with correlation id 0 for 1 topic(s) Set(COL_CONTENT_INSTANCE)

16/08/12 09:44:06 INFO SyncProducer: Connected to sda3:9092 for producing

16/08/12 09:44:06 INFO SyncProducer: Disconnecting from sda3:9092

16/08/12 09:44:06 WARN ClientUtils$: Fetching topic metadata with correlation id 0 for topics [Set(COL_CONTENT_INSTANCE)] from broker [BrokerEndPoint(2,sda3,9092)] failed

java.nio.channels.ClosedChannelException

        at kafka.network.BlockingChannel.send(BlockingChannel.scala:110)

        at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:75)

        at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:74)

        at kafka.producer.SyncProducer.send(SyncProducer.scala:119)

        at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:59)

        at kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82)

        at kafka.producer.async.DefaultEventHandler$$anonfun$handle$1.apply$mcV$sp(DefaultEventHandler.scala:68)

        at kafka.utils.CoreUtils$.swallow(CoreUtils.scala:79)

        at kafka.utils.Logging$class.swallowError(Logging.scala:106)

        at kafka.utils.CoreUtils$.swallowError(CoreUtils.scala:51)

        at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:68)

        at kafka.producer.Producer.send(Producer.scala:77)

        at kafka.javaapi.producer.Producer.send(Producer.scala:33)

        at com.gooper.icbms.sda.test.kafka.onem2m.content_instance.AvroOneM2MEmitter.send(AvroOneM2MEmitter.java:62)

        at com.gooper.icbms.sda.test.kafka.onem2m.content_instance.AvroOneM2MEmitter.main(AvroOneM2MEmitter.java:71)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:497)

        at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:729)

        at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:185)

        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:210)

        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:124)

        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

16/08/12 09:44:06 INFO SyncProducer: Disconnecting from sda3:9092

16/08/12 09:44:06 INFO ClientUtils$: Fetching metadata from broker BrokerEndPoint(1,sda2,9092) with correlation id 0 for 1 topic(s) Set(COL_CONTENT_INSTANCE)

16/08/12 09:44:06 INFO SyncProducer: Connected to sda2:9092 for producing

16/08/12 09:44:06 INFO SyncProducer: Disconnecting from sda2:9092

16/08/12 09:44:06 WARN BrokerPartitionInfo: Error while fetching metadata       partition 1     leader: none    replicas:       isr:    isUnderReplicated: false for topic partition [COL_CONTENT_INSTANCE,1]: [class kafka.common.LeaderNotAvailableException]

16/08/12 09:44:06 INFO SyncProducer: Connected to sda2:9092 for producing

Send end(ContentInstance)......................

16/08/12 09:44:06 INFO Producer: Shutting down producer

16/08/12 09:44:06 INFO ProducerPool: Closing all sync producers

16/08/12 09:44:06 INFO SyncProducer: Disconnecting from sda2:9092

16/08/12 09:44:06 INFO SyncProducer: Disconnecting from sda1:9092

16/08/12 09:44:06 INFO Producer: Producer shutdown completed in 7 ms

------topic정보확인

-bash-4.1# ./kafka-topics.sh --describe --zookeeper sda1:2181 --topic COL_CONTENT_INSTANCE

Topic:COL_CONTENT_INSTANCE      PartitionCount:3        ReplicationFactor:1     Configs:

        Topic: COL_CONTENT_INSTANCE     Partition: 0    Leader: 2       Replicas: 2     Isr: 2

        Topic: COL_CONTENT_INSTANCE     Partition: 1    Leader: -1      Replicas: 3     Isr: 

        Topic: COL_CONTENT_INSTANCE     Partition: 2    Leader: 1       Replicas: 1     Isr: 1