메뉴 건너뛰기

검색창 닫기

Cloudera, BigData, Semantic IoT, Hadoop, NoSQL

Cloudera CDH/CDP 및 Hadoop EcoSystem, Semantic IoT등의 개발/운영 기술을 정리합니다. gooper@gooper.com로 문의 주세요.


fuseki/jena/ 동시에 많은 요청이 endpoint로 몰려서java.net.NoRouteToHostException가 발생하는 경우의 처리방법

총관리자 2016.10.17 14:27 조회 수 : 664

오류내용#1처럼 fuseki endpoint를 통하여 요청을 보낼때 순간적으로 많은 요청이 발생하면 많은양의 소켓케넥션이 TIME-WAIT상태(netstat -a | grep 사용포트(예,23030))가 되면서 오류가 발생할 수 있는데 이럴때는

echo "1" > /proc/sys/net/ipv4/tcp_tw_reuse
and/or
echo "1" > /proc/sys/net/ipv4/tcp_tw_recycle
를 수행하여 TIME-WAIT상태의 소켓 커넥션을 재사용할 수 있도록 설정해준다.

이렇게 처리해주면 오류내용#2가 발생하는데 이는 커넥션 재사용에 커넥션은 형성되었는데 이를 jena가 처리하지 못하는 과부하 상태를 의미한다.

* 참고 : http://stackoverflow.com/questions/1572215/how-to-avoid-a-noroutetohostexception
                                                                                             
-------------------------------오류내용#1-------------------------------------------
[2016-10-12 13:56:57,031] [service.llluminationStateJobService] [execute(#118)] [DEBUG] Exception  ....................................> HttpException: java.net.NoRouteToHostException: 요청한 주소를 배정할 수 없습니다: Unexpected error making the query: java.net.NoRouteToHostException: 요청한 주소를 배정할 수 없습니다 
[2016-10-12 13:56:57,031] [service.LectureTimeOnStateJobService] [execute(#112)] [DEBUG] Exception  ....................................> HttpException: java.net.NoRouteToHostException: 요청한 주소를 배정할 수 없습니다: Unexpected error making the query: java.net.NoRouteToHostException: 요청한 주소를 배정할 수 없습니다 
[2016-10-12 13:56:57,031] [service.ElectricPowerStateJobService] [execute(#115)] [DEBUG] Exception  ....................................> HttpException: java.net.NoRouteToHostException: 요청한 주소를 배정할 수 없습니다: Unexpected error making the query: java.net.NoRouteToHostException: 요청한 주소를 배정할 수 없습니다 
[2016-10-12 13:56:57,032] [service.HumanExistStateJobService] [execute(#116)] [DEBUG] Exception  ....................................> HttpException: java.net.NoRouteToHostException: 요청한 주소를 배정할 수 없습니다: Unexpected error making the query: java.net.NoRouteToHostException: 요청한 주소를 배정할 수 없습니다 
[2016-10-12 13:56:57,032] [service.ClimateStateJobService] [execute(#115)] [DEBUG] Exception  ....................................> HttpException: java.net.NoRouteToHostException: 요청한 주소를 배정할 수 없습니다: Unexpected error making the query: java.net.NoRouteToHostException: 요청한 주소를 배정할 수 없습니다 
[2016-10-12 13:56:57,034] [service.ProjectorStateJobService] [execute(#115)] [DEBUG] Exception  ....................................> HttpException: java.net.NoRouteToHostException: 요청한 주소를 배정할 수 없습니다: Unexpected error making the query: java.net.NoRouteToHostException: 요청한 주소를 배정할 수 없습니다 
[2016-10-12 13:56:57,034] [service.LectureTimeOffStateJobService] [execute(#112)] [DEBUG] Exception  ....................................> HttpException: java.net.NoRouteToHostException: 요청한 주소를 배정할 수 없습니다: Unexpected error making the query: java.net.NoRouteToHostException: 요청한 주소를 배정할 수 없습니다 



-------------------------------오류내용#2-----------------------------------------
[2016-10-13 10:44:24,487] [sf.SparqlService] [updateSparql(#376)] [DEBUG] delete->insert sparql start............................
[2016-10-13 10:44:24,487] [sf.SparqlService] [makeSparql(#138)] [DEBUG] count of idxVals : 2
[2016-10-13 10:44:24,487] [sf.SparqlService] [makeSparql(#139)] [DEBUG] values of idxVals : [http://www.iotoasis.org/herit-in/herit-cse/TempSensor_LR0001TS0005/status/Data, http://www.iotoasis.org/herit-in/herit-cse/TempSens
or_LR0001TS0005/status/Data/CONTENT_INST_2409764]
[2016-10-13 10:44:24,487] [sf.SparqlService] [makeSparql(#146)] [DEBUG] sparql to make ===========>
 prefix o: <http://www.iotoasis.org/ontology/>  PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>  delete  { <@{arg0}> o:hasLatestContentInstance ?o . }  WHERE   { <@{arg0}> o:hasLatestContentInstance  ?o  .}
[2016-10-13 10:44:24,487] [sf.SparqlService] [makeSparql(#268)] [DEBUG] sparql made ===========>
 prefix o: <http://www.iotoasis.org/ontology/>  PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>  delete  { <http://www.iotoasis.org/herit-in/herit-cse/TempSensor_LR0001TS0005/status/Data> o:hasLatestContentInstance
 ?o . }  WHERE   { <http://www.iotoasis.org/herit-in/herit-cse/TempSensor_LR0001TS0005/status/Data> o:hasLatestContentInstance  ?o  .}
org.apache.jena.atlas.web.HttpException: java.net.ConnectException: 연결이 거부됨
        at org.apache.jena.riot.web.HttpOp.exec(HttpOp.java:1202)
        at org.apache.jena.riot.web.HttpOp.execHttpPost(HttpOp.java:803)
        at org.apache.jena.riot.web.HttpOp.execHttpPost(HttpOp.java:578)
        at org.apache.jena.riot.web.HttpOp.execHttpPost(HttpOp.java:534)
        at org.apache.jena.sparql.modify.UpdateProcessRemote.execute(UpdateProcessRemote.java:79)
        at com.gooper.icbms.sda.sf.SparqlService.runModifySparql(SparqlService.java:404)
        at com.gooper.icbms.sda.sf.SparqlService.updateSparql(SparqlService.java:378)
        at com.gooper.icbms.sda.sf.TripleService.addLatestContentInstance(TripleService.java:173)
        at com.gooper.icbms.sda.kafka.onem2m.AvroOneM2MDataSubscribe$1.run(AvroOneM2MDataSubscribe.java:116)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.ConnectException: 연결이 거부됨
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:589)
        at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:120)
        at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:179)
        at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:328)
        at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:612)
        at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:447)
        at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:884)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
        at org.apache.jena.riot.web.HttpOp.exec(HttpOp.java:1186)
                                                                                                                                                                                                              6182859,25-32 99%
[2016-10-13 10:44:24,487] [sf.TripleService] [addLatestContentInstance(#172)] [DEBUG] this.getInstanceUri() =====>http://www.iotoasis.org/herit-in/herit-cse/TempSensor_LR0001TS0005/status/Data/C
ONTENT_INST_2409764
[2016-10-13 10:44:24,487] [sf.SparqlService] [updateSparql(#376)] [DEBUG] delete->insert sparql start............................
[2016-10-13 10:44:24,487] [sf.SparqlService] [makeSparql(#138)] [DEBUG] count of idxVals : 2
[2016-10-13 10:44:24,487] [sf.SparqlService] [makeSparql(#139)] [DEBUG] values of idxVals : [http://www.iotoasis.org/herit-in/herit-cse/TempSensor_LR0001TS0005/status/Data, http://www.iotoasis.o
rg/herit-in/herit-cse/TempSensor_LR0001TS0005/status/Data/CONTENT_INST_2409764]
[2016-10-13 10:44:24,487] [sf.SparqlService] [makeSparql(#146)] [DEBUG] sparql to make ===========>
 prefix o: <http://www.iotoasis.org/ontology/>  PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>  delete  { <@{arg0}> o:hasLatestContentInstance ?o . }  WHERE   { <@{arg0}> o:hasLatestC
ontentInstance  ?o  .}
[2016-10-13 10:44:24,487] [sf.SparqlService] [makeSparql(#268)] [DEBUG] sparql made ===========>
 prefix o: <http://www.iotoasis.org/ontology/>  PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>  delete  { <http://www.iotoasis.org/herit-in/herit-cse/TempSensor_LR0001TS0005/status/Da
ta> o:hasLatestContentInstance ?o . }  WHERE   { <http://www.iotoasis.org/herit-in/herit-cse/TempSensor_LR0001TS0005/status/Data> o:hasLatestContentInstance  ?o  .}
org.apache.jena.atlas.web.HttpException: java.net.ConnectException: 연결이 거부됨
        at org.apache.jena.riot.web.HttpOp.exec(HttpOp.java:1202)
        at org.apache.jena.riot.web.HttpOp.execHttpPost(HttpOp.java:803)
        at org.apache.jena.riot.web.HttpOp.execHttpPost(HttpOp.java:578)
        at org.apache.jena.riot.web.HttpOp.execHttpPost(HttpOp.java:534)
        at org.apache.jena.sparql.modify.UpdateProcessRemote.execute(UpdateProcessRemote.java:79)
        at com.gooper.icbms.sda.sf.SparqlService.runModifySparql(SparqlService.java:404)
        at com.gooper.icbms.sda.sf.SparqlService.updateSparql(SparqlService.java:378)
        at com.gooper.icbms.sda.sf.TripleService.addLatestContentInstance(TripleService.java:173)
        at com.gooper.icbms.sda.kafka.onem2m.AvroOneM2MDataSubscribe$1.run(AvroOneM2MDataSubscribe.java:116)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.ConnectException: 연결이 거부됨
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:589)
        at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:120)
        at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:179)
        at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:328)
        at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:612)
        at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:447)
        at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:884)
                                                                                                                                                                                6182859,25-32 99%
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:589)
        at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:120)
        at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:179)
        at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:328)
        at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:612)
        at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:447)
        at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:884)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
        at org.apache.jena.riot.web.HttpOp.exec(HttpOp.java:1186)
        ... 11 more
[2016-10-13 10:44:24,470] [sf.SparqlService] [runModifySparql(#407)] [DEBUG] Exception message in runModifySparql() =====> java.net.ConnectException: 연결이 거부됨
[2016-10-13 10:44:24,470] [util.Utils] [restartFuseki(#494)] [DEBUG] fuseki-server restart start ...................
[2016-10-13 10:44:24,470] [util.Utils] [killFuseki(#416)] [DEBUG] killFuseki start==========================>
[2016-10-13 10:44:24,487] [onem2m.AvroOneM2MDataSubscribe$1] [run(#95)] [DEBUG] count data from kafka broker : 39995
[2016-10-13 10:44:24,487] [sf.TripleService] [getTriple(#73)] [DEBUG] === ri : CONTENT_INST_2409764,  ty : 4 ====>include
contentType : normal
[2016-10-13 10:44:24,487] [onem2m.AvroOneM2MDataSubscribe$1] [run(#115)] [DEBUG] delete->insert o:hasLatestContentInstance start....................
[2016-10-13 10:44:24,487] [sf.TripleService] [addLatestContentInstance(#171)] [DEBUG] this.getParentResourceUri() ====> http://www.iotoasis.org/herit-in/herit-cse/TempSensor_LR0001TS0005/status/Data
[2016-10-13 10:44:24,487] [sf.TripleService] [addLatestContentInstance(#172)] [DEBUG] this.getInstanceUri() =====>http://www.iotoasis.org/herit-in/herit-cse/TempSensor_LR0001TS0005/status/Data/CONTENT_INST_2409764
[2016-10-13 10:44:24,487] [sf.SparqlService] [updateSparql(#376)] [DEBUG] delete->insert sparql start............................
[2016-10-13 10:44:24,487] [sf.SparqlService] [makeSparql(#138)] [DEBUG] count of idxVals : 2
[2016-10-13 10:44:24,487] [sf.SparqlService] [makeSparql(#139)] [DEBUG] values of idxVals : [http://www.iotoasis.org/herit-in/herit-cse/TempSensor_LR0001TS0005/status/Data, http://www.iotoasis.org/herit-in/herit-cse/TempSensor_LR0001TS0005/status/Data/CONT
ENT_INST_2409764]
[2016-10-13 10:44:24,487] [sf.SparqlService] [makeSparql(#146)] [DEBUG] sparql to make ===========>
 prefix o: <http://www.iotoasis.org/ontology/>  PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>  delete  { <@{arg0}> o:hasLatestContentInstance ?o . }  WHERE   { <@{arg0}> o:hasLatestContentInstance  ?o  .}
[2016-10-13 10:44:24,487] [sf.SparqlService] [makeSparql(#268)] [DEBUG] sparql made ===========>
 prefix o: <http://www.iotoasis.org/ontology/>  PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>  delete  { <http://www.iotoasis.org/herit-in/herit-cse/TempSensor_LR0001TS0005/status/Data> o:hasLatestContentInstance ?o . }  WHERE   { <http://www.i
otoasis.org/herit-in/herit-cse/TempSensor_LR0001TS0005/status/Data> o:hasLatestContentInstance  ?o  .}
org.apache.jena.atlas.web.HttpException: java.net.ConnectException: 연결이 거부됨
        at org.apache.jena.riot.web.HttpOp.exec(HttpOp.java:1202)
        at org.apache.jena.riot.web.HttpOp.execHttpPost(HttpOp.java:803)
        at org.apache.jena.riot.web.HttpOp.execHttpPost(HttpOp.java:578)
        at org.apache.jena.riot.web.HttpOp.execHttpPost(HttpOp.java:534)
        at org.apache.jena.sparql.modify.UpdateProcessRemote.execute(UpdateProcessRemote.java:79)
        at com.gooper.icbms.sda.sf.SparqlService.runModifySparql(SparqlService.java:404)
        at com.gooper.icbms.sda.sf.SparqlService.updateSparql(SparqlService.java:378)
        at com.gooper.icbms.sda.sf.TripleService.addLatestContentInstance(TripleService.java:173)
        at com.gooper.icbms.sda.kafka.onem2m.AvroOneM2MDataSubscribe$1.run(AvroOneM2MDataSubscribe.java:116)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.ConnectException: 연결이 거부됨
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:589)
        at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:120)
        at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:179)
        at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:328)
        at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:612)
        at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:447)
        at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:884)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
        at org.apache.jena.riot.web.HttpOp.exec(HttpOp.java:1186)
        ... 11 more
이 게시물을
이 글의 추천인 목록 목록
번호 제목 날짜 조회 수
261 hive기동시 Caused by: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D 오류 발생시 조치사항 2016.09.25 617
260 hive metadata(hive, impala, kudu 정보가 있음) 테이블에서 db, table, owner, location를 조회하는 쿼리 2020.02.07 622
259 Query 1234:1234 expired due to client inactivity(timeout is 5m)및 invalid query handle 2022.06.10 626
258 spark-shell실행시 "A read-only user or a user in a read-only database is not permitted to disable read-only mode on a connection."오류가 발생하는 경우 해결방법 2016.05.20 627
257 Scala에서 countByWindow를 이용하기(예제) 2018.03.08 631
256 spark-submit 실행시 "java.lang.OutOfMemoryError: Java heap space"발생시 조치사항 2018.02.01 641
255 Container killed by the ApplicationMaster. Container killed on request. Exit code is 143 Container exited with a non-zero exit code 143 TaskAttempt killed because it ran on unusable node 오류시 조치방법 2017.04.06 646
254 [Hive canary]Hive에 Metastore canary red alert및 hive log파일에 Duplicate entry '123456' for key 'NOTIFICATION_LOG_EVENT_ID'가 발생시 조치사항 2023.03.10 646
253 [Kudu]ERROR: Unable to advance iterator for node with id '2' for Kudu table 'impala::core.pm0_abdasubjct': Network error: recv error from unknown peer: Transport endpoint is not connected (error 107) 2023.03.16 647
252 Error: Could not find or load main class nodemnager 가 발생할때 해결하는 방법 2015.06.05 652
251 db를 통째로 새로운 이름의 db로 복사하는 방법/절차 2017.11.14 652
250 Error: java.lang.RuntimeException: java.lang.OutOfMemoryError 오류가 발생하는 경우 2018.09.20 654
249 kudu의 내부 table명 변경하는 방법 2022.11.10 655
248 [Atlas Server]org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient permissions (user=atlas/node01.gooper.com@GOOPER.COM, scope=default:atlas_janus, params=[table=default:atlas_janus,], action-CREATE)] 2023.05.15 659
247 Kafka Offset Monitor로 kafka 상태 모니터링 하기 file 2016.11.08 663
246 [HDFS]Encryption Zone에 생성된 테이블 조회시 Failed to open HDFS file hdfs://nameservice1/tmp/zone1/sec_test_file.txt Error(255): Unknown error 255 Root cause: AuthorizationException: User:impala not allowd to do 'DECRYPT_EEK' on 'testkey' 2023.06.29 663
245 Job이 끝난 log을 볼수 있도록 설정하기 2016.05.30 664
» 동시에 많은 요청이 endpoint로 몰려서java.net.NoRouteToHostException가 발생하는 경우의 처리방법 2016.10.17 664
243 compile할때와 exclude할때 대상을 표현하는 명칭이 다르므로 주의할것 2016.08.10 669
242 초기 오류(java.lang.NoSuchMethodError)에 따른 후속 작업에서 오류(java.lang.NoClassDefFoundError)가 발생되는 상황(quartz에서 주기적으로 작업시) 2016.08.29 670
쓰기 태그
위로