Cloudera CDH/CDP 및 Hadoop EcoSystem, Semantic IoT등의 개발/운영 기술을 정리합니다. gooper@gooper.com로 문의 주세요.
오류메세지에 있는것처럼 Full GC가 계속 발생하면서 메모리 해제는 이루어지지 않고 VisualVM을 이용하여 스레드를 확인하면 아래와 같이 다수의 MongoCleaner가 Sleep상태(Full GC에서 모든 쓰레드는 대시 상태가 된다.)에 있고 GC상황을 보면 Old Space가 full 상태에 있으면서 모든 처리가 대기 상태에 있는 경우가 있는데 이것은 mongodb의 커넥션혹은 cursor등을 사용후 닫지 않아서 발생하는 memory leak이라고 볼수 있다. 소스에서 커넥션 혹은 cursor가 제대로 close되었는지 확인하여 조치하여 준다.
-오류 메세지
[Full GC (Ergonomics) [PSYoungGen: 345088K->345072K(694272K)] [ParOldGen: 2097094K->2097094K(2097152K)] 2442182K->2442167K(2791424K), [Metaspace: 59102K->59102K(1101824K)], 0.0743223 secs] [Times: user=0.59 sys=0.00, real=0.08 secs]
[Full GC (Ergonomics) [PSYoungGen: 345088K->345072K(694272K)] [ParOldGen: 2097094K->2097094K(2097152K)] 2442182K->2442167K(2791424K), [Metaspace: 59102K->59102K(1101824K)], 0.0779379 secs] [Times: user=0.68 sys=0.01, real=0.08 secs]
[Full GC (Ergonomics) [PSYoungGen: 345088K->345072K(694272K)] [ParOldGen: 2097094K->2097094K(2097152K)] 2442182K->2442167K(2791424K), [Metaspace: 59102K->59102K(1101824K)], 0.0737145 secs] [Times: user=0.63 sys=0.00, real=0.07 secs]
[Full GC (Ergonomics) [PSYoungGen: 345088K->345072K(694272K)] [ParOldGen: 2097094K->2097094K(2097152K)] 2442182K->2442167K(2791424K), [Metaspace: 59102K->59102K(1101824K)], 0.0729195 secs] [Times: user=0.51 sys=0.00, real=0.07 secs]
[Full GC (Ergonomics) [PSYoungGen: 345088K->345072K(694272K)] [ParOldGen: 2097094K->2097094K(2097152K)] 2442182K->2442167K(2791424K), [Metaspace: 59102K->59102K(1101824K)], 0.0822866 secs] [Times: user=0.68 sys=0.00, real=0.09 secs]
[Full GC (Ergonomics) [PSYoungGen: 345088K->345072K(694272K)] [ParOldGen: 2097094K->2097094K(2097152K)] 2442182K->2442167K(2791424K), [Metaspace: 59102K->59102K(1101824K)], 0.0788580 secs] [Times: user=0.66 sys=0.00, real=0.08 secs]
[Full GC (Ergonomics) [PSYoungGen: 345088K->345072K(694272K)] [ParOldGen: 2097094K->2097094K(2097152K)] 2442182K->2442167K(2791424K), [Metaspace: 59102K->59102K(1101824K)], 0.0719421 secs] [Times: user=0.58 sys=0.00, real=0.07 secs]
[Full GC (Ergonomics) [PSYoungGen: 345088K->345072K(694272K)] [ParOldGen: 2097094K->2097094K(2097152K)] 2442182K->2442167K(2791424K), [Metaspace: 59102K->59102K(1101824K)], 0.0724458 secs] [Times: user=0.60 sys=0.00, real=0.08 secs]
[Full GC (Ergonomics) [PSYoungGen: 345088K->345072K(694272K)] [ParOldGen: 2097094K->2097094K(2097152K)] 2442182K->2442167K(2791424K), [Metaspace: 59102K->59102K(1101824K)], 0.0724485 secs] [Times: user=0.60 sys=0.00, real=0.07 secs]
[Full GC (Ergonomics) [PSYoungGen: 345088K->345072K(694272K)] [ParOldGen: 2097094K->2097094K(2097152K)] 2442182K->2442167K(2791424K), [Metaspace: 59102K->59102K(1101824K)], 0.0744425 secs] [Times: user=0.62 sys=0.00, real=0.07 secs]
[Full GC (Ergonomics) [PSYoungGen: 345088K->345072K(694272K)] [ParOldGen: 2097094K->2097094K(2097152K)] 2442182K->2442167K(2791424K), [Metaspace: 59102K->59102K(1101824K)], 0.0751457 secs] [Times: user=0.63 sys=0.01, real=0.07 secs]
[Full GC (Ergonomics) [PSYoungGen: 345088K->345072K(694272K)] [ParOldGen: 2097094K->2097094K(2097152K)] 2442182K->2442167K(2791424K), [Metaspace: 59102K->59102K(1101824K)], 0.0697133 secs] [Times: user=0.51 sys=0.00, real=0.06 secs]
Heap
PSYoungGen total 694272K, used 345080K [0x0000000780000000, 0x00000007bfb80000, 0x00000007c0000000)
eden space 345088K, 99% used [0x0000000780000000,0x00000007950fe270,0x0000000795100000)
from space 349184K, 0% used [0x00000007aa600000,0x00000007aa600000,0x00000007bfb00000)
to space 349184K, 0% used [0x0000000795100000,0x0000000795100000,0x00000007aa600000)
ParOldGen total 2097152K, used 2097047K [0x0000000700000000, 0x0000000780000000, 0x0000000780000000)
object space 2097152K, 99% used [0x0000000700000000,0x000000077ffe5e78,0x0000000780000000)
Metaspace used 59114K, capacity 59888K, committed 60672K, reserved 1101824K
class space used 7017K, capacity 7233K, committed 7424K, reserved 1048576K