原创:扣钉日记(微信公众号ID :codelogs),接口竟又欢迎分享 ,接口竟又转载请保留出处。接口竟又
继上次我们JVM停顿十几秒的接口竟又问题解决后,我们系统终于稳定了,接口竟又再也不会无故重启了!接口竟又
这是接口竟又之前的文章:耗时几个月,终于找到了JVM停顿十几秒的接口竟又原因
但有点奇怪的是,每隔一段时间,接口竟又我们服务接口就会有一小波499超时 ,接口竟又经过查看gc日志 ,接口竟又又发现JVM停顿了好几秒!接口竟又
有了上次JVM停顿排查经验后,接口竟又我马上就检查了gc日志与safepoint日志,接口竟又发现如下日志 :
$ cat gc-*.log | awk '/application threads were stopped/ && $(NF-6)>1'|tailn2022-05-08T16:40:53.886+0800: 78328.993: Total time for which application threads were stopped: 9.4917471 seconds,接口竟又 Stopping threads took: 9.3473059 secondsn2022-05-08T17:40:32.574+0800: 81907.681: Total time for which application threads were stopped: 3.9786219 seconds, Stopping threads took: 3.9038683 secondsn2022-05-08T17:41:00.063+0800: 81935.170: Total time for which application threads were stopped: 1.2607608 seconds, Stopping threads took: 1.1258499 secondsnn$ cat safepoint.log | awk '/vmop/{ title=$0;getline;if($(NF-2)+$(NF-4)>1000){ print title;print $0}}'n vmop [threads: total initially_running wait_to_block] [time: spin block sync cleanup vmop] page_trap_countn78319.500: G1IncCollectionPause [ 428 0 2 ] [ 0 9347 9347 7 137 ] 0n vmop [threads: total initially_running wait_to_block] [time: spin block sync cleanup vmop] page_trap_countn81903.703: G1IncCollectionPause [ 428 0 4 ] [ 0 3903 3903 14 60 ] 0n vmop [threads: total initially_running wait_to_block] [time: spin block sync cleanup vmop] page_trap_countn81933.906: G1IncCollectionPause [ 442 0 1 ] [ 0 1125 1125 8 126 ] 0n
从日志上可以看到 ,JVM停顿也是由safepoint导致的,而safepoint耗时主要在block阶段!
通过添加JVM参数-XX:+SafepointTimeout -XX:SafepointTimeoutDelay=1000后,可打印出哪些线程超过1000ms没有到达safepoint,如下: