记一次生产频繁发生FullGC问题(多次生产对身体的影响)

  本篇文章为你整理了记一次生产频繁发生FullGC问题(多次生产对身体的影响)的详细内容,包含有多次生产会有什么危险 多次生产对身体的影响 生产次数多,好吗 记一次生产频繁发生FullGC问题,希望能帮助你了解 记一次生产频繁发生FullGC问题。

  问题发现早上过来,饭都没来的及吃,运维就给我发来信息,说是某个接口调用大量超时。因为最近这个接口调用量是翻倍了,所以我就去检查了下慢SQL,发现确实是有较多的慢SQL,所以我就缩减了查询的时间范围,但是效果并不好。过了一会发现,这个服务fullGC是有问题的,太频繁了,这个应该是导致接口超时的根本问题,因为时间也是对的上的。这个是最近三个小时fullGC的监控图:这个是最近三天fullGC的监控图:对比一下,就不难发现,fullGC数量是从3月15号晚上9点开始增加的,也是这个接口对外开放的时间。解决思路1、首先去服务器上面下载dump文件,分析是哪里造成了内存泄漏,频繁触发fullGC。首先找出服务器内java文件的PID,然后保存dump文件,我们公司java服务是固定端口号:1

  使用top命令:然后执行命令:jmap -dump:file=202303160924.dump,format=b 1 ,保存dump文件2、根据dump文件,分析出堆内对象的分布情况

  下载一个可以分析dump文件的工具,这里我下载是Jprofiler查看大对象的分析,发现是java.lang.ApplicationShutdownHooks的hooks占用太大内存,并且得知改熟悉是一个Map分析这个Map里面的元素引用关系,可以看到这个map里面存的都是线程对象,并且大部分都是一个名为java.util.concurrent.ScheduledThreadPoolExecutor@59648a61的线程池对象,到了这里就定位到问题代码了,是这次新加的接口里面有一个异步操作,用的guava并发包里面的一个超时等待功能的接口,具体思路就是启用一个定时任务线程池去定时去检查在规定时间内,是否返回结果。  3、看看我的业务代码是哪里出现了问题

  

//异步执行某个查询方法,并且在规定时间内返回查询结果public T T asyncWithTimeout(ScheduledThreadPoolExecutor executor, Callable T callable, long time, TimeUnit unit) { try { ListeningExecutorService listeningExecutorService = MoreExecutors.listeningDecorator(threadPoolExecutor); ListenableFuture T future = listeningExecutorService.submit(callable); //这里是创建一个定时任务线程,去定时检查是否在规定时间内查询完毕,应该就是这个去添加了钩子函数,进去看看 ScheduledExecutorService scheduledExecutorService = MoreExecutors.getExitingScheduledExecutorService(executor); return Futures.withTimeout(future, time, unit, scheduledExecutorService).get(time, unit); } catch (InterruptedException ExecutionException TimeoutException e) { log.warn("异步方法执行失败,error:{}", e.getMessage()); } return null;}//=======================guava并发包代码=======================@Beta@GwtIncompatible // TODOpublic static ScheduledExecutorService getExitingScheduledExecutorService( ScheduledThreadPoolExecutor executor) { //每次都去创建一个新的对象 return new Application().getExitingScheduledExecutorService(executor);}final ScheduledExecutorService getExitingScheduledExecutorService( ScheduledThreadPoolExecutor executor) { return getExitingScheduledExecutorService(executor, 120, TimeUnit.SECONDS);}final ScheduledExecutorService getExitingScheduledExecutorService( ScheduledThreadPoolExecutor executor, long terminationTimeout, TimeUnit timeUnit) { useDaemonThreadFactory(executor); ScheduledExecutorService service = Executors.unconfigurableScheduledExecutorService(executor); //添加构造函数的地方,进去看看 addDelayedShutdownHook(executor, terminationTimeout, timeUnit); return service;}final void addDelayedShutdownHook( final ExecutorService service, final long terminationTimeout, final TimeUnit timeUnit) { checkNotNull(service); checkNotNull(timeUnit); //继续点进去 addShutdownHook( MoreExecutors.newThread( //线程名字对上了,就在对象引用的截图里面出现过 "DelayedShutdownHook-for-" + service, new Runnable() { @Override public void run() { try { // Wed like to log progress and failures that may arise in the // following code, but unfortunately the behavior of logging // is undefined in shutdown hooks. // This is because the logging code installs a shutdown hook of its // own. See Cleaner class inside {@link LogManager}. service.shutdown(); service.awaitTermination(terminationTimeout, timeUnit); } catch (InterruptedException ignored) { // Were shutting down anyway, so just ignore. } } }));}@VisibleForTestingvoid addShutdownHook(Thread hook) { Runtime.getRuntime().addShutdownHook(hook);}//=======================guava并发包代码=======================public void addShutdownHook(Thread hook) { SecurityManager sm = System.getSecurityManager(); if (sm != null) { sm.checkPermission(new RuntimePermission("shutdownHooks")); } //定位到问题了,就是这里添加的钩子函数 ApplicationShutdownHooks.add(hook);}static synchronized void add(Thread hook) { if(hooks == null) throw new IllegalStateException("Shutdown in progress"); if (hook.isAlive()) throw new IllegalArgumentException("Hook already running"); if (hooks.containsKey(hook)) throw new IllegalArgumentException("Hook previously registered"); //存在到 hooks 这个map对象里面,就是这个大对象 hooks.put(hook, hook);}
问题解决经过上面问题的排查,造成hooks大对象的原因找到了,就是每次调用接口的时候,都会往hooks里面put一个对象。所以,解决办法很简单,就是不用每次都去生成一个ScheduledExecutorService对象,类初始化的时候创建一次就行了改造后的代码如下:
private ListeningExecutorService listeningExecutorService;private ScheduledExecutorService scheduledExecutorService;public static AsyncUtils getInstance() { return ThreadHolder.INSTANCE.getAsyncWithCallback();}@SuppressWarnings("UnstableApiUsage")private AsyncUtils() { listeningExecutorService = MoreExecutors.listeningDecorator(ThreadPoolConstant.THREAD_POOL_EXECUTOR); scheduledExecutorService = MoreExecutors.getExitingScheduledExecutorService(ThreadPoolConstant.SCHEDULED_THREAD_POOL_EXECUTOR);}@SuppressWarnings("UnstableApiUsage")public T T asyncWithTimeout(Callable T callable, long time, TimeUnit unit) { try { ListenableFuture T future = listeningExecutorService.submit(callable); return Futures.withTimeout(future, time, unit, scheduledExecutorService).get(time, unit); } catch (InterruptedException ExecutionException TimeoutException e) { log.warn("异步方法执行失败,error:{}", e.getMessage()); } return null;}private enum ThreadHolder { /** * 线程持有类 INSTANCE */ INSTANCE; private final AsyncUtils asyncUtils; ThreadHolder() { asyncUtils = new AsyncUtils(); } public AsyncUtils getAsyncWithCallback() { return asyncUtils; }}
posted @ 2023-03-16 13:55theonenini

 

  阅读(1259)

  评论(4)

  编辑

  收藏

  举报

  以上就是记一次生产频繁发生FullGC问题(多次生产对身体的影响)的详细内容,想要了解更多 记一次生产频繁发生FullGC问题的内容,请持续关注盛行IT软件开发工作室。

郑重声明:本文由网友发布,不代表盛行IT的观点,版权归原作者所有,仅为传播更多信息之目的,如有侵权请联系,我们将第一时间修改或删除,多谢。

留言与评论(共有 条评论)
   
验证码: