MAPREDUCE服务 MRS-spark-sql退出时打印RejectedExecutionException异常栈:问题

时间:2024-07-02 16:39:47

问题

执行大数据量的Spark任务(如2T的TPCDS测试套),任务运行成功后,在spark-sql退出时概率性出现RejectedExecutionException的异常栈信息,相关日志如下所示:

16/07/16 10:19:56 ERROR TransportResponseHandler: Still have 2 requests outstanding when connection from linux-192/10.1.1.5:59250 is closed
java.util.concurrent.RejectedExecutionException: Task scala.concurrent.impl.CallbackRunnable@5fc1ab rejected from java.util.concurrent.ThreadPoolExecutor@52fa7e19[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 3025]
support.huaweicloud.com/cmpntguide-lts-mrs/mrs_01_2037.html