MAPREDUCE服务 MRS-重启MRS集群组件:组件重启参考信息

时间:2024-08-18 05:11:16

组件重启参考信息

当前 MRS 集群中,服务和实例是否支持滚动重启如表2所示。

表2 服务和实例是否支持滚动重启

服务

实例

是否支持滚动重启

Alluxio

AlluxioJobMaster

AlluxioMaster

ClickHouse

ClickHouseServer

ClickHouseBalancer

CDL

CDLConnector

CDLService

Flink

FlinkResource

FlinkServer

Flume

Flume

MonitorServer

Guardian

TokenServer

HBase

HMaster

RegionServer

ThriftServer

RESTServer

HetuEngine

HSBroker

HSConsole

HSFabric

QAS

HDFS

NameNode

Zkfc

JournalNode

HttpFS

DataNode

Hive

MetaStore

WebHCat

HiveServer

Hue

Hue

Impala

Impalad

StateStore

Catalog

IoTDB

IoTDBServer

Kafka

Broker

KafkaUI

Kudu

KuduTserver

KuduMaster

Loader

Sqoop

Mapreduce

JobHistoryServer

Oozie

oozie

Presto

Coordinator

Worker

Ranger

RangerAdmin

UserSync

TagSync

Spark

JobHistory

JD BCS erver

SparkResource

Storm

Nimbus

UI

Supervisor

Logviewer

Tez

TezUI

Yarn

ResourceManager

NodeManager

ZooKeeper

Quorumpeer

实例启动时长如表3所示:

表3 重启时长参考

服务名称

重启时长

启动时长

附加说明

IoTDB

3min

IoTDBServer:3min

-

CDL

2min

  • CDLConnector:1min
  • CDLService:1min

-

ClickHouse

4min

  • ClickHouseServer:2min
  • ClickHouseBalancer:2min

-

HDFS

10min+x

  • NameNode:4min+x
  • DataNode:2min
  • JournalNode:2min
  • Zkfc:2min

x为NameNode元数据加载时长,每千万文件大约耗时2分钟,例如5000万文件x为10分钟。

由于受DataNode数据块上报影响启动时间有一定浮动。

Yarn

5min+x

  • ResourceManager:3min+x
  • NodeManager:2min

x为ResourceManager保留任务数恢复时长,每1万保留任务大约需要1分钟。

Mapreduce

2min+x

JobHistoryServer:2min+x

x为历史任务扫描时长,每10万任务大约2.5分钟。

ZooKeeper

2min+x

quorumpeer:2min+x

x为加载znode节点时长,每100万znode大约1分钟。

Hive

3.5min

  • HiveServer:3min
  • MetaStore:1min30s
  • WebHcat:1min
  • Hive整体服务:3min

-

Spark2x

5min

  • JobHistory2x:5min
  • SparkResource2x:5min
  • JDB CS erver2x:5min

-

Flink

4min

  • FlinkResource:1min
  • FlinkServer:3min

-

Kafka

2min+x

  • Broker:1min+x
  • KafkaUI:5 min

x为数据恢复时长,单实例20000 partition启动所需时长大约2分钟。

Storm

6min

  • Nimbus:3mins
  • UI:1min
  • Supervisor:1min
  • Logviewer:1min

-

Flume

3min

  • Flume:2 min
  • MonitorServer:1min

-

Doris

2分钟

  • FE:1min
  • BE:1min
  • DBroker:1min

-

support.huaweicloud.com/usermanual-mrs/mrs_01_0628.html