操作步骤

1. 获取运行在Yarn上的任务的具体信息。
   • 命令：

     curl -k -i --negotiate -u : "https://10-120-85-2:8090/ws/v1/cluster/apps/"

     其中10-120-85-2为ResourceManager主节点的hostname，8090为ResourceManager的端口号。

   • 用户能看到哪个队列的任务，要看这个用户是否有这个队列的admin权限。
     

     如果当前组件使用了Ranger进行权限控制，需基于Ranger配置相关策略进行权限管理。

   • 运行结果：

     {    "apps": {        "app": [            {                "id": "application_1461743120947_0001",                "user": "spark",                "name": "Spark-JD BCS erver",                "queue": "default",                "state": "RUNNING",                "finalStatus": "UNDEFINED",                "progress": 10,                "trackingUI": "ApplicationMaster",                "trackingUrl": "https://10-120-85-2:8090/proxy/application_1461743120947_0001/",                "diagnostics": "AM is launched. ",                "clusterId": 1461743120947,                "applicationType": "SPARK",                "applicationTags": "",                "startedTime": 1461804906260,                "finishedTime": 0,                "elapsedTime": 6888848,                "amContainerLogs": "https://10-120-85-2:8044/node/containerlogs/container_e12_1461743120947_0001_01_000001/spark",                "amHostHttpAddress": "10-120-85-2:8044",                "allocatedMB": 1024,                "allocatedVCores": 1,                "runningContainers": 1,                "memorySeconds": 7053309,                "vcoreSeconds": 6887,                "preemptedResourceMB": 0,                "preemptedResourceVCores": 0,                "numNonAMContainerPreempted": 0,                "numAMContainerPreempted": 0,                "resourceRequests": [                    {                        "capability": {                            "memory": 1024,                            "virtualCores": 1                        },                        "nodeLabelExpression": "",                        "numContainers": 0,                        "priority": {                            "priority": 0                        },                        "relaxLocality": true,                        "resourceName": "*"                    }                ],                "logAggregationStatus": "NOT_START",                "amNodeLabelExpression": ""            },            {                "id": "application_1461722876897_0002",                "user": "admin",                "name": "QuasiMonteCarlo",                "queue": "default",                "state": "FINISHED",                "finalStatus": "SUCCEEDED",                "progress": 100,                "trackingUI": "History",                "trackingUrl": "https://10-120-85-2:8090/proxy/application_1461722876897_0002/",                "diagnostics": "Attempt recovered after RM restart",                "clusterId": 1461743120947,                "applicationType": "MAPREDUCE",                "applicationTags": "",                "startedTime": 1461741052993,                "finishedTime": 1461741079483,                "elapsedTime": 26490,                "amContainerLogs": "https://10-120-85-2:8044/node/containerlogs/container_e11_1461722876897_0002_01_000001/admin",                "amHostHttpAddress": "10-120-85-2:8044",                "allocatedMB": -1,                "allocatedVCores": -1,                "runningContainers": -1,                "memorySeconds": 158664,                "vcoreSeconds": 52,                "preemptedResourceMB": 0,                "preemptedResourceVCores": 0,                "numNonAMContainerPreempted": 0,                "numAMContainerPreempted": 0,                "amNodeLabelExpression": ""            }        ]    }}

   • 结果分析：
     通过这个接口，可以查询当前集群中Yarn上的任务，并且可以得到如下表1。

     表1 常用信息

     参数	参数描述
     user	运行这个任务的用户。
     applicationType	例如MAPREDUCE或者SPARK等。
     finalStatus	可以知道任务是成功还是失败。
     elapsedTime	任务运行的时间。

2. 获取Yarn资源的总体信息。
   • 命令：

     curl -k -i --negotiate -u : "https://10-120-85-102:8090/ws/v1/cluster/metrics"

   • 运行结果：

     {    "clusterMetrics": {        "appsSubmitted": 2,        "appsCompleted": 1,        "appsPending": 0,        "appsRunning": 1,        "appsFailed": 0,        "appsKilled": 0,        "reservedMB": 0,        "availableMB": 23552,        "allocatedMB": 1024,        "reservedVirtualCores": 0,        "availableVirtualCores": 23,        "allocatedVirtualCores": 1,        "containersAllocated": 1,        "containersReserved": 0,        "containersPending": 0,        "totalMB": 24576,        "totalVirtualCores": 24,        "totalNodes": 3,        "lostNodes": 0,        "unhealthyNodes": 0,        "decommissionedNodes": 0,        "rebootedNodes": 0,        "activeNodes": 3,        "rmMainQueueSize": 0,        "schedulerQueueSize": 0,        "stateStoreQueueSize": 0    }}

   • 结果分析：
     通过这个接口，可以查询当前集群中如表2。

     表2 常用信息

     参数	参数描述
     appsSubmitted	已经提交的任务数。
     appsCompleted	已经完成的任务数。
     appsPending	正在挂起的任务数。
     appsRunning	正在运行的任务数。
     appsFailed	已经失败的任务数。
     appsKilled	已经被kill的任务数。
     totalMB	Yarn资源总的内存。
     totalVirtualCores	Yarn资源总的VCore数。