Configuring opportunist container

The opportunist container configuration goes into YARN-site.xml. The first step is to enable opportunist container allocation, as follows: 

<properties>
    <name>YARN.resourcemanager.opportunistic-container-allocation.enabled</name>
     <value>true</value>
</properties>

The minimum number of opportunist containers that can be queued at a Node Manager is determined by the following property:

<properties>
 <name>YARN.nodemanager.opportunistic-containers-max-queue-length</name>
 <value>15</value>
</properties>

An efficient value is determined by job characteristics, the cluster configuration, and the target utilization. The allocation can happen using either a centralized or distributed node. By default, YARN uses the centralized way to allocate resources. But if we want to enable the distributed allocation method, we can enable the following property:

<properties>
     <name>YARN.nodemanager.distributed-scheduling.enabled</name>
     <value>true</value>
</properties>

The following parameter tells us the amount of memory that can be used while submitting the job to indicate the percentage of mappers that can run using an opportunist container: 

-Dmapreduce.job.num-opportunistic-maps-percent="30"