This is documentation for MapR Version 5.0. You can also refer to MapR documentation for the latest release.

Skip to end of metadata
Go to start of metadata

When you run configure.sh on a node, warden allocates memory for the operating system, mfs service, MapR Hadoop services, and applications using the settings in the warden.conf and warden.<servicename>.conf file.  

Warden allocates memory to the following components in the following order:

  1. Operating system
  2. mfs service
  3. MapR Hadoop services
  4. Jobs and Applications, such as MapReduce v1 or YARN applications
  5. If NodeManager is not installed on the node, the remaining available memory is allocated to the MapR file system. If NodeManager is installed on the node, the remaining available memory is allocated to run YARN applications.
Icon

In general, you should not modify the settings in the warden files. However, if you modify the values in warden.conf or a warden.<servicename>.conf file, you must restart warden. Otherwise, updated parameters will not be used to allocate resources

Memory for the OS, mfs Service, and Hadoop Services

The warden allocates memory to the operating system, mfs service, and MapR Hadoop services based on the following parameters:

ParameterDefaultDescription
 OSmfs serviceHadoop Service(s) 


service.command.<os|mfs|servicename>.heapsize.percent

10

varies

variesDefines the heap size percentage.


service.command. <os|mfs|servicename>.heapsize.max

4000855000Defines the maximum heap size in MB.
service.command. <os|mfs|servicename>.heapsize.min256512256Defines the minimum heap size in MB.

The memory settings for the operating system, mfs service, and Hadoop services such as TaskTracker and JobTracker, are configured in the warden.conf file. The warden.conf file is located in /opt/mapr/conf. Other services such as NodeManager and ResourceManager have their own warden.<servicename>.conf file within /opt/mapr/conf/conf.d. For more information about the warden files, see warden.conf and warden.<servicename>.conf.

Note: Warden only allocates resources for MapR Hadoop services associated with roles that are installed on the node.

Memory for the mfs service

By default, Warden adds up the total memory consumed by all services and the OS and then gives 85% of what is left to the mfs service. If you do not intend to use MapR-DB, you can set the -noDB option in configure.sh to specify that 20% of the memory available should be allocated to mfs service. 

Memory for Jobs and Applications

Warden allocates memory to MapReduce v1 jobs and applications based on the services installed on the node and also the mr1.memory.percent value in warden.conf. For more information, see Resource Allocation for Jobs and Applications.


  • No labels