This is documentation for MapR Version 5.0. You can also refer to MapR documentation for the latest release.

Skip to end of metadata
Go to start of metadata

This procedure configures Spark to use the mounted NFS directory instead of the /tmp directory on the local file system:

  1. Install the mapr-nfs and nfs-utils packages if they are not already installed. For reference, see Get Started: Install MapR.
  2. Start the NFS service by using the MapR Control System (MCS) or maprcli (NFS gateway): 

  3. Mount NFS to MapR-FS on a cluster node by following the steps in Mounting a Single-Node Cluster Via NFS.
  4. To configure Spark Shuffle on NFS, complete these steps on all nodes:
    1. Create a local volume for Spark Shuffle: 

    2. Point the NodeManager local directory to the Spark Shuffle volume mounted through NFS by setting the following property in the yarn-site.xml file on the NodeManager nodes: 

    3. Restart the NodeManager service (and the Resource Manager service on the main node) to pick up the yarn-site.xml changes: 

 

  • No labels