This procedure configures Spark to use the mounted NFS directory instead of the
/tmp directory on the local file system:
- Install the
nfs-utilspackages if they are not already installed. For reference, see Get Started: Install MapR.
Start the NFS service by using the MapR Control System (MCS) or maprcli (NFS gateway):
- Mount NFS to MapR-FS on a cluster node by following the steps in Mounting a Single-Node Cluster Via NFS.
- To configure Spark Shuffle on NFS, complete these steps on all nodes:
Create a local volume for Spark Shuffle:
Point the NodeManager local directory to the Spark Shuffle volume mounted through NFS by setting the following property in the
yarn-site.xmlfile on the NodeManager nodes:
Restart the NodeManager service (and the Resource Manager service on the main node) to pick up the