This is documentation for MapR Version 5.0. You can also refer to MapR documentation for the latest release.

Skip to end of metadata
Go to start of metadata

To use MapR Metrics, set up a MySQL database to log metrics data. The MapR distribution for Apache Hadoop does not include MySQL. Download and install MySQL separately and then perform the configuration steps to enable the MapR Metrics database.

Requirements

  • Enable remote access on the MySQL server. All JobTracker and webserver nodes in the cluster must have access to the MySQL server.
  • Use MySQL version 5.1 or greater.
  • Install the mapr-metrics package on all JobTracker and webserver nodes.

Installation

Install mysql server from the EPEL repository.

Icon

The MapR Job Metrics service depends on version 16 of the MySQL client library. SUSE 11 systems have version 15 by default. On SUSE systems, manually install version 16 of the MySQL client library by following these steps:

  1. Set up the repository by creating a file named openSUSE-11.4-11.4-0.repo in the /etc/zypp/repos.d directory with the following text:

    [openSUSE-11.4-11.4-0]
    name=openSUSE-11.4-11.4-0
    enabled=1
    autorefresh=0
    baseurl=http://download.opensuse.org/distribution/11.4/repo/oss/
    type=yast2

  2. Issue the following commands to install the library:
    zypper clean
    zypper refresh
    zypper install libmysqlclient16

Initial Configuration

  1. Start the MySQL server:

    # /etc/init.d/mysqld start 

     

  2. Set a password for the MySQL root user.

    # mysqladmin -u root password <new password>
    Icon

    MySQL passwords cannot contain the reserved character &.

  3. Log into MySQL and create a new database and schema for use with MapR Metrics.

    # mysql -u root -p 
    Enter password:
    mysql> CREATE DATABASE metrics;
    mysql> SHOW DATABASES;
  4. Create a 'maprmetrics' user, grant the user privileges, and verify that the user has the granted privileges.

    For remote SQL servers, provide the following grants:

    For SQL servers on a local node with the metrics role installed, provide the following grants:

  5. Create the maprmetrics schema.

  6. Restart the hoststats service.

Icon

The MapR Metrics database uses the InnoDB table storage engine by default. The Metrics database uses transactional tables, which require a storage engine that supports locking to prevent data corruption. To change the storage engine, edit the following line in the setup.sql script:
SET storage_engine='InnoDB';

To display a list of the available storage engines, issue the SHOW ENGINES command from the mysql> prompt.

Client Installation

Install the MySQL client on JobTracker and webserver nodes in your cluster.

  1. Test the MySQL connection from the JobTracker and webserver nodes to the MySQL server:

    # mysql -u maprmetrics -h <hostname or IP where sql server is running> -p 
    Enter password:	

Troubleshooting the Client

  • Check that the port used by the MySQL server is set to the default of 3306. To use a non-default port, specify the port number with the -P option for the mysql command.
  • Check the /etc/my.cnf file to verify that the following lines are not present:

  • Restart the MySQL server after any changes to the /etc/my.cnf:

    # /etc/init.d/mysqld restart

Specify MySQL parameters to MapR

  1. On each node in the cluster that has the mapr-metrics package installed, specify your MySQL database parameters in one of the following ways:
  • To specify the MySQL database parameters from the command line, run the configure.sh script:

  • To specify the MySQL database parameters from the MapR Control System (MCS), click Navigation > System Settings > Metrics to display the Configure Metrics Database dialog. In the URL field, enter the hostname and port of the machine running the MySQL server. In the Username and Password fields, enter the username and password of the MySQL user. Schema is set to metrics by default.
Icon

The username you provide must have full permissions when logged in from any node in the cluster.
When you change the Metrics configuration information from the initial settings, you must restart the hoststats service on each node that reports Metrics data. You can restart the hoststats service with the command maprcli node services -name hoststats -action restart.

Customizing the Database Name

You can change the name of the database from the default name of metrics by editing the setup.sql script before sourcing the script.

Troubleshooting Metrics

  • Verify that the mapr-metrics package is installed on all JobTracker and webserver nodes.
  • Verify that all nodes have the correct configuration information in the /opt/mapr/conf/db.conf and /opt/mapr/conf/hibernate.cfg.xml files.
  • Check the /opt/mapr/logs/adminuiapp.log file for database-related messages or errors similar to this example:

  • Verify that the hoststats service is running:

  • Check the /opt/mapr/logs/hoststats.log file for database-related errors.
  • Verify that the /opt/mapr/hadoop/hadoop-0.20.2/conf/hadoop-metrics.properties file exists and contains the following entries:

    Add these entries if they are absent, then restart the JobTracker.

  • Verify that the /opt/mapr/conf/warden.conf file has the following entries:

  • Check the JobTracker logs for errors or entries related to the string com.mapr.job.mngmnt.hadoop.metrics.
  • No labels