This site contains release notes for MapR Version 5.0 and below.  You can also refer to the release notes for the latest release.

Skip to end of metadata
Go to start of metadata

This page contains the following topics:

New In This Release

  • New Feature: Wire-Level Security. In this release, the MapR Distribution for Hadoop provides secure authentication and encrypted data transmission for MapR clusters.
  • New Feature: Security for MapR Tables. Access Control Expressions (ACEs) control user authorization for operations on MapR tables and column families.
  • Change: Installation improvements. A new set of installer scripts provides a smoother install experience.
  • Change: ZooKeeper version has been updated to 3.4.5 in order to support security features and add support for new APIs for several cluster components.
  • Change: The default MapReduce slot computation algorithm has been updated to improve scalability and performance.

WebHDFS Support

The 3.1 release of the MapR distribution for Hadoop supports the WebHDFS client,  enabling access to HDFS in Apache clusters from the MapR cluster. The WebHDFS server is not supported. The MapR file system cannot be accessed with the webhdfs:// protocol.

Resolved Issues

Resolved issues are listed in the sections below.

General

  • (Issue 7618) The hadoop mfs -lsd command now correctly displays volumes as links instead of directories.
  • (Issue 8098) Improved documentation for resolving the INODES_EXCEEDED alarm.
  • (Issue 9658) Fixed an error condition that occasionally prevented configuration information in mapred-site.xml from loading.
  • (Issue 9900) The post-install script now runs configure.sh -R after removal of any of the following packages to perform post-uninstall tasks.
    • mapr-cldb
    • mapr-fileserver
    • mapr-jobtracker
    • mapr-tasktracker
    • mapr-nfs
    • mapr-webserver
    • mapr-zookeeper 
  • (Issue 10610) Changed handling of orphan tasks to prevent excessive logging.
  • (Issue 10665) Added hadoop mfs -lssr command to recursively list directories.
  • (Issue 10722) Improved autodetection of installed Java versions and automated setting of the $JAVA_HOME environment variable. Also addresses Issue 7910.
  • (Issue 10735) The configure.sh script no longer adds extraneous mfs.cache.lru.sizes lines to the mfs.conf file with each execution.
  • (Issue 11002) The hadoop.bat script on Windows now allows more than nine parameters.
  • (Issue 11201) The MapR Control System and the maprcli disk list command now show the storage pool to which a given disk belongs.
  • (Issue 11635) The configure.sh script now correctly sets the permissions of the maprexecute binary when when central configuration is active.
  • (Issue 11778) Removing a license from a cluster no longer causes services unaffected by that license to go down.
  • (Issue 11968) Eliminated a hoststats crash. Also addresses Issue 11826.
  • (Issue 12005) Files in the conf directory and all open source components no longer have the execute permission set.
  • (Issue 12047) Corrected an error in the Oozie shell example script that prevented the example from running.
  • (Issue 12104) Changed the memory management schema to address a hoststats core generation issue.
  • (Issue 12552) Added the file identifier (FID) to the logged information for error messages.
  • (Issue 12649) The rpm utility now replaces the *.conf files under the conf/conf.d directory when you upgrade Hive, Oozie, HTTPFS, or HUE.

MapR Control System

  • (Issue 9324) The default query for the Job Management display has been restricted to a smaller number of results to prevent out-of-memory conditions.
  • (Issue 10356) Error for the actual replication value of a volume now displays in the correct region of the MCS.
  • (Issue 10982) GUI refactored to improve handling of long scrolling lists and checkbox alignment.
  • (Issue 11868) Users with the - character in the username can now be added to ACLs normally.
  • (Issue 12251) Webserver updated to handle Job Management queries with over 10000 results.

MapR Command-Line Interface

  • (Issue 10427) The volume create rootdirperms command now supports setting the setgroupid (SGID) bit.
  • (Issue 10655) Added the hadoop mfs -lssr command.
  • (Issue 10889) The hadoop fs -test -e command processes wildcard matching correctly.

MapR File System

  • (Issue 9707) MFS logs for failed disks include the reason why the disk was marked as failed.
  • (Issue 9014) The mrconfig cntr movereplica command has been added to move a container from one node to another.
  • (Issue 10314) Eliminated spurious reports of zero disk space available, triggering the NODE_ALARM_TT_LOCALDIR_FULL alarm.
  • (Issue 11771) Eliminated a condition where rearranging node topology resulted in spurious VOLUME_ALARM_NO_NODES_IN_TOPOLOGY alarms .

MapR Metrics

  • (Issue 12361) You can now explicitly specify the storage engine used by the Metrics database. The default engine is InnoDB.

MapR Tables

  • (Issue 7700) Suppressed spurious warnings about MAPR_SUBNETS appearing in commands unrelated to that functionality.
  • (Issue 10236) The 32-bit client now handles puts and gets involving long integers correctly.
  • (Issue 11667) Improved handling of compactions and space allocation for MapR tables.
  • (Issue 12062) Table mapping rules instantiate correctly while querying data from MapR tables.
  • (Issue 12615) Addressed by Issues 12664, 12665, and 12666, which improve data integrity for MapR tables, implement a failsafe in the database layer to prevent data corruption, and adds a tool to assist with data recovery.

MapReduce

  • (Issue 8239) Enhances the way task attempts report their status to the cluster. The cluster can now differentiate between an active task attempt that is not making progress and a completely unresponsive task attempt. New parameters in mapred-site.xml enable you to specify the inactivity timeout that marks a task as inactive and the reporting interval that running tasks use to send progress or heartbeats to the TaskTracker.
  • (Issue 8282) When a TaskTracker is blacklisted, the job ID that triggered the blacklisting is now present in the blacklisting log message.
  • (Issue 8686) The message Too many RPC requests for metrics processing. Start dropping them has had its alert level changed from WARN to DEBUG to prevent flooding the log with messages.
  • (Issue 9260) Revises the MapReduce slot computation algorithm to increase scalability and performance.
     
  • (Issue 9684) Task attempt logs for a given task are no longer split across into the logs for the following task.
  • (Issue 9732) Fixes a task JVM hang when Central Logging is enabled.
  • (Issue 10570) Additional logging for TaskTracker to provide more information for debugging and support.
  • (Issue 10863) Debug logging for JobTracker output now includes source or destination paths.
  • (Issue 11065) A new property in mapred-site.xml enables administrators to specify an upper bound for the user-configurable number of hours for which to retain job logs.
  • (Issue 11075) A new property in mapred-site.xml enables users to specify the number of records to process during a merge before sending a progress notification to the TaskTracker.

  • (Issue 11342) Incorporates MAPREDUCE-5047.
  • (Issue 11425) Default retention of Metrics data changed to 30 days from 365. New parameter in db.conf introduced to clean up metrics data based on total disk space used.
  • (Issue 11780) Eliminated spurious raising of the NODE_ALARM_TT_LOCALDIR_FULL alarm .
  • (Issue 11952) Changes to the configure-common.sh script run by eliminate a condition where running configure.sh while the TaskTracker was running made all future job initialization fail.
  • (Issue 12358) Fixes a JobTracker Null Pointer Exception.
  • (Issue 12437) Reduced JobTracker lock time with FairScheduler to prevent deadlocks.

FileClient

  • (Issue 9987) The delete() call no longer throws FileNotFoundException when used to delete a non-existing file, bringing its behavior in line with the Java API.
  • (Issue 10391) API improvements to enable Java applications to query multiple clusters.
  • (Issue 10857) Corrected text for error messages reported by FileClient.
  • (Issue 11479) Added support for providing MapR functionality through fatJARs.
  • (Issue 11813) The Windows FileClient now prints the thread ID when debug logging is enabled.
  • (Issue 12264) Addresses a loop condition where the FileClient was unable to send write requests to a master container.

FileServer

  • (Issue 7089) Read operations release global locks earlier, improving performance for renames of large numbers of files or directories.
  • (Issue 8696) Bad block information is now logged to mfs.err in addition to mfs.log to increase the amount of historical information logged.
  • (Issue 10672) Disk setup no longer fails with a permission denied error when you run configure.sh with the disk add option.
  • (Issue 11367) Fixed an issue where incorrect permissions on a file's inode caused read failures.
  • (Issue 12244) Fixes an integer overflow condition that shortened MFS log retention.
  • (Issue 12286) The fileserver no longer sends an old heartbeat message to the CLDB after establishing a new connection.
  • (Issue 12357) Addresses an MFS crash condition related to adding new disks to a node.

Security

  • (Issue 11953) Scripts that call the chownchgrp, and chmod utilities now check that the targets of those utilities aren't symbolic links.

Warden 

  • (Issue 10132) The wardeninit.log has been consolidated into warden.log.
  • (Issue 10734) The Warden does not run the createsystemvolumes.sh script at startup on a node that does not have the Fileserver role configured.
  • (Issue 10898) It is no longer possible to launch multiple instances of the Warden.
  • (Issue 11844) The Warden now starts the CLDB service automatically when the CLDB role is added to the node.
  • (Issue 12001) The file warden.pid is now removed when Warden stops.

NFS

  • (Issue 11494) The NFS server now reads up to 2048 characters per entry in the exports file, instead of 128, to accommodate 255 characters for the maximum path length and up to 100 15-character IP addresses.
  • (Issue 12097) Addresses a segmentation fault on the NFS server.
  • (Issue 12371) NFS no longer logs error messages for a normal operating condition.

CLDB

  • (Issue 10148) Adds the -service option to the maprcli node remove command to enable removing the fileserver and nfsserver roles from a node individually. This prevents a condition where nodes that have been renamed have the same fileserver ID.
  • (Issue 11021) The RPC timeout value for RPCs involving disk removal has been increased to improve CLDB responsiveness.
  • (Issue 11456) Oozie no longer fails during upgrade if you upgrade from an older version of MapR to MapR version 3.1.0 or later with Oozie version 3.3.2-1401.
  • (Issue 11514) Corrected the settings for raising the CLUSTER_ALARM_CLDB_HEAPSIZE alarm, eliminating spurious reports.
  • (Issue 12352) The CLDB no longer stops unexpectedly with a container failure notice caused by failed kvstore heartbeats.
  • (Issue 12388) The CLDB no longer stops unexpectedly when new CLDB nodes are not in the same mapr.cldb.internal volume topology.

ZooKeeper

  • (Issue 11550) A message now appends to the zookeeper.log file after the ZooKeeper has been administratively killed.
  • (Issue 12245) After upgrading ZooKeeper to 3.4.5, the snapshot cleanup interval is tunable by users. The cron job that handled snapshot cleanup has been removed.

HBase

  • (Issue 10625) The Hadoop class path is now updated to correctly incorporate the JAR files for ZooKeeper 3.4.5.

MapReduce and Hadoop Patches Integrated Since Last Release

MapR 3.1 includes the following Apache MapReduce and Apache Hadoop patches:

  • [MAPREDUCE-2187] Large map tasks no longer time out during the merge sort phase.
  • [MAPREDUCE-5047] MapReduce tasks launched by a user other than the user running TaskTracker no longer fail when the value of the keep.failed.task.files parameter is set to true.

Server Configuration Support

Operating System

Version

Binary Type

JDK

HBase

Hive

Flume

Sqoop

Mahout

Oozie

Pig

Whirr

HUE1HTTPFS

Cascading

Redhat Enterprise Linux

5.7
5.8
5.9
6.0
6.1
6.2
6.3
6.4

64-bit

Oracle JDK 1.6
Oracle JDK 1.7
Open JDK 1.6
Open JDK 1.7

94.13

0.11 (HS2)
0.12 (HS2)

1.4.0

1.4.4

0.7
0.8

3.3.2

0.11

0.8.1

2.51.0

2.1

CentOS

5.7
5.8
5.9
6.0
6.1
6.2
6.3
6.4

64-bit

Oracle JDK 1.6
Oracle JDK 1.7
Open JDK 1.6
Open JDK 1.7

94.13

0.11 (HS2)
0.12 (HS2)

1.4.0

1.4.4

0.7
0.8

3.3.2

0.11

0.8.1

2.51.0

2.1

Suse Linux Enterprise Server

11 SP1
11 SP2

64-bit

Oracle JDK 1.6
Oracle JDK 1.7
Open JDK 1.6
Open JDK 1.7

94.13

0.11 (HS2)
0.12 (HS2)

1.4.0

1.4.4

0.7
0.8

3.3.2

0.11

0.8.1

2.51.0

2.1

Ubuntu

9.04
9.10
10.04
10.10
11.04
11.10
12.04

64-bit

Oracle JDK 1.6
Oracle JDK 1.7
Open JDK 1.6
Open JDK 1.7

94.13

0.11 (HS2)
0.12 (HS2)

1.4.0

1.4.4

0.7
0.8

3.3.2

0.11

0.8.1

2.51.0

2.1

Amazon EMR2

 

64-bit

Oracle JDK 1.6
Oracle JDK 1.7
Open JDK 1.6
Open JDK 1.7

94.13

0.11 (HS2)
0.12 (HS2)

1.4.0

1.4.4

0.7
0.8

3.3.2

0.11

0.8.1

2.51.0

2.1

1 Hue 2.5 does not support secure connections to the cluster. Hue 2.5 is only supported for Red Hat/CentOS versions 6.0 and later.

2 Amazon EMR does not support wire-level security.

Client Configuration Support

Operating System

Version

Binary Type

Windows

7, 2008, v8

32bit

Windows

7, 2008, v8

64bit

Linux

Server Version

64bit

Mac OS X

10.8.2

64bit

Browser Support

Browser

Version

Firefox

19

IE

10

Chrome

26+

Safari

6.0.2

  • No labels