This page contains the following topics:
New In This Release
- New Feature: Wire-Level Security. In this release, the MapR Distribution for Hadoop provides secure authentication and encrypted data transmission for MapR clusters.
- New Feature: . Access Control Expressions (ACEs) control user authorization for operations on MapR tables and column families.
- Change: Installation improvements. A new set of installer scripts provides a smoother install experience.
- Change: ZooKeeper version has been updated to 3.4.5 in order to support security features and add support for new APIs for several cluster components.
- Change: The default MapReduce slot computation algorithm has been updated to improve scalability and performance.
The 3.1 release of the MapR distribution for Hadoop supports the WebHDFS client, enabling access to HDFS in Apache clusters from the MapR cluster. The WebHDFS server is not supported. The MapR file system cannot be accessed with the
Resolved issues are listed in the sections below.
- (Issue 7618) The
hadoop mfs -lsdcommand now correctly displays volumes as links instead of directories.
- (Issue 8098) Improved documentation for resolving the
- (Issue 9658) Fixed an error condition that occasionally prevented configuration information in
- (Issue 9900) The post-install script now runs
configure.sh -Rafter removal of any of the following packages to perform post-uninstall tasks.
- (Issue 10610) Changed handling of orphan tasks to prevent excessive logging.
- (Issue 10665) Added
hadoop mfs -lssrcommand to recursively list directories.
- (Issue 10722) Improved autodetection of installed Java versions and automated setting of the
$JAVA_HOMEenvironment variable. Also addresses Issue 7910.
- (Issue 10735) The configure.sh script no longer adds extraneous mfs.cache.lru.sizes lines to the mfs.conf file with each execution.
- (Issue 11002) The
hadoop.batscript on Windows now allows more than nine parameters.
- (Issue 11201) The MapR Control System and the maprcli disk list command now show the storage pool to which a given disk belongs.
- (Issue 11635) The
configure.shscript now correctly sets the permissions of the
maprexecutebinary when when central configuration is active.
- (Issue 11778) Removing a license from a cluster no longer causes services unaffected by that license to go down.
- (Issue 11968) Eliminated a hoststats crash. Also addresses Issue 11826.
- (Issue 12005) Files in the
confdirectory and all open source components no longer have the
- (Issue 12047) Corrected an error in the Oozie shell example script that prevented the example from running.
- (Issue 12104) Changed the memory management schema to address a hoststats core generation issue.
- (Issue 12552) Added the file identifier (FID) to the logged information for error messages.
- (Issue 12649) The
rpmutility now replaces the
*.conffiles under the
conf/conf.ddirectory when you upgrade Hive, Oozie, HTTPFS, or HUE.
MapR Control System
- (Issue 9324) The default query for the Job Management display has been restricted to a smaller number of results to prevent out-of-memory conditions.
- (Issue 10356) Error for the actual replication value of a volume now displays in the correct region of the MCS.
- (Issue 10982) GUI refactored to improve handling of long scrolling lists and checkbox alignment.
- (Issue 11868) Users with the
-character in the username can now be added to ACLs normally.
- (Issue 12251) Webserver updated to handle Job Management queries with over 10000 results.
MapR Command-Line Interface
- (Issue 10427) The
volume create rootdirpermscommand now supports setting the
- (Issue 10655) Added the
hadoop mfs -lssrcommand.
- (Issue 10889) The
hadoop fs -test -ecommand processes wildcard matching correctly.
MapR File System
- (Issue 9707) MFS logs for failed disks include the reason why the disk was marked as failed.
- (Issue 9014) The
mrconfig cntr movereplicacommand has been added to move a container from one node to another.
- (Issue 10314) Eliminated spurious reports of zero disk space available, triggering the
- (Issue 11771) Eliminated a condition where rearranging node topology resulted in spurious
- (Issue 12361) You can now explicitly specify the storage engine used by the Metrics database. The default engine is InnoDB.
- (Issue 7700) Suppressed spurious warnings about
MAPR_SUBNETSappearing in commands unrelated to that functionality.
- (Issue 10236) The 32-bit client now handles puts and gets involving long integers correctly.
- (Issue 11667) Improved handling of compactions and space allocation for MapR tables.
- (Issue 12062) Table mapping rules instantiate correctly while querying data from MapR tables.
- (Issue 12615) Addressed by Issues 12664, 12665, and 12666, which improve data integrity for MapR tables, implement a failsafe in the database layer to prevent data corruption, and adds a tool to assist with data recovery.
- (Issue 8239) Enhances the way task attempts report their status to the cluster. The cluster can now differentiate between an active task attempt that is not making progress and a completely unresponsive task attempt. New parameters in mapred-site.xml enable you to specify the inactivity timeout that marks a task as inactive and the reporting interval that running tasks use to send progress or heartbeats to the TaskTracker.
- (Issue 8282) When a TaskTracker is blacklisted, the job ID that triggered the blacklisting is now present in the blacklisting log message.
- (Issue 8686) The message
oo many RPC requests for metrics processing. Start dropping themhas had its alert level changed from
DEBUGto prevent flooding the log with messages.
- (Issue 9260) Revises the MapReduce slot computation algorithm to increase scalability and performance.
- (Issue 9684) Task attempt logs for a given task are no longer split across into the logs for the following task.
- (Issue 9732) Fixes a task JVM hang when Central Logging is enabled.
(Issue 10570) Additional logging for TaskTracker to provide more information for debugging and support.
(Issue 10863) Debug logging for JobTracker output now includes source or destination paths.
(Issue 11065) A new property in mapred-site.xml enables administrators to specify an upper bound for the user-configurable number of hours for which to retain job logs.
(Issue 11075) A new property in enables users to specify the number of records to process during a merge before sending a progress notification to the TaskTracker.
(Issue 11342) Incorporates MAPREDUCE-5047.
(Issue 11425) Default retention of Metrics data changed to 30 days from 365. New parameter in db.conf introduced to clean up metrics data based on total disk space used.
- (Issue 11780) Eliminated spurious raising of the
- (Issue 11952) Changes to the
configure-common.shscript run by eliminate a condition where running
configure.shwhile the TaskTracker was running made all future job initialization fail.
- (Issue 12358) Fixes a JobTracker Null Pointer Exception.
- (Issue 12437) Reduced JobTracker lock time with FairScheduler to prevent deadlocks.
- (Issue 9987) The
delete()call no longer throws
FileNotFoundExceptionwhen used to delete a non-existing file, bringing its behavior in line with the Java API.
- (Issue 10391) API improvements to enable Java applications to query multiple clusters.
- (Issue 10857) Corrected text for error messages reported by FileClient.
- (Issue 11479) Added support for providing MapR functionality through fatJARs.
- (Issue 11813) The Windows FileClient now prints the thread ID when debug logging is enabled.
- (Issue 12264) Addresses a loop condition where the FileClient was unable to send write requests to a master container.
- (Issue 7089) Read operations release global locks earlier, improving performance for renames of large numbers of files or directories.
- (Issue 8696) Bad block information is now logged to
mfs.errin addition to
mfs.logto increase the amount of historical information logged.
- (Issue 10672) Disk setup no longer fails with a permission denied error when you run
configure.shwith the disk add option.
- (Issue 11367) Fixed an issue where incorrect permissions on a file's inode caused read failures.
- (Issue 12244) Fixes an integer overflow condition that shortened MFS log retention.
- (Issue 12286) The fileserver no longer sends an old heartbeat message to the CLDB after establishing a new connection.
- (Issue 12357) Addresses an MFS crash condition related to adding new disks to a node.
- (Issue 11953) Scripts that call the
chmodutilities now check that the targets of those utilities aren't symbolic links.
- (Issue 10132) The
wardeninit.loghas been consolidated into
- (Issue 10734) The Warden does not run the
createsystemvolumes.shscript at startup on a node that does not have the Fileserver role configured.
- (Issue 10898) It is no longer possible to launch multiple instances of the Warden.
- (Issue 11844) The Warden now starts the CLDB service automatically when the CLDB role is added to the node.
- (Issue 12001) The file
warden.pidis now removed when Warden stops.
- (Issue 11494) The NFS server now reads up to 2048 characters per entry in the
exportsfile, instead of 128, to accommodate 255 characters for the maximum path length and up to 100 15-character IP addresses.
- (Issue 12097) Addresses a segmentation fault on the NFS server.
- (Issue 12371) NFS no longer logs error messages for a normal operating condition.
- (Issue 10148) Adds the
-serviceoption to the
maprcli node removecommand to enable removing the fileserver and nfsserver roles from a node individually. This prevents a condition where nodes that have been renamed have the same fileserver ID.
- (Issue 11021) The RPC timeout value for RPCs involving disk removal has been increased to improve CLDB responsiveness.
- (Issue 11456) Oozie no longer fails during upgrade if you upgrade from an older version of MapR to MapR version 3.1.0 or later with Oozie version 3.3.2-1401.
- (Issue 11514) Corrected the settings for raising the
CLUSTER_ALARM_CLDB_HEAPSIZEalarm, eliminating spurious reports.
- (Issue 12352) The CLDB no longer stops unexpectedly with a container failure notice caused by failed kvstore heartbeats.
- (Issue 12388) The CLDB no longer stops unexpectedly when new CLDB nodes are not in the same
- (Issue 11550) A message now appends to the
zookeeper.logfile after the ZooKeeper has been administratively killed.
- (Issue 12245) After upgrading ZooKeeper to 3.4.5, the snapshot cleanup interval is tunable by users. The
cronjob that handled snapshot cleanup has been removed.
- (Issue 10625) The Hadoop class path is now updated to correctly incorporate the JAR files for ZooKeeper 3.4.5.
MapReduce and Hadoop Patches Integrated Since Last Release
MapR 3.1 includes the following Apache MapReduce and Apache Hadoop patches:
- [MAPREDUCE-2187] Large map tasks no longer time out during the merge sort phase.
- [MAPREDUCE-5047] MapReduce tasks launched by a user other than the user running TaskTracker no longer fail when the value of the
keep.failed.task.filesparameter is set to
Server Configuration Support
Redhat Enterprise Linux
Oracle JDK 1.6
Oracle JDK 1.6
Suse Linux Enterprise Server
Oracle JDK 1.6
Oracle JDK 1.6
Oracle JDK 1.6
1 Hue 2.5 does not support secure connections to the cluster. Hue 2.5 is only supported for Red Hat/CentOS versions 6.0 and later.
2 Amazon EMR does not support wire-level security.
Client Configuration Support
7, 2008, v8
7, 2008, v8
Mac OS X