This page contains the following topics:
New In This Release
- Change: Substantially improved performance for MapR Tables. This release features throughput and latency improvements for insert, read, scan, and mixed-load operations.
- Feature: Rolling Upgrade. See Scripted Rolling Upgrade for complete documentation on this new feature.
- Change: The 3.0.1 release of the MapR distribution for Hadoop supports the 94.9-1308 and 92.2-1308 versions of HBase. The 0.94.5 version of HBase can only access Apache HBase tables and cannot read MapR native tables.
- Change: The 3.0.1 release is available for all license levels of the MapR distribution for Hadoop.
- Change: A new alarm (
NODE_ALARM_M7_CONFIG_MISMATCH) raises when an M7 license is applied to a cluster before the nodes in the cluster have been configured for M7.
- Change: The
HTableobjects is enabled by default. To set a behavior for
autoFlush, call the
HTable.setAutoFlush(boolean)function in your client application code. The
autoFlushsetting was forced to false in version 3.0.0 of the MapR distribution for Hadoop. To maintain continuity of behavior of client applications with MapR 3.0.0, update the application source code to explicitly call
Resolved issues are listed in the sections below. This release also includes all changes made for the 184.108.40.206 release.
- (Issue 8629) The ownership of files related to a job is no longer changed to the
mapruser after a Warden restart. Task attempt failures related to this ownership change no longer occur.
- (Issue 8933) ACL entries with empty permissions are now purged. The
maprcli acl showcommand now shows only UIDs that are actually present on the system and part of the ACL.
- (Issue 9608) Scans and gets return correct results for time-range queries.
- (Issue 9721) The
disklist.shscript has been updated to correctly report information about MFS disks and accurately report failed disks.
- (Issue 9977) MFS memory allocation percentage now changes correctly in response to changing roles on a node.
- (Issue 10238) The Remove Volume button in the MCS web UI now works correctly.
- (Issue 10408) Upgraded the version of
libjpam.soto prevent MCS crashes related to expiring passwords.
- (Issue 10452) [Critical] The Task JVM is no longer receiving spurious KILL signals during Hive JOIN operations, leading to incorrectly blacklisted TaskTracker nodes. The system now uses
tgkillto kill a specific PID, instead of
- (Issue 10490) HBase build changed to remove duplicated presence of
aspectjtools-1.6.5.jar, which was causing Null Pointer exceptions in Hive.
- (Issue 10498) MCS now uses GB as the unit for all disk reporting information.
- (Issue 10851) Managing user permissions from the MCS UI now works correctly.
- (Issue 10924) The spillscanner now correctly prints out the number of bytes read to accurately diagnose scan related problems.
- (Issue 11019) The
diskremove.shcommand now logs its output for diagnostic purposes.
- (Issue 11123) Large numbers (over 100) of configured disks no longer cause the
hoststatsservice to stop.
- (Issue 11216) Database memory sizing units changed. Database memory is now allocated in Kb.
- (Issue 10803) MapR Tables support the
- (Issue 10932) MapR-FS now reports more detailed server side per-operation statistics for FileClient RPCs relating to M7 tables.
- (Issue 11144) Table cells are now deserialized in the correct order.
- (Issue 11155) The
deleteColumn()operation throws an
UnsupportedOperationExceptionexception when called without a datestamp.
- (Issue 11187) Table regions that are on a single partition can now be split in the same way as regions on multiple partitions.
- (Issue 11313) Optimized scan response times for small quantities of data, resulting in improved performance.
- (Issue 11375) The size of the db value cache no longer negatively affects CPU performance due to an overflow condition.
- (Issue 8446) A watcher now prevents operations from starting before the ZooKeeper connection is fully established.
- (Issue 8584) Added new admin-only command
hadoop job -unblacklist-tracker <hostname>to remove a given TaskTracker from the JobTracker's global blacklist.
- (Issue 9706) Name resolution is now case-insensitive, conforming to RFC 4343.
- (Issue 9861) The JobTracker UI now catches Counter Exceeded exceptions properly.
- (Issue 10519) Exceptions due to an overflow condition with file ID uniquifiers in excess of 2^31 no longer occur.
- (Issue 10569) TaskTrackers now correctly clean up the distributed cache.
- (Issue 10835) Updated the
createTTVolume.shto increase the time between retries after a failed attempt. This change avoids fatal errors due to exhausting the number of retries too early.
- (Issue 10869) Fixed an error condition where ReduceTasks attempting to open a file ID inside a failed container would retry indefinitely, leading to reducer hangs.
- (Issue 11219) The
mapred.job.shuffle.merge.percentparameters can now be altered from their defaults without causing job failures.
- (Issue 9484) The log level of a missing
mapr-clusters.conffile has been changed from ERROR to WARN.
- (Issue 10279) ReadRPC no longer continues to read past the end of a file with a size that is an exact multiple of the chunk size.
- (Issue 10973) The
hadoop mfs -setcompressioncommand no longer fails silently when run on an M7 table.
- (Issue 7621) Resync network activity now throttles proportionately to the number of resync send threads and round trip time. Implements two new parameters in the
- (Issue 8668) MFS now correctly handles storage pools with duplicate names by incrementing unique numbers appended to the name.
- (Issue 10482) Fixed MFS core dump related to invalid file IDs during deferred call execution.
- (Issue 10815) [Critical] When a node failure triggers a resync, another full resync triggers if the number of keys at the destination inode does not match the number of keys at the source.
- (Issue 10947) The fileserver now initializes
inodeRsyncSencWAActiveListto prevent the dump thread from attempting to access an invalid memory location.
- (Issue 10985) Scanning an empty M7 table in a volume that has a snapshot no longer causes a segmentation fault.
- (Issue 11029) Fixes a MapR-FS crash on Get actions.
- (Issue 11052) The
mapruser is now able to restart the TaskTracker after an upgrade.
- (Issue 11326) Fixed core generation related to thread mutexes.
- (Issue 9702) The Hadoop classes for the
chowncommands now correctly re-throw exceptions when the JNI layer detects an error.
- (Issue 4877) The memory footprint for containers in the CLDB is now reduced, leading to a smaller memory footprint for the CLDB.
- (Issue 7844) Nodes with large numbers of containers no longer significantly affect cluster restart operations due to extended resync activity.
- (Issue 10644) Alarms related to nodes that are no longer in the cluster no longer persist indefinitely.
- (Issue 10811) Unavailable containers in a volume are now logged.
- (Issue 10937) A new alarm reports low-memory conditions for the CLDB when the free memory space falls below 30% of total heap. When this alarm is raised, the system makes a recommendation for new CLDB memory settings.
- (Issue 10951) Copy-on-write range lookups no longer inaccurately reports non-contiguous keys as contiguous. The CLDB now takes an exclusive file ID lock when
kvstoremultiophas a delete message.
- (Issue 11084) A new alarm reports containers that are in a non-local topology.
- (Issue 9899) The
fsstatimplementation properly returns ESTALE, preventing NFS server crashes due to attempting to access a null memory location.
- (Issue 10659) The NFS server now makes fewer
write()calls, resulting in substantially increased throughput.
- (Issue 11077) The NFS processes no longer leak Volume Lookup Request or Get Attribute Request handles, fixing a runaway memory leak condition.
- (Issue 9800) Users can no longer kill jobs submitted by other users, including
mapruser correctly has full privileges to kill jobs.
When a node's IP number is reassigned to another node, remote procedure calls are misdirected to the wrong node. This can result in widespread errors affecting any system that use the RPC binding layer, including container resync operations that never finish or the creation of several superfluous containers. Restart the cluster to clear this condition.
If you use the latest version of any browser to view the MCS, the CLDB, HBase, JobTracker, and job log pages do not load when you click on the respective links. As a workaround, use the following instructions to disable mixed-content-blocking in your browser:
Scroll to the bottom of the screen, and click Show all content. The page refreshes and displays mixed content.
- At the top of the page, to the left of the address bar, click (the shield icon).
- In the pop-up window that appears, click the down arrow next to "Keep Blocking", and select Disable Protection on This Page. The page refreshes and displays mixed content. The shield icon is replaced with a warning symbol to indicate that the page is displaying both secure and non-secure items.
- Click (the shield icon) on the right side of the address bar.
- In the icon dialog box, click Load anyway. The page refreshes and displays mixed content. The URL in the address bar shows https crossed out to indicate that the page is displaying both secure and non-secure items.
Server Configuration Support
HCatalog* (For Hive 0.9 and 0.10)
Redhat Enterprise Linux
Oracle JDK 1.6
Oracle JDK 1.6
Suse Linux Enterprise Server
Oracle JDK 1.6
Oracle JDK 1.6
Oracle JDK 1.6
- HCatalog and WebHCat are included in Hive as of version 0.11
Client Configuration Support
7, 2008, v8
7, 2008, v8
Mac OS X