发布于 2015-08-03 10:28:01 | 257 次阅读 | 评论: 0 | 来源: 网友投递
HBase分布式面向列的开源数据库
HBase是一个分布式的、面向列的开源数据库,该技术来源于 Fay Chang 所撰写的Google论文“Bigtable:一个结构化数据的分布式存储系统”。就像Bigtable利用了Google文件系统(File System)所提供的分布式数据存储一样,HBase在Hadoop之上提供了类似于Bigtable的能力。HBase是Apache的Hadoop项目的子项目。HBase不同于一般的关系数据库,它是一个适合于非结构化数据存储的数据库。另一个不同的是HBase基于列的而不是基于行的模式。
HBase 1.1.1 发布下载:
更新内容如下:
** 子任务
* [HBASE-13470] - High level Integration test for master DDL operations
* [HBASE-13497] - Remove MVCC stamps from HFile when that is safe
* [HBASE-13579] - Avoid isCellTTLExpired() for NO-TAG cases
* [HBASE-13593] - Quota support for namespace should take snapshot restore and clone into account
* [HBASE-13658] - Improve the test run time for TestAccessController class
* [HBASE-13937] - Partially revert HBASE-13172
** Bug 修复
* [HBASE-11830] - TestReplicationThrottler.testThrottling failed on virtual boxes
* [HBASE-12413] - Mismatch in the equals and hashcode methods of KeyValue
* [HBASE-13200] - Improper configuration can leads to endless lease recovery during failover
* [HBASE-13217] - Procedure fails due to ZK issue
* [HBASE-13312] - SmallScannerCallable does not increment scan metrics
* [HBASE-13325] - Protocol Buffers 2.5 no longer available for download on code.google.com
* [HBASE-13333] - Renew Scanner Lease without advancing the RegionScanner
* [HBASE-13377] - Canary may generate false alarm on the first region when there are many delete markers
* [HBASE-13411] - Misleading error message when request size quota limit exceeds
* [HBASE-13564] - Master MBeans are not published
* [HBASE-13574] - Broken TestHBaseFsck in master with hadoop 2.6.0
* [HBASE-13576] - HBCK enhancement: Failure in checking one region should not fail the entire HBCK operation.
* [HBASE-13600] - check_compatibility.sh should ignore shaded jars
* [HBASE-13604] - bin/hbase mapredcp does not include yammer-metrics jar
* [HBASE-13606] - AssignmentManager.assign() is not sync in both path
* [HBASE-13607] - TestSplitLogManager.testGetPreviousRecoveryMode consistently failing
* [HBASE-13608] - 413 Error with Stargate through Knox, using AD, SPNEGO, and Pre-Auth
* [HBASE-13611] - update clover to work for current versions
* [HBASE-13612] - TestRegionFavoredNodes doesn't guard against setup failure
* [HBASE-13617] - TestReplicaWithCluster.testChangeTable timeout
* [HBASE-13618] - ReplicationSource is too eager to remove sinks
* [HBASE-13625] - Use HDFS for HFileOutputFormat2 partitioner's path
* [HBASE-13626] - ZKTableStateManager logs table state changes at WARN
* [HBASE-13628] - Use AtomicLong as size in BoundedConcurrentLinkedQueue
* [HBASE-13632] - Backport HBASE-13368 to branch-1 and 0.98
* [HBASE-13635] - Regions stuck in transition because master is incorrectly assumed dead
* [HBASE-13638] - Put copy constructor is shallow
* [HBASE-13647] - Default value for hbase.client.operation.timeout is too high
* [HBASE-13653] - Uninitialized HRegionServer#walFactory may result in NullPointerException at region server startup
* [HBASE-13662] - RSRpcService.scan() throws an OutOfOrderScannerNext if the scan has a retriable failure
* [HBASE-13663] - HMaster fails to restart 'HMaster: Failed to become active master'
* [HBASE-13664] - Use HBase 1.0 interfaces in ConnectionCache
* [HBASE-13668] - TestFlushRegionEntry is flaky
* [HBASE-13686] - Fail to limit rate in RateLimiter
* [HBASE-13703] - ReplicateContext should not be a member of ReplicationSource
* [HBASE-13704] - Hbase throws OutOfOrderScannerNextException when MultiRowRangeFilter is used
* [HBASE-13709] - Updates to meta table server columns may be eclipsed
* [HBASE-13711] - Provide an API to set min and max versions in HColumnDescriptor
* [HBASE-13717] - TestBoundedRegionGroupingProvider#setMembershipDedups need to set HDFS diretory for WAL
* [HBASE-13721] - Improve shell scan performances when using LIMIT
* [HBASE-13723] - In table.rb scanners are never closed.
* [HBASE-13727] - Codehaus repository is out of service
* [HBASE-13729] - Old hbase.regionserver.global.memstore.upperLimit and lowerLimit properties are ignored if present
* [HBASE-13731] - TestReplicationAdmin should clean up MiniZKCluster resource
* [HBASE-13732] - TestHBaseFsck#testParallelWithRetriesHbck fails intermittently
* [HBASE-13733] - Failed MiniZooKeeperCluster startup did not shutdown ZK servers
* [HBASE-13734] - Improper timestamp checking with VisibilityScanDeleteTracker
* [HBASE-13741] - Disable TestRegionObserverInterface#testRecovery and testLegacyRecovery
* [HBASE-13746] - list_replicated_tables command is not listing table in hbase shell.
* [HBASE-13767] - Allow ZKAclReset to set and not just clear ZK ACLs
* [HBASE-13768] - ZooKeeper znodes are bootstrapped with insecure ACLs in a secure configuration
* [HBASE-13776] - Setting illegal versions for HColumnDescriptor does not throw IllegalArgumentException
* [HBASE-13777] - Table fragmentation display triggers NPE on master status page
* [HBASE-13778] - BoundedByteBufferPool incorrectly increasing runningAverage buffer length
* [HBASE-13779] - Calling table.exists() before table.get() end up with an empty Result
* [HBASE-13789] - ForeignException should not be sent to the client
* [HBASE-13797] - Fix resource leak in HBaseFsck
* [HBASE-13800] - TestStore#testDeleteExpiredStoreFiles should create unique data/log directory for each call
* [HBASE-13801] - Hadoop src checksum is shown instead of HBase src checksum in master / RS UI
* [HBASE-13802] - Procedure V2: Master fails to come up due to rollback of create namespace table
* [HBASE-13809] - TestRowTooBig should use HDFS directory for its region directory
* [HBASE-13810] - Table is left unclosed in VerifyReplication#Verifier
* [HBASE-13811] - Splitting WALs, we are filtering out too many edits -> DATALOSS
* [HBASE-13812] - Deleting of last Column Family of a table should not be allowed
* [HBASE-13821] - WARN if hbase.bucketcache.percentage.in.combinedcache is set
* [HBASE-13822] - shell is broken with NoMethodError
* [HBASE-13824] - TestGenerateDelegationToken: Master fails to start in Windows environment
* [HBASE-13826] - Unable to create table when group acls are appropriately set.
* [HBASE-13831] - TestHBaseFsck#testParallelHbck is flaky against hadoop 2.6+
* [HBASE-13833] - LoadIncrementalHFile.doBulkLoad(Path,HTable) doesn't handle unmanaged connections when using SecureBulkLoad
* [HBASE-13834] - Evict count not properly passed to HeapMemoryTuner.
* [HBASE-13845] - Expire of one region server carrying meta can bring down the master
* [HBASE-13847] - getWriteRequestCount function in HRegionServer uses int variable to return the count.
* [HBASE-13851] - RpcClientImpl.close() can hang with cancelled replica RPCs
* [HBASE-13853] - ITBLL improvements after HBASE-13811
* [HBASE-13873] - LoadTestTool addAuthInfoToConf throws UnsupportedOperationException
* [HBASE-13875] - Clock skew between master and region server may render restored region without server address
* [HBASE-13877] - Interrupt to flush from TableFlushProcedure causes dataloss in ITBLL
* [HBASE-13878] - Set hbase.fs.tmp.dir config in HBaseTestingUtility.java for Phoenix UT to use
* [HBASE-13885] - ZK watches leaks during snapshots
* [HBASE-13892] - Scanner with all results filtered out results in NPE
* [HBASE-13901] - Error while calling watcher on creating and deleting an HBase table
* [HBASE-13904] - TestAssignmentManager.testBalanceOnMasterFailoverScenarioWithOfflineNode failing consistently on branch-1.1
* [HBASE-13905] - TestRecoveredEdits.testReplayWorksThoughLotsOfFlushing failing consistently on branch-1.1
* [HBASE-13933] - DBE's seekBefore with tags corrupts the tag's offset information thus leading to incorrect results
* [HBASE-13935] - Orphaned namespace table ZK node should not prevent master to start
* [HBASE-13938] - Deletes done during the region merge transaction may get eclipsed
** 改进
* [HBASE-12415] - Add add(byte[][] arrays) to Bytes.
* [HBASE-13247] - Change BufferedMutatorExample to use addColumn() since add() is deprecated
* [HBASE-13344] - Add enforcer rule that matches our JDK support statement
* [HBASE-13366] - Throw DoNotRetryIOException instead of read only IOException
* [HBASE-13420] - RegionEnvironment.offerExecutionLatency Blocks Threads under Heavy Load
* [HBASE-13671] - More classes to add to the invoking repository of org.apache.hadoop.hbase.mapreduce.driver
* [HBASE-13673] - WALProcedureStore procedure is chatty
* [HBASE-13675] - ProcedureExecutor completion report should be at DEBUG log level
* [HBASE-13761] - Optimize FuzzyRowFilter
* [HBASE-13780] - Default to 700 for HDFS root dir permissions for secure deployments
* [HBASE-13816] - Build shaded modules only in release profile
* [HBASE-13828] - Add group permissions testing coverage to AC.
* [HBASE-13846] - Run MiniCluster on top of other MiniDfsCluster
* [HBASE-13894] - Avoid visitor alloc each call of ByteBufferArray get/putMultiple()
** Task
* [HBASE-13666] - book.pdf is not renamed during site build
* [HBASE-13697] - publish updated ref guide with prereq tables for 1.1 release train
* [HBASE-13764] - Backport HBASE-7782 (HBaseTestingUtility.truncateTable() not acting like CLI) to branch-1.x
* [HBASE-13799] - javadoc how Scan gets polluted when used; if you set attributes or ask for scan metrics
* [HBASE-13929] - make_rc.sh publishes empty shaded artifacts
** Test
* [HBASE-13591] - TestHBaseFsck is flakey
* [HBASE-13609] - TestFastFail is still failing
* [HBASE-13623] - TestSplitLogManager.testGetPreviousRecoveryMode is still flaky
* [HBASE-13940] - IntegrationTestBulkLoad needs option to specify output folders used by test
HBase是一个分布式的、面向列的开源数据库,该技术来源于 Fay Chang 所撰写的Google论文“Bigtable:一个结构化数据的分布式存储系统”。就像Bigtable利用了Google文件系统(File System)所提供的分布式数据存储一样,HBase在Hadoop之上提供了类似于Bigtable的能力。HBase是Apache的Hadoop项目的子项目。HBase不同于一般的关系数据库,它是一个适合于非结构化数据存储的数据库。另一个不同的是HBase基于列的而不是基于行的模式。
HBase – Hadoop Database,是一个高可靠性、高性能、面向列、可伸缩的分布式存储系统,利用HBase技术可在廉价PC Server上搭建起大规模结构化存储集群。
HBase是Google Bigtable的开源实现,类似Google Bigtable利用GFS作为其文件存储系统,HBase利用Hadoop HDFS作为其文件存储系统;Google运行MapReduce来处理Bigtable中的海量数据,HBase同样利用Hadoop MapReduce来处理HBase中的海量数据;Google Bigtable利用 Chubby作为协同服务,HBase利用Zookeeper作为对应。