发布于 2015-11-06 07:20:49 | 300 次阅读 | 评论: 0 | 来源: 网友投递
Apache Storm 分布式实时计算系统
Apache Storm 的前身是 Twitter Storm 平台,目前已经归于 Apache 基金会管辖。Apache Storm 是一个免费开源的分布式实时计算系统。简化了流数据的可靠处理,像 Hadoop 一样实现实时批处理。Storm 很简单,可用于任意编程语言。Apache Storm 采用 Clojure 开发。
Apache Storm 0.10.0 发布,此版本是个稳定版本,相比之前的 Beta 版本主要包括 bug 修复和改进:
STORM-1108: Fix NPE in simulated time
STORM-1106: Netty should not limit attempts to reconnect
STORM-1099: Fix worker childopts as arraylist of strings
STORM-1096: Fix some issues with impersonation on the UI
STORM-912: Support SSL on Logviewer
STORM-1094: advance kafka offset when deserializer yields no object
STORM-1066: Specify current directory when supervisor launches a worker
STORM-1012: Shaded everything that was not already shaded
STORM-967: Shaded everything that was not already shaded
STORM-922: Shaded everything that was not already shaded
STORM-1042: Shaded everything that was not already shaded
STORM-1026: Adding external classpath elements does not work
STORM-1055: storm-jdbc README needs fixes and context
STORM-1044: Setting dop to zero does not raise an error
STORM-1050: Topologies with same name run on one cluster
STORM-1005: Supervisor do not get running workers after restart.
STORM-803: Cleanup travis-ci build and logs
STORM-1027: Use overflow buffer for emitting metrics
STORM-1024: log4j changes leaving ${sys:storm.log.dir} under STORM_HOME dir
STORM-944: storm-hive pom.xml has a dependency conflict with calcite
STORM-994: Connection leak between nimbus and supervisors
STORM-1001: Undefined STORM_EXT_CLASSPATH adds '::' to classpath of workers
STORM-977: Incorrect signal (-9) when as-user is true
STORM-843: [storm-redis] Add Javadoc to storm-redis
STORM-866: Use storm.log.dir instead of storm.home in log4j2 config
STORM-810: PartitionManager in storm-kafka should commit latest offset before close
STORM-928: Add sources->streams->fields map to Multi-Lang Handshake
STORM-945: element is not a policy,and should not be putted in the element.
STORM-857: create logs metadata dir when running securely
STORM-793: Made change to logviewer.clj in order to remove the invalid http 500 response
STORM-139: hashCode does not work for byte[]
STORM-860: UI: while topology is transitioned to killed, "Activate" button is enabled but not functioning
STORM-966: ConfigValidation.DoubleValidator doesn't really validate whether the type of the object is a double
STORM-742: Let ShellBolt treat all messages to update heartbeat
STORM-992: A bug in the timer.clj might cause unexpected delay to schedule new event
Apache Storm 0.10.0 发行说明:
https://storm.apache.org/2015/11/05/storm0100-released.html
Maven Central:
groupId: org.apache.storm
artifactId: storm-core
version: 0.10.0
同时还发布了 Apache Storm 0.9.6 版本,这是个维护版本,主要是 bug 修复和改进:
STORM-1027: Use overflow buffer for emitting metrics
STORM-996: netty-unit-tests/test-batch demonstrates out-of-order delivery
STORM-1056: allow supervisor log filename to be configurable via ENV variable
STORM-1051: Netty Client.java's flushMessages produces a NullPointerException
STORM-763: nimbus reassigned worker A to another machine, but other worker's netty client can't connect to the new worker A
STORM-935: Update Disruptor queue version to 2.10.4
STORM-503: Short disruptor queue wait time leads to high CPU usage when idle
STORM-728: Put emitted and transferred stats under correct columns
STORM-643: KafkaUtils repeatedly fetches messages whose offset is out of range
STORM-933: NullPointerException during KafkaSpout deactivation
下载:http://storm.apache.org/downloads.html
Apache Storm 的前身是 Twitter Storm 平台,目前已经归于 Apache 基金会管辖。
Apache Storm 是一个免费开源的分布式实时计算系统。简化了流数据的可靠处理,像 Hadoop 一样实现实时批处理。Storm 很简单,可用于任意编程语言。Apache Storm 采用 Clojure 开发。
Storm 有很多应用场景,包括实时数据分析、联机学习、持续计算、分布式 RPC、ETL 等。Storm 速度非常快,一个测试在单节点上实现每秒一百万的组处理。
目前已经有包括阿里百度在内的数家大型互联网公司在使用该平台。