发布于 2015-05-02 00:00:52 | 157 次阅读 | 评论: 0 | 来源: 网友投递
Spring XD
Spring XD是一个统一的,分布式,可扩展的系统用于 data ingestion,实时分析,批量处理和数据导出。该项目的目标是简化大数据应用的开发。
Spring XD 1.2 M1/1.1.2 发布,Spring XD 1.1.2 是个维护版本,现已提供下载:
Spring XD 1.1.2 完整的issues 列表提供在 JIRA,支持 PHD 3.0。
Spring XD 1.2 M1 包括 bug 修复,新特性和一些改进:
MongoDB Source, a community contribution from Abhinav Gandhi
Module registry backed by HDFS
Greenplum gpload as provided batch job. This allows for efficient loading from CSV files into Greenplum DB/HAWQ.
gpfdist sink that adheres to the gpfdist protocol. This allows for streaming data in parallel into Greenplum DB/HAWQ.
Zookeeper distributed queue based deployment for streams and jobs.
Improved error handling for RabbitMQ with Dead Letter Queue and durable queue support for pub/sub named channels (tap: and topic:)
Sqoop integration improvements, support for merge and codegen commands as well as running against a secured Hadoop cluster.
Kafka message bus improvements, customized partition count for topics created by the message bus. (module.[modulename].producer.minParitionCount)
Improved performance characteristics for TupleBuilder and the JDBC to HDFS job
Spark Streaming integration improvements, reliable receiver support and bug fixes.
完整改进列表请看issues fixed for 1.2 M1。
Spring XD 1.2 RC1 计划特性:
Kafka message bus improvements
Cassandra sink
Improved HA support for RabbitMQ
Incremental data import for JDBC to HDFS job
更多内容请看发行说明。
Spring XD是一个统一的,分布式,可扩展的系统用于 data ingestion,实时分析,批量处理和数据导出。该项目的目标是简化大数据应用的开发。