2017-03 阿里冯嘉:分布式消息引擎Apache RocketMQ最佳实践

文字内容
1. Trillion Messaging Engine – Apache RocketMQ Alibaba @vongosling © 201 7 A l i b ab a Mi d d l eware Gro u p
2. PROFILE Master of Computer Science,Over 7 plus years of industry software experience, especially in distributed tech., reliable web architecture and performance tuning. Possess many patents in distributed system,recommender etc. Open source fanatic, multiple open-source projects contributor, committer on GitHub and Apache. Focus on distributed, cloud computing and big data fields. Apache RocketMQ creator. Software craftsman, architect at alibaba, currently in charge of Aliware MQ. Email: vongosling@apache.org © 201 7 A l i b ab a Mi d d l eware Gro u p
3. CONTENTS 01 Distributed Messaging and Streaming Domain 02 RocketMQ History、Architecture、Advanced Features 03 Typical Scene、Best Practice、Performance Tuning 04 RocketMQ 4.X © 201 7 A l i b ab a Mi d d l eware Gro u p
4. 01 Distributed Messaging and Streaming Domain …… 合抱之木生于毫末;九层之台起于累土;千里之行始于足下…… © 201 7 A l i b ab a Mi d d l eware Gro u p
5. Distributed Systems Domain © 201 7 A l i b ab a Mi d d l eware Gro u p
6. Distributed Messaging and Streaming Domain © 201 7 A l i b ab a Mi d d l eware Gro u p
7. Distributed Messaging and Streaming Projects © 201 7 A l i b ab a Mi d d l eware Gro u p
8. 02 RocketMQ History、Architecture、Advanced Features 千磨万击还坚韧,任尔东西南北风 © 201 7 A l i b ab a Mi d d l eware Gro u p
9. Napoli MetaQ v2.0 v3.0 Apache ActiveMQ kernal,used for RocketMQ v3.0 RocketMQ B2B Trading and other OS scenarios 2007 2010 2011 2012 2015 2016 Notify MetaQ v1.0 Aliware MQ v1.0 Born from multicolored Design for orderly message, stone project, used for especial for messages Notify v3.0 Taobao B2C Trading accumulation © 201 7 A l i b ab a Mi d d l eware Gro u p
10. MetaQ Orderly message,Pull model RocketMQ Transaction message, Push model Notify Aliware MQ © 201 7 A l i b ab a Mi d d l eware Gro u p Professional Business Cloud MQ
11. 1400 Total number of Message(Unit: Billion) 650 194.2 33.2 10 2012 double 11 day 2013 double 11 day 2014 double 11 day © 201 7 A l i b ab a Mi d d l eware Gro u p 2015 double 11 day 2016 double 11 day
12. © 201 7 A l i b ab a Mi d d l eware Gro u p
13. Aliware MQ © 201 7 A l i b ab a Mi d d l eware Gro u p
14. Component Model © 201 7 A l i b ab a Mi d d l eware Gro u p
15. Domain Model © 201 7 A l i b ab a Mi d d l eware Gro u p
16. Server Macro Architecture © 201 7 A l i b ab a Mi d d l eware Gro u p
17. SDK Macro Architecture © 201 7 A l i b ab a Mi d d l eware Gro u p
18. Deploy Architecture © 201 7 A l i b ab a Mi d d l eware Gro u p
19. Core Advanced features – Producer transaction © 201 7 A l i b ab a Mi d d l eware Gro u p
20. Core Advanced features – HA © 201 7 A l i b ab a Mi d d l eware Gro u p
21. Core Advanced features – Message Tracing © 201 7 A l i b ab a Mi d d l eware Gro u p
22. Core Advanced features – Relaying © 201 7 A l i b ab a Mi d d l eware Gro u p
23. 03 Typical Scene、Best Practice 行路难,行路难,多歧路,今安在。 长风破浪会有时,直挂云帆济沧海。 © 201 7 A l i b ab a Mi d d l eware Gro u p
24. 1 Heterogeneous Application Integration 2 3 loosen the coupling between applications Backbone for EDA or CEP 4 5 © 201 7 A l i b ab a Mi d d l eware Gro u p Streaming Channel Data Replicator Channel
25. © 201 7 A l i b ab a Mi d d l eware Gro u p
26.  Producer send backoff send reliability send oneway  Consumer consume duplicate consume slow consume batch © 201 7 A l i b ab a Mi d d l eware Gro u p
27. © 201 7 A l i b ab a Mi d d l eware Gro u p
28.       File Descriptors (FD) Limits CPU affinity Memory lock limits NUMA Disable PageCache flush tuning Deadline I/O Schedule © 201 7 A l i b ab a Mi d d l eware Gro u p
29. 04 RocketMQ 4.X 咬定青山不放松,立根原在破岩中 © 201 7 A l i b ab a Mi d d l eware Gro u p
30. E-commerce Domain IOT Domain Massive Terminal Online Massive Throughput High Concurrency High Reliability Financial Domain Big data Domain © 201 7 A l i b ab a Mi d d l eware Gro u p
31. © 201 7 A l i b ab a Mi d d l eware Gro u p
32. © 201 7 A l i b ab a Mi d d l eware Gro u p
33. © 201 7 A l i b ab a Mi d d l eware Gro u p
34. © 201 7 A l i b ab a Mi d d l eware Gro u p
35. Community Ecosystem © 201 7 A l i b ab a Mi d d l eware Gro u p
36. Community Ecosystem  Workshop,Meetups  Code Marathon  Committers © 201 7 A l i b ab a Mi d d l eware Gro u p
37. collaborative software development; commercial-friendly standard license; consistently high-quality software; respectful, honest, technical-based interaction; faithful implementation of standards; security as a mandatory feature © 201 7 A l i b ab a Mi d d l eware Gro u p
38. © 201 7 A l i b ab a Mi d d l eware Gro u p
39. © 201 7 A l i b ab a Mi d d l eware Gro u p
40. © 201 7 A l i b ab a Mi d d l eware Gro u p
41. © 201 7 A l i b ab a Mi d d l eware Gro u p
42. 谢谢 THANKS! Alibaba Middleware Recommended Refers: 。1. 专访RocketMQ联合创始人:项目思路、技术细节和未来规划. InfoQ 2. 万亿级数据洪峰下的分布式消息引擎. 电子书节选章节 3. 捐赠背后那些鲜为人知的故事. 中间件博客节选 © 201 7 A l i b ab a Mi d d l eware Gro u p