Airbnb 贾云涛 从单结构到面向服务的系统架构

CodeWarrior

2019/07/08 发布于 编程 分类

GIAC2019 

文字内容
1. The great migration - From Monolith to Service Oriented Architecture 从单结构到微服务的系统迁移 Yuntao Jia, Airbnb(爱彼迎) 6/22/2019@GIAC
3. QUIZ Airbnb为什么要迁移到 微服务? A: 赶时髦, trend B: 迫不得已, survival C: 网站太慢, latency D: 技术革新, technical revolution
4. QUIZ Airbnb为什么要迁移到 微服务? A: 赶时髦, trend B: 迫不得已, survival ✔ C: 网站太慢, latency ✔ D: 技术革新, technical revolution
5. Airbnb/爱彼迎
6. ● 2015-Present: Airbnb, EM ● 2010-2014: Facebook, IC, EM About me ● ● ● 人,产品,科技 linkedin/yuntaojia wechat
7. 迫不得已 Why not monolith? 2 如何着手 How to move to SOA? 3 学到了 Challenges and learnings 4 成绩单 Outcome (so far) Agenda 1
8. 1 迫不得已 Why not monolith?
9. “Monorails” Source: wikipedia
10. Spaghetti Monorails code Source: wikipedia Source: taste
11. Ex: checkout page
12. Airbnb Monolithic Rails app Why not monolith - Airbnb “Monorails” app ● A single large app => Dev bottleneck ● A single deployment queue => Deploy bottleneck ● A single thread => performance bottleneck
13. Airbnb Monolithic Rails app Why not monolith - Airbnb “Monorails” app ● A single large app => Dev bottleneck ● A single deployment queue => Deploy bottleneck ● A single thread => performance bottleneck
14. Airbnb Monolithic Rails app Why not monolith - Airbnb “Monorails” app ● A single large app => Dev bottleneck ● A single deployment queue => Deploy bottleneck ● A single thread => performance bottleneck ● 3h deployment blocked per day in 2016 on average
15. Airbnb Monolithic Rails app Why not monolith - Airbnb “Monorails” app ● A single large app => Dev bottleneck ● A single deployment queue => Deploy bottleneck ● A single thread => performance bottleneck Source: Indiatimes
16. Why not Monolith? 迁移到微服务 Migrate to SOA
17. From monolithic to Service-oriented architecture (SOA) Why not Monolith - Migrate to SOA Monorails Source: pwc
18. Many have done it Why not Monolith - Migrate to SOA Monorails ● ● ● ● ● Twitter: 2012-2015 (link) Uber: (link) Medium: (link) Wish And so on
19. Why not Monolith? 我们的观点 What is Airbnb’s vision of SOA?
20. Why not Monolith - What is Airbnb’s vision of SOA Monorails SOA design in Airbnb
21. Why not Monolith - What is Airbnb’s vision of SOA Monorails SOA design in Airbnb
22. Why not Monolith - What is Airbnb’s vision of SOA Monorails SOA design in Airbnb
23. Why not Monolith - What is Airbnb’s vision of SOA Monorails SOA design in Airbnb
24. 2 如何着手 How to migrate to SOA?
25. How to migrate to SOA? 初尝试 First attempt
26. First attempt How to migrate to SOA - First attempt Replacing homes data services backend, isolated, critical Mysql adapter Mysql database
27. First attempt How to migrate to SOA - First attempt Replacing homes data services, 2016-2017 Service request {host_id = 4} Mysql database Home service
28. How to migrate to SOA? 策略 Migration strategy
29. How to migrate to SOA - Migration strategy Migration strategy Create Home data service A few core services, Tools and frameworks migrate all actively developed products 2016 2017 2018-now migrate long tail services when needed Source: cultofpedagogy
30. How to migrate to SOA - Migration strategy Interim architecture
31. How to migrate to SOA - Migration strategy Service types
32. How to migrate to SOA - Migration strategy Service types Ex: checkout page
33. How to migrate to SOA? 工具 Toolings
34. How to migrate to SOA - Toolings New service framework Monorails
35. New service framework with Thrift IDL Monorails How to migrate to SOA - Toolings Auto generate api code and java/ruby clients
36. New service framework with Thrift IDL Monorails How to migrate to SOA - Toolings Templated metrics, dashboards and alerts
37. Monorails How to migrate to SOA - Toolings Compare for differences SOA design in Airbnb Source: funnyordie
38. How to migrate to SOA - Toolings Dual read comparison
39. How to migrate to SOA - Toolings Dual read comparison
40. How to migrate to SOA - Toolings Dual write comparison
41. How to migrate to SOA - Toolings Dual write comparison
42. 3 学到了 Challenges and learnings
43. Migration complex product to SOA is hard Challenges and learnings “Changing tires while driving is hard” Source: gizmodo
44. Challenges and learnings - Migration is hard Migration complex product to SOA is hard ● Product development is slowed down ● Shared mid-tier services are hard to scope ● Manage dependencies between teams is challenging ● Taking up to 50% of product resources at peak migration time
45. Challenges and learnings - Migration is hard Migration complex product to SOA is hard ● Product development is slowed down ● Shared mid-tier services are hard to scope ● Manage dependencies between teams is challenging Home creation Home management Home batch edit ? Homes data validation middle-tire services Homes data service Airbnb plus ?
46. Challenges and learnings - Migration is hard Migration complex product to SOA is hard ● Product development is slowed down ● Shared mid-tier services are hard to scope ● Manage dependencies between teams is challenging
47. Challenges and learnings SOA is not for everyone Monolithic app works well SOA works well
48. Challenges and learnings SOA is not for everyone ● High investment cost ● Distributed services ● Complex service orchestration ● ● ● Tools Frameworks Developer docs
49. Challenges and learnings SOA is not for everyone ● High investment cost ● ● Distributed services ● Complex service orchestration ● ● ● ● Multiple requests instead of one Network latency and failures Consistency becomes hard Observability becomes hard Debugging becomes hard
50. Challenges and learnings SOA is not for everyone ● High investment cost ● Distributed services ● Complex service orchestration ● ● ● Maintain services Onboard new engineers EC2 => Kubernetes
51. 4 成绩单 SOA outcome
52. SOA outcome Services & Adoption 1 250+ 1000+ Monorails app Services Endpoints Before After
53. SOA outcome Faster deployment Hours to deploy Minutes to deploy 2x One Monorails deployment queue Individual service deployment queue Deploys per eng from 18 to 17 Before After
54. SOA outcome Better latency Single thread Multiple threaded 3x faster 10x faster Monorails app Java services Search experience Home description page Before After
55. More work ahead 2016 - now SOA outcome We are here
56. Takeaways 要点 Why not Monolith - Migrate to SOA Monorails ● ● SOA的时机很重要,timing of SOA SOA的代价高,SOA is not for everyone
57. Q&A
58. Special thanks ● Jessical Thai @Airbnb for helping me build this presentation (link) ● Xiao Ma @Medium for discussions