AS深圳2017《Walmart eCommerce Search system architecture;intro and Evolution》 曹宁

阿木的花园

2019/06/25 发布于 编程 分类

文字内容
1. Walmart eCommerce Search System Architecture: intro and evolution Ning Cao WalmartLabs Engineering Manager
4. ็ਘ WalmartLabs Engineering Manager • Ning Cao is an engineering manager in search runtime team at WalmartLabs. Prior to that, he worked at Google, Huawei. • Ning received his Ph.D. in Electrical and Computer Engineering at Worcester Polytechnic Institute. His publications have 4000+ citations.
5. • Walmart eCommerce Search • Search Architecture Evolution • Experience & Lessons
6. Walmart eCommerce Search • Search • Browse • Category Pages
7. Walmart eCommerce Search Performance Challenges in Search Backend • • Increasing index size • • ~8x in past 3 years Real time update
8. • Walmart eCommerce Search • Search Architecture Evolution • Architecture Overview • Distributed Search Cloud • Re-rank Migration Metadata Store Experience & Lessons • •
9. Search Runtime Architecture Overview
10. Search Runtime Architecture Overview
11. • Walmart eCommerce Search • Search Architecture Evolution • Architecture Overview • Distributed Search Cloud • Re-rank Migration Metadata Store Experience & Lessons • •
12. Distributed Search • Load-balanced Shard VIP
13. Distributed Search Problems with Load-balanced Shard VIP • • Performance bottleneck • Hard to troubleshoot • Unnecessary re-routing • Increasing open connections between VIP and Solr shard
14. Why Not SolrCloud Problems with SolrCloud • • Unable to utilize offline SolrCloud for index update • Inefficient indexing: # of shards • Must build two sets of SolrCloud to index all shards at same time
15. Distributed Search Cloud • Tuple-based Polaris Cloud
16. Distributed Search Cloud Tuple-based Polaris Cloud • • Fallback • Last-known-state use • Cached sharding data from ZooKeeper • Sharding data from live update
17. • Walmart eCommerce Search • Search Architecture Evolution • Architecture Overview • Distributed Search Cloud • Re-rank Migration Metadata Store Experience & Lessons • •
18. Re-rank Migration • Re-rank plugin in Solr
19. Re-rank Migration Re-rank plugin in Solr • • Implemented before Solr Re-rank • Need code change during Solr update • Unable to evaluate/migrate to other search engines
20. Re-rank Migration • Re-rank in Polaris
21. Re-rank Migration • Re-rank plugin in Polaris • • Pros: • Solr update • Search engine migration Cons: • Network overload between Polaris and Solr • Serialization & Deserialization
22. • Walmart eCommerce Search • Search Architecture Evolution • Architecture Overview • Distributed Search Cloud • Re-rank Migration Metadata Store Experience & Lessons • •
23. Metadata Store Backend challenges caused by large Solr index • • Search performance • Full index generation time • Index replication overhead • Real time update throughput
24. Metadata Store How to reduce Solr index size? • • Move stored fields out of Solr
25. Metadata Store Design Goal: store fields for re-rank and response • • Scalability: easy to scale • Performance: fast data retrieval, high read and write throughput • Functionality: support structured queries
26. Metadata Store Metadata Store Design • • Primary data store:'>store: Couchbase • Secondary data store:'>store: Elastic Search • Data format: Avro
27. Metadata Store • Search with metadata store
28. Metadata Store • Update with metadata service
29. Metadata Store Gains: • • Search latency: 10% • Index size: 25% • Real time update • Larger re-rank size
30. • Walmart eCommerce Search • Search Architecture Evolution • • Architecture Overview • Distributed Search Cloud • Re-rank Migration • Metadata Store Experience & Lessons
31. Experience & Lessons How to utilize open source software • • Adopt • Customize • Replace/Self-dev
32. Experience & Lessons Microservices • Different tech stacks • • Operation overhead, Performance tuning Microservices framework • • Rate control, perf monitor, config management, authentication, logging, etc.