腾讯 朱仕麟 打造新一代智能可视化爬取平台

CodeWarrior

2019/07/08 发布于 编程 分类

GIAC2019 

文字内容
1. web
3. CONTENT 01 02 03 04 d html
4. ……
6. Grab
7. s Q Q
10. Grafana Q kk k k Header kk ElasticSearch Mysql Redis ETCD
12. • s iframe •
13. • A CSS selector Cookie • Q
15. kk • • a n Header • • • • IP URL • • • • User-Agent • Referer • X-Forwarded-For • Cookie •
16. • User-Agent UA • Referer Referer CSRF • X-Forwarded-For k
18. • proxy-server • Q • • • • xss
19. Q html
20. p 01 03 05 p html JavaScript CSS p 02 04 e 06
21. p 01 03 05 p html JavaScript CSS p 02 04 e 06
23. k T ul li 25 class id a list item T class id a img scroller slider album class id a T video player video
24. d
25. • • a
26. Task Server CPU Q Q n Client Client Client
27. h
29. H lucence ElasticSearch
31. & 6 8 4 @ 8. r