从HTTP性能基准测试谈Vert.x高性能的秘密:从JIT编译到网络优化 英文

微风

2019/03/24 发布于 技术 分类

文字内容
1. Real-world HTTP performance benchmarking, lessons learned Julien Viet QCon Shangai
3. Once upon a time
4. Round #8
5. Every one was happy
6. But one day...
7. Round #14
8. Round #14
9. Round #14
10. Round #14
11. Round #14
12. Real-world HTTP performance benchmarking, lessons learned
13. Julien Viet Open source developer for 16+ years @vertx_project lead Principal software engineer at Marseille JUG Leader ! https://www.julienviet.com/ " http://github.com/vietj # @julienviet  https://www.mixcloud.com/cooperdbi/
14. Eclipse Vert.x Open source project started in 2012 Eclipse / Apache licensing A toolkit for building reactive applications for the JVM 8K ⋆ on " Built on top of ! https://vertx.io # @vertx_project
15. Techempower Framework Benchmark ✓ Performance of production grade deployments of real-world application frameworks and platforms ✓ 464 frameworks - 26 languages ✓ Community ✓ Physical of contributors on GitHub server or cloud (Azure)
16. 6 benchmarks ✓ "/plaintext", "/json" # ✓ "/db", ! "/queries", "/updates", "/fortunes" # ! "
17. Things to remember ✓ Benchmarking is hard ✓ Benchmarking != load testing ✓ Measure ✓ Be critic don't guess
18. The lab
22. /plaintext
23. Benchmark ✓ Simple Hello World ✓ 16,384 concurrent connections ✓ HTTP ✓ No pipelining (16) back-end ✓ Heavily CPU bound
24. Keep-alive GET OK PUT OK GET OK
25. Head of line blocking
26. Pipelining GET PUT GET OK OK OK
27. Our weapons ✓ Async-profiler ✓ Jitwatch ✓ Wireshark + Flame graphs
28. Code inlining
29. process error process request process body
30. process error process request process body
31. process error process request process body reduce method size to favor inlining
32. b2073fa091d64a1dfe06699bca1a8befddb5a805 process error process request process body $ 2. inline by hand
33. Batch to amortise costs
34. chctx.fireChannelRead(msg) // class VertxHandler void channelRead(Object msg) { Connection conn = getConnection(); Context ctx = conn.getContext(); context.executeFromIO(conn::startRead()); channelRead(conn, msg); } // class VertxHttpHandler extends VertxHandler void startRead() { ... } void channelRead(Connection conn, Object msg) { conn.handleMessage(msg); } void handleMessage(Object msg) { ... }
35. 799df9e602eabcd51b56052e20cc7d05134!901 chctx.fireChannelRead(msg) Batch here // class VertxHandler public void channelRead(ChannelHandlerContext chctx, Object msg) { Connection conn = getConnection(); Context ctx = conn.getContext(); context.executeFromIO(() -> { conn.startRead(); conn.handleMessage(msg); }); } void startRead() { } void handleMessage(Object msg) { ... }
36. The fastest code is the code that never runs
37. Netty Vert.x Application req.response() .end("Hello World");
38. Netty Vert.x void end(Bu!er bu!er) { FullHttpResponse msg = ... queueForWrite(msg); } Application req.response() .end("Hello World");
39. Netty Vert.x void end(Bu!er bu!er) { FullHttpResponse msg = ... queueForWrite(msg); } void queueForWrite(Object msg) { needsFlush = true; channel.write(encode(obj)); } Application req.response() .end("Hello World");
40. Netty Vert.x void end(Bu!er bu!er) { FullHttpResponse msg = ... queueForWrite(msg); } ChannelFuture write(Object msg) { return pipeline.write(msg); void queueForWrite(Object msg) { } needsFlush = true; channel.write(encode(obj)); } Application req.response() .end("Hello World");
41. Netty Vert.x void end(Bu!er bu!er) { FullHttpResponse msg = ... queueForWrite(msg); } ChannelFuture write(Object msg) { return pipeline.write(msg); void queueForWrite(Object msg) { } needsFlush = true; channel.write(encode(obj)); } // default implementation (inherited) void write(ChannelHandlerContext ctx, Object msg, ChannelPromise promise) { ctx.write(msg, promise); } Application req.response() .end("Hello World");
42. Netty Vert.x void end(Bu!er bu!er) { FullHttpResponse msg = ... queueForWrite(msg); } ChannelFuture write(Object msg) { return pipeline.write(msg); void queueForWrite(Object msg) { } needsFlush = true; channel.write(encode(obj)); } void write(Object msg, ChannelPromise promise) { next.invoke(msg, promise) } // default implementation (inherited) void write(ChannelHandlerContext ctx, Object msg, ChannelPromise promise) { ctx.write(msg, promise); } Application req.response() .end("Hello World");
43. Netty Vert.x void end(Bu!er bu!er) { FullHttpResponse msg = ... queueForWrite(msg); } ChannelFuture write(Object msg) { return pipeline.write(msg); void queueForWrite(Object msg) { } needsFlush = true; channel.write(encode(obj)); } void write(Object msg, ChannelPromise promise) { next.invoke(msg, promise) } // default implementation (inherited) void write(ChannelHandlerContext ctx, Object msg, ChannelPromise promise) { ctx.write(msg, promise); } Application req.response() .end("Hello World");
44. Netty the fastest code is the code that never runs void write(Object msg, ChannelPromise promise) { next.invoke(msg, promise) } Vert.x void end(Bu!er bu!er) { FullHttpResponse msg = ... queueForWrite(msg); } Application req.response() .end("Hello World"); void queueForWrite(Object msg) { needsFlush = true; chctx.write(encode(obj)); } 217b17c78cd54103ae98557510a7ac431e17c5ea
45. Reduce object allocation
46. Netty Vert.x void end(Bu!er bu!er) { FullHttpResponse msg = ... queueForWrite(msg); } void write(Object msg) { write(msg, newPromise()); } void write(Object msg, ChannelPromise promise) { next.invoke(msg, promise) } void queueForWrite(Object msg) { needsFlush = true; chctx.write(encode(obj)); } Application req.response() .end("Hello World");
47. Netty Vert.x void end(Bu!er bu!er) { FullHttpResponse msg = ... queueForWrite(msg); } void write(Object msg) { write(msg, newPromise()); } void write(Object msg, ChannelPromise promise) { next.invoke(msg, promise) } void queueForWrite(Object msg) { needsFlush = true; chctx.write(encode(obj)); } Application req.response() .end("Hello World");
48. Netty Vert.x void end(Bu!er bu!er) { FullHttpResponse msg = ... queueForWrite(msg); } void queueForWrite(Object msg) { needsFlush = true; chctx.write(obj, channel.voidPromise()); } void write(Object msg, ChannelPromise promise) { next.invoke(msg, promise) } Application req.response() .end("Hello World"); reduce GC allocation by using VoidPromise 6b9788dec6e1147782a3a7017ead067778095cba
49. Cache expensive operations
50. void setConnection(Connection conn) { this.conn = conn; } Vert.x void channelReadComplete(ChannelHandlerContext ctx) { Runnable task = conn::endReadAndFlush(); // Need to use executeFromIO to avoid race conditions context.executeFromIO(task); } void endReadAndFlush() { if (needFlush) { needFlush = false; channel.flush(); } }
51. void setConnection(Connection conn) { this.conn = conn; } Vert.x void channelReadComplete(ChannelHandlerContext ctx) { Runnable task = conn::endReadAndFlush(); // Need to use executeFromIO to avoid race conditions context.executeFromIO(task); } void endReadAndFlush() { if (needFlush) { needFlush = false; channel.flush(); } } Called for every each flush
52. void setConnection(Connection conn) { this.conn = conn; this.task = conn::endReadAndFlush();'>conn::endReadAndFlush(); } Vert.x void channelReadComplete(ChannelHandlerContext ctx) { Runnable task = conn::endReadAndFlush();'>conn::endReadAndFlush(); // Need to use executeFromIO to avoid race conditions context.executeFromIO(task); } void endReadAndFlush() { if (needFlush) { needFlush = false; channel.flush(); } } Now called when the connection is created
53. Extra optimisations ✓ Faster HTTP header encoding ✓ Cache complex conditions
54. Round #15
55. Plaintext recap ✓A very aggressive benchmark ✓ Bottleneck ✓ Less are CPU and networking flushing is more
56. /db benchmark
57. /db ✓ Choice to use PostgreSQL ✓ Determine Database ? ✓ 256 the actual bottleneck: CPU ? Network ? concurrent connections: non-blocking versus blocking
58. First step ✓ First improvement: the /updates was actually not using a transaction
59. The reactive PostgreSQL client ✓ Goals - Simple, clean and straightforward API - Performant - Be a client - Lightweight ✓ Non goals - Be a driver - Be an abstraction
60. // Connect directly PgClient.connect(uri, connection -> { // Handle result }); // Or create a pool of connections PgClient pool = PgClient.pool(uri); pool.getConnection(connection -> { // Handle result });
61. // Sequential queries connection.query(query1, result1 -> { // Got result 1 connection.query(query2, result2 -> { // Got result 2 }); });
62. // What if we do ? connection.query(query1, result1 -> { // Got result 1 }); connection.query(query2, result2 -> { // Got result 2 }); - the 2 queries executes concurrently ? query1 executes then query2 ? query1 executes, query2 executes after ? QUIZ
63. Head of line blocking ✓ PostgreSQL ✓ Send process one request at a time the response after processing ✓ Sounds familiar ?
64. Let's pipeline it
65. The Reactive Postgres Client ✓A PostgreSQL client ✓ Simple and direct API ✓ Focusing on performance and low overhead
66. Other cool features ✓ Direct memory to object without intermediary memory copy ✓ Efficient ✓ RxJava support ✓ Domain ✓ Proxy flush to minimise expensive system calls sockets support support
67. Round #15
75. Let there be pipelining