5 从Thriftpy中学习RPC协议 张汝家

CodeWarrior

2019/10/11 发布于 技术 分类

文字内容
1. thriftpy 张汝家 rujiazhang@foxmail.com
2. thriftpy 0. 微服务的RPC 1. thrift作为IDL
3. 微服务 • microservice • 轻量量级的服务粒度、统⼀一的通信协议、交付⾃自动化、快速 迭代且⽆无历史包袱的互联⽹网企业
4. RPC • 微服务间通信:RPC(Remote Procedure Call) • 本地调⽤用->远程调⽤用
5. RPC • • • 协议约定问题 • 语法 • 数据表示 传输问题 • 通信性能 • 通信质量量 服务发现问题
6. thrift • thrift:节约;节俭 • 由facebook开发,转交Apache,后fb⼜又再次开源 • 接⼝口描述语⾔言(Interface description language, IDL) • RPC协议 • 跨语⾔言 • 论⽂文:http://thrift.apache.org/static/files/thrift-20070401.pdf
7. thriftpy 0. 微服务的RPC 1. thrift作为IDL
8. thrift IDL file demo # person.thrift const i16 DEFAULT_LIST_SIZE = 10 typedef i32 timestamp enum PhoneType { MOBILE = 0, HOME, WORK, } struct PhoneNumber { 1: optional PhoneType type = PhoneType.MOBILE, 2: optional string number, } struct 1: 2: 3: } Person { required string name, optional list phones, optional timestamp created_at, exception PersonNotExistsError { 1: optional string message = "Person Not Exists!", } service PersonService { bool add(1: required Person person); bool remove(1: string name) throws (1: PersonNotExistsError not_exists); Person get(1: string name) throws (1: PersonNotExistsError not_exists); }
9. thrift IDL Types • Services • Base Types • bool • byte • i16/i32/i64 • double • string • Structs • Containers • Exceptions
10. thriftpy 0. 微服务的RPC 1. thrift作为IDL 2. thrift作为RPC协议
11. thrift RPC协议
12. thrift RPC协议 • Server: single-threaded, event-driven etc • Processor: compiler generated • Protocol: binary, JSON, compact etc • Transport: raw TCP, HTTP etc
13. thrift transport • • 提供I/O抽象 • 典型的基于TCP栈的使⽤用流式socket • ⾮非典型情况:file, disk… 处理理读写数据 • • interface: open, close, read, write, flush raw transport & transport wrapper
14. thrift protocol • 基于transport提供的I/O • 提供内存中的数据结构与⽹网络流的相互转化 • interface: readDataStructBegin/writeDataStructEnd • DataStruct: message, struct, field, map, list, string, i16… • 编码/解码、序列列号/反序列列化… • JSON、XML、Plain Text、Binary、Compact Binary… • version
15. thrift processor • • Processor • 基于protocol提供的I/O • 概括读写能⼒力力 • 组合IDL的⽣生成代码 Server • 最上层
16. thrift RPC协议框架
17. thriftpy 0. 微服务的RPC 1. thrift作为IDL 2. thrift作为RPC协议 3. thriftpy的实现
18. thriftpy socket+binary实现server
19. thriftpy socket+binary实现client
20. thriftpy socket+binary protocol的读写
21. thriftpy 0. 微服务的RPC 1. thrift作为IDL 2. thrift作为RPC协议 3. thriftpy的实现 4. ⼀一个socket+binary实现示例例
22. thriftpy generate code • Person.thrift —> Person.py
23. thriftpy generate code • TPayload • thrift_spec • default_spec • struct • Service.api_args • Service.api_result
24. thriftpy socket+binary 实现示例 # person_client.py # -*- coding: utf-8 -*import time import thriftpy2 from thriftpy2.rpc import client_context person_thrift = thriftpy2.load("person.thrift", module_name="person_thrift") def main(): with client_context(person_thrift.PersonService, '127.0.0.1', 6000) as c: name = u'张汝家' number = '156****7119' phone_number = person_thrift.PhoneNumber(person_thrift.PhoneType.MOBILE, number) person = person_thrift.Person(name, [phone_number], int(time.time())) print c.add(person) print c.get(name) try: print c.get(name[1:]) except person_thrift.PersonNotExistsError as e: print e.message print c.remove(name) if __name__ == '__main__': main()
25. thriftpy socket+binary 实现示例 # person_server.py # -*- coding: utf-8 -*import thriftpy2 from thriftpy2.rpc import make_server person_thrift = thriftpy2.load("person.thrift", module_name="person_thrift") class Dispatcher(object): def __init__(self): self.persons = dict() def add(self, person): self.persons[person.name] = person return True def remove(self, name): if name in self.persons:'>self.persons: self.persons.pop(name) return True raise person_thrift.PersonNotExistsError(u"{} not exists".format(name)) def get(self, name): if name in self.persons:'>self.persons: return self.persons[name] raise person_thrift.PersonNotExistsError(u"{} not exists".format(name)) def main(): server = make_server(person_thrift.PersonService, Dispatcher(), '127.0.0.1', 6000) print("serving...") server.serve() if __name__ == '__main__': main()
26. thriftpy socket+binary 实现示例 • run person_server.py • run person_client.py
27. thriftpy socket+binary 实现示例 def pack_i8(byte): return struct.pack("!b", byte) def pack_string(string): return struct.pack("!i%ds" % len(string), len(string), string) def unpack_i32(buf): return struct.unpack("!i", buf)[0] def unpack_double(buf): return struct.unpack("!d", buf)[0] • 最基本的转换 • String的转换 • Structs, Containers, Exceptions, Services的转换
28. thriftpy socket+binary 实现示例
29. thriftpy socket+binary 实现示例
30. thriftpy more • transport Wrap • buffered, framed, memory • http client & http server • version • tracking • header