读懂才会用 : Redis的多线程

发布于: 2020 年 05 月 06 日
读懂才会用 : Redis的多线程

2020.5.2日,Redis 6.0.1正式发布。除了增加新功能和新的API,支持多线程是最大变化。本文以简化的方式理解Redis线程模型的演进。

Redis单线程

严格讲,Redis 并不是单线程。有后台线程在工作,处理一些较为缓慢的操作,例如无用连接的释放、大key的删除等。client端命令的请求获取 (socket 读)、解析、执行、内容返回 (socket 写) 都是由一个线程处理,因此我们常说的“单线程”指的是处理核心处理的线程只有一个。

处理流程

如下图:

Redis5单线程模型

Redis采用evport,epoll,kqueue和select四种方式实现多路复用,提升链接处理,Linux平台默认是epoll

参考ae.c文件L47

/* Include the best multiplexing layer supported by this system.
* The following should be ordered by performances, descending. */
#ifdef HAVE_EVPORT
#include "ae_evport.c"
#else
#ifdef HAVE_EPOLL
#include "ae_epoll.c"
#else
#ifdef HAVE_KQUEUE
#include "ae_kqueue.c"
#else
#include "ae_select.c"
#endif
#endif
#endif

优点 VS 缺点

优势:

1、不存在锁的问题

2、避免线程间CPU切换

缺点:

1、单线程无法利用多CPU

2、串行操作,某个操作“出问题”会“阻塞”后续操作

Redis 6 多线程

处理流程

Redis 6 多线程流程

默认并不开启多线程,需要参数设置,IO_THREADS_MAX_NUM 最大为128

network.c文件 L2886

/* Initialize the data structures needed for threaded I/O. */
void initThreadedIO(void) {
io_threads_active = 0; /* We start with threads not active. */
/* Don't spawn any thread if the user selected a single thread:
* we'll handle I/O directly from the main thread. */
if (server.io_threads_num == 1) return;
if (server.io_threads_num > IO_THREADS_MAX_NUM) {
serverLog(LL_WARNING,"Fatal: too many I/O threads configured. "
"The maximum number is %d.", IO_THREADS_MAX_NUM);
exit(1);
}
/* Spawn and initialize the I/O threads. */
for (int i = 0; i < server.io_threads_num; i++) {
/* Things we do for all the threads including the main thread. */
io_threads_list[i] = listCreate();
if (i == 0) continue; /* Thread 0 is the main thread. */
/* Things we do only for the additional threads. */
pthread_t tid;
pthread_mutex_init(&io_threads_mutex[i],NULL);
io_threads_pending[i] = 0;
pthread_mutex_lock(&io_threads_mutex[i]); /* Thread will be stopped. */
if (pthread_create(&tid,NULL,IOThreadMain,(void*)(long)i) != 0) {
serverLog(LL_WARNING,"Fatal: Can't initialize IO thread.");
exit(1);
}
io_threads[i] = tid;
}
}

加入队列

network.c文件L3040

/* Return 1 if we want to handle the client read later using threaded I/O.
* This is called by the readable handler of the event loop.
* As a side effect of calling this function the client is put in the
* pending read clients and flagged as such. */
int postponeClientRead(client *c) {
if (io_threads_active &&
server.io_threads_do_reads &&
!ProcessingEventsWhileBlocked &&
!(c->flags & (CLIENT_MASTER|CLIENT_SLAVE|CLIENT_PENDING_READ)))
{
c->flags |= CLIENT_PENDING_READ;
listAddNodeHead(server.clients_pending_read,c);
return 1;
} else {
return 0;
}
}
/* When threaded I/O is also enabled for the reading + parsing side, the
* readable handler will just put normal clients into a queue of clients to
* process (instead of serving them synchronously). This function runs
* the queue using the I/O threads, and process them in order to accumulate
* the reads in the buffers, and also parse the first command available
* rendering it in the client structures. */
int handleClientsWithPendingReadsUsingThreads(void) {

基础类图

优点 VS 缺点

优点:

1、提高响应速度,充分使用CPU

缺点:

1、增加了代码复杂性

总结

  1. Redis的多路复用技术,支持epoll、kqueue、selector

  2. 5.0版本及以前,处理客户端请求的线程只有一个,串行处理

  3. 6.0版本引入了worker Thread,只处理网络IO读取和写入,核心IO负责串行处理客户端指令

关注我

如果您在微信阅读,请您点击链接 关注我 ,如果您在 PC 上阅读请扫码关注我,欢迎与我交流随时指出错误

发布于: 2020 年 05 月 06 日 阅读数: 270
用户头像

小眼睛聊技术

关注

欢迎关注公众号“小眼睛聊技术” 2018.11.12 加入

互联网老兵,关注产品、技术、管理

评论 (1 条评论)

发布
没有更多了
读懂才会用 : Redis的多线程