利用Redis简化延迟任务处理(使用redis做延迟处理)

Delay tasks have become increasingly important for distributed system, such as order expiration, timed task, etc. Traditional solutions using cron solutions can achieve scheduling, but it has certn limitations, such as the lack of flexibility, the non-real-time experience, and the lack of scalability. Fortunately, with the emergence of Redis technology, We can use Redis to easily implement delayed job processing.

Redis is an open source, high performance distributed memory database, which can store k-v structures, and support data persistence (with AOF and RDB). Therefore, we can use Redis as a message queue to delay tasks.

The principle of Redis delaying task is to save tasks in Redis database with a specific score, and expiration time. When the score expired, Redis will return all tasks with expired scores, and then our job can be executed.

The specific solutions are as follows:

1. Add a delayed task list:

“`py

conn.zadd(REDIS_KEY, {COMMAND_NAME + ‘_’ + NOW_TIME: NOW_TIME}


2. Set up a scheduled task of 1 minute to get the task that has expired:

```py
def get_expired_task():
valid_time = time.time()
expired_task = conn.zrangebyscore(REDIS_KEY, 0, valid_time)
for item in expired_task:
job_name = item.rsplit('_', 1)[0]
# Execute the job here
conn.zrem(REDIS_KEY, item)

3. Add corresponding scheduled task to cron according to the granularity of the task:

“`sh

* * * * * python get_expired_task.py


Redis is not only efficient and flexible, but also supports distributed clustering, which is suitable for large scale delayed tasks. With the support of cron tasks and python access Redis, we can easily implement timed task scheduling and delay job processing.

数据运维技术 » 利用Redis简化延迟任务处理(使用redis做延迟处理)