ReadWeb.ai : Free Instant Multi-Language Web Page Translation & Bilingual Viewing for Everyone

Redis heavy users should have encountered situations where using the DEL command to delete large keys, or when using FLUSHDB and FLUSHALL to delete databases containing a large number of keys, causing Redis to block; Additionally, when Redis cleans up expired data and eliminates data exceeding memory limits, encountering large keys can also cause server blocking.

To address the above issues, Redis 4.0 introduced the mechanism of lazyfree, which can move the deletion of keys or database operations to be executed in the background thread, thus minimizing server blocking as much as possible.

To address the above issues, Redis 4.0 introduced the mechanism of lazyfree, which allows deletion of keys or database operations to be executed in the background thread, thus minimizing server blocking as much as possible.

To address the above issues, Redis 4.0 introduced the mechanism of lazyfree, which allows deletion of keys or database operations to be performed in the background thread, thus minimizing server blocking as much as possible.

lazyfree mechanism

The principle of lazyfree is not difficult to imagine, which is to perform logical deletion when deleting objects, and then hand over the objects to the background, allowing the background thread to execute the real destruct, avoiding blocking caused by the large size of objects. This is how Redis implements lazyfree. Below, we will introduce the implementation of lazyfree through several commands.

The principle of lazyfree is not difficult to imagine, which is to perform logical deletion when deleting objects, and then hand over the objects to the background, allowing the background thread to execute the actual destruct, avoiding blocking caused by the large size of the object. This is how Redis implements lazyfree. Below, we will introduce the implementation of lazyfree through several commands.

First, let's take a look at the new unlink command:

void unlinkCommand(client *c) {
    delGenericCommand(c, 1);
}

void unlinkCommand(client *c) {
    delGenericCommand(c, 1);
}

void unlinkCommand(client *c) {
    delGenericCommand(c, 1);
}

The entry is very simple, just call delGenericCommand, the second parameter is 1 to indicate asynchronous deletion.

/* This command implements DEL and LAZYDEL. */
void delGenericCommand(client *c, int lazy) {
    int numdel = 0, j;

/* This command implements DEL and LAZYDEL. */
void delGenericCommand(client *c, int lazy) {
    int numdel = 0, j;

    for (j = 1; j < c->argc; j++) {
        expireIfNeeded(c->db,c->argv[j]);
        int deleted  = lazy ? dbAsyncDelete(c->db,c->argv[j]) :
                              dbSyncDelete(c->db,c->argv[j]);
        if (deleted) {
            signalModifiedKey(c->db,c->argv[j]);
            notifyKeyspaceEvent(REDIS_NOTIFY_GENERIC,
                "del",c->argv[j],c->db->id);
            server.dirty++;
            numdel++;
        }
    }
    addReplyLongLong(c,numdel);
}

    for (j = 1; j < c->argc; j++) {
        expireIfNeeded(c->db,c->argv[j]);
        int deleted  = lazy ? dbAsyncDelete(c->db,c->argv[j]) :
                              dbSyncDelete(c->db,c->argv[j]);
        if (deleted) {
            signalModifiedKey(c->db,c->argv[j]);
            notifyKeyspaceEvent(REDIS_NOTIFY_GENERIC,
                "del",c->argv[j],c->db->id);
            server.dirty++;
            numdel++;
        }
    }
    addReplyLongLong(c,numdel);
}

    for (j = 1; j < c->argc; j++) {
        expireIfNeeded(c->db,c->argv[j]);
        int deleted  = lazy ? dbAsyncDelete(c->db,c->argv[j]) :
                              dbSyncDelete(c->db,c->argv[j]);
        if (deleted) {
            signalModifiedKey(c->db,c->argv[j]);
            notifyKeyspaceEvent(REDIS_NOTIFY_GENERIC,
                "del",c->argv[j],c->db->id);
            server.dirty++;
            numdel++;
        }
    }
    addReplyLongLong(c,numdel);
}

The delGenericCommand function determines whether to delete synchronously or asynchronously based on the lazy parameter. We won't go into detail about the synchronous deletion logic as it hasn't changed much. Let's focus on the implementation of the new asynchronous deletion.

The delGenericCommand function determines whether to delete synchronously or asynchronously based on the lazy parameter. The logic for synchronous deletion remains unchanged, so we won't go into detail on that. Let's focus on the implementation of the new asynchronous deletion.

#define LAZYFREE_THRESHOLD 64
// First define the threshold for enabling background deletion. Only when the number of elements in the object exceeds this threshold will it be handed over to a background thread for deletion. If the object contains too few elements, there is no need to hand it over to a background thread, as thread synchronization also incurs a certain cost.
int dbAsyncDelete(redisDb *db, robj *key) {
    if (dictSize(db->expires) > 0) dictDelete(db->expires,key->ptr);
    // Clear the expiration time of the key to be deleted

#define LAZYFREE_THRESHOLD 64
// First define the threshold for enabling background deletion. Only when the number of elements in the object exceeds this threshold will it be handed over to the background thread for deletion. If the object contains too few elements, there is no need to hand it over to the background thread, as thread synchronization also incurs a certain cost.
int dbAsyncDelete(redisDb *db, robj *key) {
    if (dictSize(db->expires) > 0) dictDelete(db->expires,key->ptr);
    // Clear the expiration time of the key to be deleted

#define LAZYFREE_THRESHOLD 64
// First define the threshold for enabling background deletion. Only when the number of elements in the object exceeds this threshold will it be handed over to a background thread for deletion. If the object contains too few elements, there is no need to hand it over to a background thread, as thread synchronization also incurs a certain cost.
int dbAsyncDelete(redisDb *db, robj *key) {
    if (dictSize(db->expires) > 0) dictDelete(db->expires,key->ptr);
    // Clear the expiration time of the key to be deleted

    dictEntry *de = dictUnlink(db->dict,key->ptr);
    //dictUnlink returns a pointer to the entry in the database dictionary containing the key and removes that entry from the database dictionary (does not free resources)
    if (de) {
        robj *val = dictGetVal(de);
        size_t free_effort = lazyfreeGetFreeEffort(val);
        //lazyfreeGetFreeEffort is used to obtain the number of elements contained in the val object

    dictEntry *de = dictUnlink(db->dict,key->ptr);
    //dictUnlink returns a pointer to the entry in the database dictionary that contains the key and removes that entry from the database dictionary (does not free resources)
    if (de) {
        robj *val = dictGetVal(de);
        size_t free_effort = lazyfreeGetFreeEffort(val);
        //lazyfreeGetFreeEffort is used to obtain the number of elements contained in the val object

    dictEntry *de = dictUnlink(db->dict,key->ptr);
    //dictUnlink returns a pointer to the entry in the database dictionary that contains the key and removes that entry from the database dictionary (does not free resources)
    if (de) {
        robj *val = dictGetVal(de);
        size_t free_effort = lazyfreeGetFreeEffort(val);
        //lazyfreeGetFreeEffort is used to obtain the number of elements contained in the val object

        if (free_effort > LAZYFREE_THRESHOLD) {
            atomicIncr(lazyfree_objects,1);
            //Atomic operation to increment lazyfree_objects by 1 for info command to check how many objects are pending for background thread deletion
            bioCreateBackgroundJob(BIO_LAZY_FREE ,val,NULL,NULL);
            //This actually puts object val into the background thread's job queue
            dictSetVal(db->dict,de,NULL);
            //Set the val pointer in the entry to NULL to prevent duplicate deletion of val object when deleting database dictionary entries
        }
    }

        if (free_effort > LAZYFREE_THRESHOLD) {
            atomicIncr(lazyfree_objects,1);
            //Atomic operation to increment lazyfree_objects by 1 for info command to check how many objects are pending deletion by background threads
            bioCreateBackgroundJob(BIO_LAZY_FREE ,val,NULL,NULL);
            //Actually queue the object val to the background thread's job queue
            dictSetVal(db->dict,de,NULL);
            //Set the val pointer in the entry to NULL to prevent duplicate deletion of val object when deleting database dictionary entries
        }
    }

        if (free_effort > LAZYFREE_THRESHOLD) {
            atomicIncr(lazyfree_objects,1);
            //Atomic operation to increment lazyfree_objects by 1 for info command to check how many objects are pending for background thread deletion
            bioCreateBackgroundJob(BIO_LAZY_FREE ,val,NULL,NULL);
            //This is where the object val is actually put into the background thread's job queue
            dictSetVal(db->dict,de,NULL);
            //Set the val pointer in the entry to NULL to prevent duplicate deletion of val object when deleting database dictionary entries
        }
    }

    if (de) {
        dictFreeUnlinkedEntry(db->dict,de);
        //Delete the database dictionary entry, release resources
        return 1;
    } else {
        return 0;
    }
}

    if (de) {
        dictFreeUnlinkedEntry(db->dict,de);
        //Delete the database dictionary entry, release resources
        return 1;
    } else {
        return 0;
    }
}

    if (de) {
        dictFreeUnlinkedEntry(db->dict,de);
        //Delete the database dictionary entry, release resources
        return 1;
    } else {
        return 0;
    }
}

The above is the logic of asynchronous deletion. First, it will clear the expiration time, then call dictUnlink to remove the object to be deleted from the database dictionary, then check the size of the object (if it is too small, there is no need to delete it in the background), if it is large enough, hand it over to the background thread, and finally clean up the entry information in the database dictionary.

Based on the above logic, when unlinking a large volume key, the actual deletion is handled by a background thread, so it does not block Redis.

2. FLUSHALL, FLUSHDB Commands

4.0 added a new option to the flush command - async. When the async option is added after the flush command, it will enter the background deletion logic, as shown in the following code:

4.0 added a new option to the flush command - async. When the async option is added after the flush command, it will enter the background deletion logic. The code is as follows:

/* FLUSHDB [ASYNC]
 *
 * Flushes the currently SELECTed Redis DB. */
void flushdbCommand(client *c) {
    int flags;

/* FLUSHDB [ASYNC]
 *
 * Flushes the currently SELECTed Redis DB. */
void flushdbCommand(client *c) {
    int flags;

/* FLUSHDB [ASYNC]
 *
 * Flushes the currently SELECTed Redis DB. */
void flushdbCommand(client *c) {
    int flags;

    if (getFlushCommandFlags(c,&flags) == C_ERR) return;
    signalFlushedDb(c->db->id);
    server.dirty += emptyDb(c->db->id,flags,NULL);
    addReply(c,shared.ok);

    if (getFlushCommandFlags(c,&flags) == C_ERR) return;
    signalFlushedDb(c->db->id);
    server.dirty += emptyDb(c->db->id,flags,NULL);
    addReply(c,shared.ok);

    if (getFlushCommandFlags(c,&flags) == C_ERR) return;
    signalFlushedDb(c->db->id);
    server.dirty += emptyDb(c->db->id,flags,NULL);
    addReply(c,shared.ok);

    sds client = catClientInfoString(sdsempty(),c);
    serverLog(LL_NOTICE, "flushdb called by client %s", client);
    sdsfree(client);
}

    sds client = catClientInfoString(sdsempty(),c);
    serverLog(LL_NOTICE, "flushdb called by client %s", client);
    sdsfree(client);
}

    sds client = catClientInfoString(sdsempty(),c);
    serverLog(LL_NOTICE, "flushdb called by client %s", client);
    sdsfree(client);
}

/* FLUSHALL [ASYNC]
 *
 * Flushes the whole server data set. */
void flushallCommand(client *c) {
    int flags;

/* FLUSHALL [ASYNC]
 *
 * Flushes the whole server data set. */
void flushallCommand(client *c) {
    int flags;

/* FLUSHALL [ASYNC]
 *
 * Flushes the whole server data set. */
void flushallCommand(client *c) {
    int flags;

    if (getFlushCommandFlags(c,&flags) == C_ERR) return;
    signalFlushedDb(-1);
    server.dirty += emptyDb(-1,flags,NULL);
    addReply(c,shared.ok);
    ...
}

    if (getFlushCommandFlags(c,&flags) == C_ERR) return;
    signalFlushedDb(-1);
    server.dirty += emptyDb(-1,flags,NULL);
    addReply(c,shared.ok);
    ...
}

    if (getFlushCommandFlags(c,&flags) == C_ERR) return;
    signalFlushedDb(-1);
    server.dirty += emptyDb(-1,flags,NULL);
    addReply(c,shared.ok);
    ...
}

flushdb and flushall have similar logic. They both first call getFlushCommandFlags to obtain flags (used to indicate whether asynchronous deletion is used), and then call emptyDb to clear the database. When the first parameter is -1, it means all databases should be cleared.

flushdb and flushall have similar logic. They both first call getFlushCommandFlags to obtain flags (used to indicate whether asynchronous deletion is used), and then call emptyDb to clear the database. When the first parameter is -1, it means that all databases should be cleared.

long long emptyDb(int dbnum, int flags, void(callback)(void*)) {
    int j, async = (flags & EMPTYDB_ASYNC);
    long long removed = 0;

long long emptyDb(int dbnum, int flags, void(callback)(void*)) {
    int j, async = (flags & EMPTYDB_ASYNC);
    long long removed = 0;

long long emptyDb(int dbnum, int flags, void(callback)(void*)) {
    int j, async = (flags & EMPTYDB_ASYNC);
    long long removed = 0;

    if (dbnum < -1 || dbnum >= server.dbnum) {
        errno = EINVAL;
        return -1;
    }

    if (dbnum < -1 || dbnum >= server.dbnum) {
        errno = EINVAL;
        return -1;
    }

    if (dbnum < -1 || dbnum >= server.dbnum) {
        errno = EINVAL;
        return -1;
    }

    for (j = 0; j < server.dbnum; j++) {
        if (dbnum != -1 && dbnum != j) continue;
        removed += dictSize(server.db[j].dict);
        if (async) {
            emptyDbAsync(&server.db[j]);
        } else {
            dictEmpty(server.db[j].dict,callback);
            dictEmpty(server.db[j].expires,callback);
        }
    }
    return removed;
}

    for (j = 0; j < server.dbnum; j++) {
        if (dbnum != -1 && dbnum != j) continue;
        removed += dictSize(server.db[j].dict);
        if (async) {
            emptyDbAsync(&server.db[j]);
        } else {
            dictEmpty(server.db[j].dict,callback);
            dictEmpty(server.db[j].expires,callback);
        }
    }
    return removed;
}

    for (j = 0; j < server.dbnum; j++) {
        if (dbnum != -1 && dbnum != j) continue;
        removed += dictSize(server.db[j].dict);
        if (async) {
            emptyDbAsync(&server.db[j]);
        } else {
            dictEmpty(server.db[j].dict,callback);
            dictEmpty(server.db[j].expires,callback);
        }
    }
    return removed;
}

Upon entering emptyDb, the first steps are some validation checks. After passing the validation, the database is cleared, and synchronous deletion involves calling dictEmpty to iterate through all objects in the database and delete them (which can easily block Redis). The focus today is on the asynchronous deletion function emptyDbAsync.

Upon entering emptyDb, the first steps are some validation checks. After passing the validation, the database is cleared, and synchronous deletion involves calling dictEmpty to iterate through all objects in the database and delete them (which can easily block Redis). The focus today is on the asynchronous deletion in the emptyDbAsync function.

Upon entering emptyDb, the first steps are some validation checks. After passing the validation, the database is cleared, and synchronous deletion involves calling dictEmpty to iterate through all objects in the database and delete them (which can easily block Redis). The focus today is on the asynchronous deletion function emptyDbAsync.

/* Empty a Redis DB asynchronously. What the function does actually is to
 * create a new empty set of hash tables and scheduling the old ones for
 * lazy freeing. */
void emptyDbAsync(redisDb *db) {
    dict *oldht1 = db->dict, *oldht2 = db->expires;
    db->dict = dictCreate(&dbDictType,NULL);
    db->expires = dictCreate(&keyptrDictType,NULL);
    atomicIncr(lazyfree_objects,dictSize(oldht1));
    bioCreateBackgroundJob(BIO_LAZY_FREE,NULL,oldht1,oldht2);
}

/* Empty a Redis DB asynchronously. What the function does actually is to
 * create a new empty set of hash tables and scheduling the old ones for
 * lazy freeing. */
void emptyDbAsync(redisDb *db) {
    dict *oldht1 = db->dict, *oldht2 = db->expires;
    db->dict = dictCreate(&dbDictType,NULL);
    db->expires = dictCreate(&keyptrDictType,NULL);
    atomicIncr(lazyfree_objects,dictSize(oldht1));
    bioCreateBackgroundJob(BIO_LAZY_FREE,NULL,oldht1,oldht2);
}

/* Empty a Redis DB asynchronously. What the function does actually is to
 * create a new empty set of hash tables and scheduling the old ones for
 * lazy freeing. */
void emptyDbAsync(redisDb *db) {
    dict *oldht1 = db->dict, *oldht2 = db->expires;
    db->dict = dictCreate(&dbDictType,NULL);
    db->expires = dictCreate(&keyptrDictType,NULL);
    atomicIncr(lazyfree_objects,dictSize(oldht1));
    bioCreateBackgroundJob(BIO_LAZY_FREE,NULL,oldht1,oldht2);
}

Here, db->dict and db->expires are directly pointed to two newly created empty dictionaries, and then the original two dictionaries are placed in the task queue of the background thread. It's simple and efficient, and no longer afraid of blocking redis.

Here, directly point db->dict and db->expires to two newly created empty dictionaries, then put the original two dictionaries into the task queue of the background thread, simple and efficient, no longer afraid of blocking redis.

Here db->dict and db->expires are directly pointed to two newly created empty dictionaries, and then the original two dictionaries are thrown into the task queue of the background thread, simple and efficient, no longer afraid of blocking redis.

lazyfree thread

Introducing the lazyfree thread that really does the work next.

First of all, it is necessary to clarify a misconception. Many people mention Redis as a single-threaded in-memory database, but that's not the case. Although Redis places operations such as handling network communication and executing commands in the main working thread, there are also many background I/O threads diligently working on tasks such as closing files and flushing data, in addition to that. This time, a new member, the lazyfree thread, has joined the bio family.

First of all, it is necessary to clarify a misconception. Many people mention Redis as a single-threaded in-memory database, but that's not the case. Although Redis places operations such as handling network communication and executing commands in the main working thread, there are also many background I/O threads diligently working. For example, there are threads dedicated to handling heavy I/O operations like closing files and flushing data to disk. This time, a new member has joined the I/O family - the lazyfree thread.

First of all, it is necessary to clarify a misconception. Many people mention Redis as a single-threaded in-memory database, but that's not the case. Although Redis places operations like handling network communication and executing commands in the main working thread, there are also many background I/O threads diligently working on tasks such as closing files and flushing data, in addition to that, the bio family has welcomed a new member - the lazyfree thread.

void *bioProcessBackgroundJobs(void *arg) {
    ...
        if (type == BIO_LAZY_FREE) {
            /* What we free changes depending on what arguments are set:
             * arg1 -> free the object at pointer.
             * arg2 & arg3 -> free two dictionaries (a Redis DB).
             * only arg3 -> free the skiplist. */
            if (job->arg1)
                lazyfreeFreeObjectFromBioThread(job->arg1);
            else if (job->arg2 && job->arg3)
                lazyfreeFreeDatabaseFromBioThread(job->arg2, job->arg3);
            else if (job->arg3)
                lazyfreeFreeSlotsMapFromBioThread(job->arg3);
        }
    ...
}

void *bioProcessBackgroundJobs(void *arg) {
    ...
        if (type == BIO_LAZY_FREE) {
            /* What we free changes depending on what arguments are set:
             * arg1 -> free the object at pointer.
             * arg2 & arg3 -> free two dictionaries (a Redis DB).
             * only arg3 -> free the skiplist. */
            if (job->arg1)
                lazyfreeFreeObjectFromBioThread(job->arg1);
            else if (job->arg2 && job->arg3)
                lazyfreeFreeDatabaseFromBioThread(job->arg2, job->arg3);
            else if (job->arg3)
                lazyfreeFreeSlotsMapFromBioThread(job->arg3);
        }
    ...
}

void *bioProcessBackgroundJobs(void *arg) {
    ...
        if (type == BIO_LAZY_FREE) {
            /* What we free changes depending on what arguments are set:
             * arg1 -> free the object at pointer.
             * arg2 & arg3 -> free two dictionaries (a Redis DB).
             * only arg3 -> free the skiplist. */
            if (job->arg1)
                lazyfreeFreeObjectFromBioThread(job->arg1);
            else if (job->arg2 && job->arg3)
                lazyfreeFreeDatabaseFromBioThread(job->arg2, job->arg3);
            else if (job->arg3)
                lazyfreeFreeSlotsMapFromBioThread(job->arg3);
        }
    ...
}

redis named the newly added lazyfree thread BIO_LAZY_FREE, the background thread determines itself as a lazyfree thread based on the type, and then executes the corresponding function based on the parameters in bio_job.

Delete the object in the background, call decrRefCount to reduce the reference count of the object, and release resources when the reference count is 0.

When an object is deleted in the background, decrRefCount is called to reduce the object's reference count. The resources are released only when the reference count reaches 0.

     void lazyfreeFreeObjectFromBioThread(robj *o) {
         decrRefCount(o);
         atomicDecr(lazyfree_objects,1);
     }

     void lazyfreeFreeObjectFromBioThread(robj *o) {
         decrRefCount(o);
         atomicDecr(lazyfree_objects,1);
     }

     void lazyfreeFreeObjectFromBioThread(robj *o) {
         decrRefCount(o);
         atomicDecr(lazyfree_objects,1);
     }

Here it is also necessary to add that starting from redis 4.0, the reference count of key-value objects stored in redis is only in one of two states: 1 or shared. In other words, objects handed over to the lazyfree thread are always in state 1, thus avoiding multi-threading competition issues.

Here also need to add extra, since redis 4.0, the reference count of key-value objects stored in redis is only 1 or shared, in other words, the objects handed over to the lazyfree thread must be 1, which also avoids multi-threading competition issues.

Here it is also necessary to add that starting from redis 4.0, the reference count of key-value objects stored in redis is only in one of two states: 1 or shared. In other words, objects handed over to the lazyfree thread are always in the state of 1, thus avoiding multi-threading competition issues.

Clear the database dictionary in the background, call dictRelease to loop through the database dictionary and delete all objects.

Clear the database dictionary in the background, call dictRelease to loop through the database dictionary and delete all objects.

     void lazyfreeFreeDatabaseFromBioThread(dict *ht1, dict *ht2) {
         size_t numkeys = dictSize(ht1);
         dictRelease(ht1);
         dictRelease(ht2);
         atomicDecr(lazyfree_objects,numkeys);
     }

     void lazyfreeFreeDatabaseFromBioThread(dict *ht1, dict *ht2) {
         size_t numkeys = dictSize(ht1);
         dictRelease(ht1);
         dictRelease(ht2);
         atomicDecr(lazyfree_objects,numkeys);
     }

     void lazyfreeFreeDatabaseFromBioThread(dict *ht1, dict *ht2) {
         size_t numkeys = dictSize(ht1);
         dictRelease(ht1);
         dictRelease(ht2);
         atomicDecr(lazyfree_objects,numkeys);
     }

Delete the key-slots mapping table in the background. The original Redis will use it if running in cluster mode, but the self-developed cluster mode used by cloud Redis does not currently call this function.

     void lazyfreeFreeSlotsMapFromBioThread(rax *rt) {
     size_t len = rt->numele;
     raxFree(rt);
     atomicDecr(lazyfree_objects,len);
     }

     void lazyfreeFreeSlotsMapFromBioThread(rax *rt) {
     size_t len = rt->numele;
     raxFree(rt);
     atomicDecr(lazyfree_objects,len);
     }

Expired and Evicted

Redis supports setting expiration times and eviction, and the resulting deletion actions may also block Redis.

Redis supports setting expiration time and eviction, and the resulting deletion action may also block Redis.

So in addition to adding the unlink, flushdb async, and flushall async commands, Redis 4.0 has also added four background deletion configuration options:

So in addition to adding the unlink, flushdb async, and flushall async commands, Redis 4.0 this time also added four background deletion configuration options, which are:

So in addition to adding the unlink, flushdb async, and flushall async commands, Redis 4.0 this time also added four background deletion configuration options, namely:

slave-lazy-flush: Option to clear data after slave receives RDB file
lazyfree-lazy-eviction: Option for memory eviction when full
lazyfree-lazy-expire: Option for deleting expired keys
lazyfree-lazy-server-del: Internal deletion option, such as when renaming oldkey to newkey, if newkey exists, newkey needs to be deleted

slave-lazy-flush: Option to clear data after slave receives RDB file
lazyfree-lazy-eviction: Option for memory eviction when full
lazyfree-lazy-expire: Option for deleting expired keys
lazyfree-lazy-server-del: Internal deletion option, such as when renaming oldkey to newkey, if newkey exists, newkey needs to be deleted

slave-lazy-flush: Option to clear data after slave receives RDB file
lazyfree-lazy-eviction: Option for memory eviction when full
lazyfree-lazy-expire: Option for deleting expired keys
lazyfree-lazy-server-del: Internal deletion option, such as when renaming oldkey to newkey, if newkey exists, newkey needs to be deleted

The above 4 options are set to synchronous deletion by default, and the background deletion function can be enabled by config set \[parameter\] yes.

The above 4 options are set to be deleted synchronously by default, and the background deletion function can be enabled by config set \[parameter\] yes.

The above 4 options are set to be deleted synchronously by default, and the background deletion function can be enabled by using config set [parameter] yes.

The function deleted in the background has not been modified much. It is just to choose whether to call dbAsyncDelete or emptyDbAsync for asynchronous deletion based on the above four configuration items in the place where synchronous deletion was originally performed. The specific code can be seen at:

The function deleted in the background has not been modified much, but only in the place where synchronous deletion was originally performed, it now selects whether to call dbAsyncDelete or emptyDbAsync for asynchronous deletion based on the above 4 configuration items, specific code can be seen:

slave-lazy-flush

slave-lazy-flush

     void readSyncBulkPayload(aeEventLoop *el, int fd, void *privdata, int mask) {
         ...
         if (eof_reached) {
             ...
             emptyDb(
                 -1,
                 server.repl_slave_lazy_flush ? EMPTYDB_ASYNC : EMPTYDB_NO_FLAGS,
                 replicationEmptyDbCallback);
             ...
         }
         ...
     }

     void readSyncBulkPayload(aeEventLoop *el, int fd, void *privdata, int mask) {
         ...
         if (eof_reached) {
             ...
             emptyDb(
                 -1,
                 server.repl_slave_lazy_flush ? EMPTYDB_ASYNC : EMPTYDB_NO_FLAGS,
                 replicationEmptyDbCallback);
             ...
         }
         ...
     }

lazyfree-lazy-eviction

lazyfree-lazy-eviction

     int freeMemoryIfNeeded(long long timelimit) {
         ...
                 /* Finally remove the selected key. */
                 if (bestkey) {
                     ...
                     propagateExpire(db,keyobj,server.lazyfree_lazy_eviction);
                     if (server.lazyfree_lazy_eviction)
                         dbAsyncDelete(db,keyobj);
                     else
                         dbSyncDelete(db,keyobj);
                     ...
                 }
         ...
     }

lazyfree-lazy-expire

     int activeExpireCycleTryExpire(redisDb *db, struct dictEntry *de, long long now) {
         ...
         if (now > t) {
             ...
             propagateExpire(db,keyobj,server.lazyfree_lazy_expire);
             if (server.lazyfree_lazy_expire)
                 dbAsyncDelete(db,keyobj);
             else
                 dbSyncDelete(db,keyobj);
             ...
         }
         ...
     }

     int activeExpireCycleTryExpire(redisDb *db, struct dictEntry *de, long long now) {
         ...
         if (now > t) {
             ...
             propagateExpire(db,keyobj,server.lazyfree_lazy_expire);
             if (server.lazyfree_lazy_expire)
                 dbAsyncDelete(db,keyobj);
             else
                 dbSyncDelete(db,keyobj);
             ...
         }
         ...
     }

lazyfree-lazy-server-del

lazyfree-lazy-server-del

     int dbDelete(redisDb *db, robj *key) {
         return server.lazyfree_lazy_server_del ? dbAsyncDelete(db,key) :
                                                  dbSyncDelete(db,key);
     }

In addition, YunRedis has made some minor improvements to expiration and eviction.

Optimization of expire and evict

Redis will enter the activeExpireCycle loop to delete expired keys when idle. Each cycle will first calculate an execution time, and it will not traverse the entire database during the loop. Instead, it will randomly select a portion of keys to check for expiration. Therefore, sometimes time will not be exhausted (which will speed up the cleaning of expired keys when using asynchronous deletion), and the remaining time can be handed over to freeMemoryIfNeeded for execution.

void activeExpireCycle(int type) {
    ...
afterexpire:
    if (!g_redis_c_timelimit_exit &&
        server.maxmemory > 0 &&
        zmalloc_used_memory() > server.maxmemory)
    {
        long long time_canbe_used = timelimit - (ustime() - start);
        if (time_canbe_used > 0) freeMemoryIfNeeded(time_canbe_used);
    }
}