Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing hashTypeTryConversion call that leads unexpected encoding type (ziplist) #4550

Open
peter-sj opened this issue Dec 19, 2017 · 0 comments

Comments

@peter-sj
Copy link

peter-sj commented Dec 19, 2017

  1. RM_HashSet in module.c. Seems func "hashTypeTryConversion" is not called so even ht encoding should be used (either length or element size limit is reached), data is still ziplist encoded.

  2. Seems func hincrbyCommand and hincrbyfloatCommand (in t_hash.c) also missing the call, not sure if it is intended.

The impact is defrag not worked as expected as those ptr will skip moving/compacting because they could be allocated in large bin/run (otherwise in small bin/run, depends on jemalloc's "tcache_maxclass" variable in jemalloc lib, tcache.c), and defrag skip moving for such case, i.e, je_get_defrag_hint return 0. So in our case, defrag keep running but no progress made.

Also I guess that might cause perf issue when size of the ziplist is large.

JEMALLOC_EXPORT int JEMALLOC_NOTHROW
je_get_defrag_hint(void* ptr, int *bin_util, int *run_util) {
    int defrag = 0;
    arena_chunk_t *chunk = (arena_chunk_t *)CHUNK_ADDR2BASE(ptr);
    if (likely(chunk != ptr)) { /* indication that this is not a HUGE alloc */
        size_t pageind = ((uintptr_t)ptr - (uintptr_t)chunk) >> LG_PAGE;
        size_t mapbits = arena_mapbits_get(chunk, pageind);
       if (likely((mapbits & CHUNK_MAP_LARGE) == 0)) { /* indication that this is not a LARGE alloc */
            arena_t *arena = extent_node_arena_get(&chunk->node);
            size_t rpages_ind = pageind - arena_mapbits_small_runind_get(chunk, pageind);
            arena_run_t *run = &arena_miscelm_get_mutable(chunk, rpages_ind)->run;
            arena_bin_t *bin = &arena->bins[run->binind];
            tsd_t *tsd = tsd_fetch();
            malloc_mutex_lock(tsd_tsdn(tsd), &bin->lock);
            /* runs that are in the same chunk in as the current chunk, are likely to be the next currun */
            if (chunk != (arena_chunk_t *)CHUNK_ADDR2BASE(bin->runcur)) {
                arena_bin_info_t *bin_info = &arena_bin_info[run->binind];
                size_t availregs = bin_info->nregs * bin->stats.curruns;
                *bin_util = (bin->stats.curregs<<16) / availregs;
                *run_util = ((bin_info->nregs - run->nfree)<<16) / bin_info->nregs;
                defrag = 1;
            }
        malloc_mutex_unlock(tsd_tsdn(tsd), &bin->lock);
        }
    }
    return defrag;
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant