Skip to content
This repository has been archived by the owner on Jul 14, 2022. It is now read-only.

redis: ranges generate ZSet spam into redis table #100

Closed
scbizu opened this issue May 29, 2019 · 1 comment
Closed

redis: ranges generate ZSet spam into redis table #100

scbizu opened this issue May 29, 2019 · 1 comment
Assignees

Comments

@scbizu
Copy link
Member

scbizu commented May 29, 2019

Issue Description

With the yaml definition like this:

Blog:
  dbs: [redis]
  dbname: blog
  dbtable: user_blog
  fields:
    - ID: int64
      flags: [index]
    - Content: string
    - CreateDate: int64
    - UpdateDate: int64
primary: [ID]

In object.go, it will pass the o.primary.IsRange(), and create o.ranges:

if o.primary.IsRange() {
	index := NewIndex(o)
	index.FieldNames = o.primary.FieldNames
	o.ranges = append(o.ranges, index)
}
---
func (pk *PrimaryKey) IsRange() bool {
	c := len(pk.Fields)
	if c > 0 {
		return pk.Fields[c-1].IsNumber()
	}
	return false
}

And then, once we invoke our orm function addToPipeline, it will save the range info into redis table:

	//! ranges
	rg_key_0 := []string{
		"Id",
	}
	rg_pip_0 := IdOfUserRNGRelationRedisMgr().BeginPipeline(pipe.Pipeline)
	rg_rel_0 := IdOfUserRNGRelationRedisMgr().NewIdOfUserRNGRelation(strings.Join(rg_key_0, ":"))
	score_rg_0, err := orm.ToFloat64(obj.Id)
	if err != nil {
		return err
	}
	rg_rel_0.Score = score_rg_0
	rg_rel_0.Value = key
	if err := rg_pip_0.ZSetAdd(rg_rel_0); err != nil {
		return err
	}
	if expire > 0 {
		pipe.Expire(keyOfObject(obj, key), expire)
	}

But, we do not need this kind of range, we only just need the primary mechanism .

What 's more worse is that we store the 50%+ redis ranges(spam) into our redis , and they all have no TTL !!!

Workarounds

How about add the norange flag into our redis column definition ?

Columns with norange flag will quit the range code generation, and then we can change our primary.key.go into this:

func (pk *PrimaryKey) IsRange() bool {
      var fs []*Fields
      for _,f := range pk.Fields{
          if  !f.IsRange(){
            continue
         }
          fs = append(fs,f)
       }
   	c := len(fs)
	if c > 0 {
		return pk.Fields[c-1].IsNumber()
	}
  return false
}
@scbizu
Copy link
Member Author

scbizu commented May 29, 2019

It will not generate IDRange anymore.

Blog:
  dbs: [redis]
  dbname: blog
  dbtable: user_blog
  fields:
    - ID: int64
      flags: [norange]
    - Content: string
    - CreateDate: int64
    - UpdateDate: int64
primary: [ID]

@scbizu scbizu closed this as completed Sep 4, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

1 participant