Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix #2555: add backreference in CQs #3876

Merged
merged 11 commits into from
Sep 3, 2015
Merged

fix #2555: add backreference in CQs #3876

merged 11 commits into from
Sep 3, 2015

Conversation

dgnorton
Copy link
Contributor

Add new query syntax to allow the following in CQs:

INTO "1hPolicy".:MEASUREMENT FROM /.*/

In this example, the regular expression in the FROM clause would expand to all measurements and the points selected from each of those measurements would be written into the "1hPolicy" retention policy with the same measurement names.

@otoolep
Copy link
Contributor

otoolep commented Aug 28, 2015

Looks like Circle blew up due to memory pressure. I've seen this before, and it can happen if the code allocates huge buffers by mistake, perhaps in a loop.

@@ -699,6 +702,8 @@ func (p *Parser) parsePrivilege() (Privilege, error) {
// parseSelectStatement parses a select string and returns a Statement AST object.
// This function assumes the SELECT token has already been consumed.
func (p *Parser) parseSelectStatement(tr targetRequirement) (*SelectStatement, error) {
fmt.Println("parseSelectStatement: start")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

forgot to take this out :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bah!! good catch.

@pauldix
Copy link
Member

pauldix commented Aug 28, 2015

Can you add a test to ensure that GROUP BY * works with this as well to bring over the tag data? Would actually like to see more integration level tests if possible.

@otoolep
Copy link
Contributor

otoolep commented Aug 28, 2015

@dgnorton -- did you work out what caused Circle to fail due to memory pressure?

@@ -346,7 +351,11 @@ func (s *Service) runContinuousQueryAndWriteResult(cq *ContinuousQuery) error {
}

if s.loggingEnabled {
s.Logger.Printf("wrote %d point(s) to %s.%s.%s", len(points), cq.intoDB(), cq.intoRP(), cq.Info.Name)
db := cq.intoDB()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is purely a logging thing? This looks like logic that should be in the cq object, no?

@dgnorton
Copy link
Contributor Author

@otoolep I added a Parent Node field to type Measurement which caused json.Marshal call in the tests to go into an infinite loop.

@dgnorton dgnorton force-pushed the dgn-cq-backrefs branch 2 times, most recently from 5b550d7 to a3a38ad Compare September 2, 2015 17:40
@dgnorton
Copy link
Contributor Author

dgnorton commented Sep 3, 2015

@pauldix added integration tests but issue #3968 prevents the GROUP BY * test from passing, so I've disabled that test until the issue is fixed.

exp: `{"results":[{"series":[{"name":"cpu","columns":["time","count","host","region","value"],"values":[["` + interval2.UTC().Format(time.RFC3339Nano) + `",3,null,null,null]]},{"name":"gpu","columns":["time","count","host","region","value"],"values":[["` + interval1.UTC().Format(time.RFC3339Nano) + `",2,null,null,null],["` + interval0.UTC().Format(time.RFC3339Nano) + `",1,null,null,null]]}]}]}`,
params: url.Values{"db": []string{"db0"}},
},
// TODO: restore this test once this is fixed: https://github.com/influxdb/influxdb/issues/3968
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there anything unique about this test other than duplicate tags? Also, the idiom I like to use is put a skip in my test struct, then the code isn't commented out, and if you do a verbose log, you can t.Logf out any skipped table tests.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It checks the results of a CQ with group by *.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, that's what I figured, just wanted to check.

@corylanou
Copy link
Contributor

This might be beyond the scope of this PR, but how do we handle:

  • A long running CQ or potentially a hung CQ?
  • Can the same CQ run again if the previous one is still running?

Looks good. +1

@dgnorton
Copy link
Contributor Author

dgnorton commented Sep 3, 2015

@corylanou

  • Long running - wait for it
  • Hung - not sure. Maybe ExecuteQuery times out?
  • I don't think it can run again because the service waits for results.

I mocked enough of the interfaces in the CQ service tests that it should be relatively easy to add tests for those things but I don't think they're currently tested.

dgnorton added a commit that referenced this pull request Sep 3, 2015
@dgnorton dgnorton merged commit 1c29e59 into master Sep 3, 2015
@QQShan
Copy link

QQShan commented Aug 5, 2017

I want to add suffix for measurement names ,may by like that
INTO "1hPolicy".'$1_sss' FROM /.*/
is that supported?if it is not supported ,how to add suuffix for all measurement names?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants