Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Master list: Feature completion for complex-valued columns #3690

Closed
10 of 11 tasks
MichaelChirico opened this issue Jul 10, 2019 · 6 comments
Closed
10 of 11 tasks

Master list: Feature completion for complex-valued columns #3690

MichaelChirico opened this issue Jul 10, 2019 · 6 comments
Milestone

Comments

@MichaelChirico
Copy link
Member

@MichaelChirico MichaelChirico commented Jul 10, 2019

As a follow-up to #1444 and #3689, I think it's useful to keep a list of all the features missing for users who want to do work involving complex-valued columns z:

  • 1. Sorting by z (#1703); by extension frank(z), rowid(z), dcast, setkey, uniqueN
  • 2. Grouping by z
  • 3. z I/O (fread (without colClasses) & fwrite; overlap with #2247)
  • 4. CJ(z)
  • 5. Assign z by group
  • 6. GForce on z
  • 7. Joins on z wait until a user requests it with justification (that part of bmerge.c is complex)
  • 8. coalesce(z)
  • 9. shift(z)
  • 10. unique(z)
  • 11. := #3639

v1.12.4 NEWS contains :

image


Examples

DT = data.table(a = 1:5, z = 6:10 + (5:1)*1i)

1.

DT[order(z)]

Error in forderv(ans, cols, sort = TRUE, retGrp = FALSE, order = if (decreasing) -order else order, :
Column 1 of by= (1) is type 'complex', not yet supported. Please use the by= argument to specify columns with types that are supported. See NEWS item in v1.12.2 for more information.

2.

DT[ , .N, by = z]

Error in [.data.table(DT, , .N, by = z) :
column or expression 1 of 'by' or 'keyby' is type complex. Do not quote column names. Usage: DT[,sum(colC),by=list(colA,month(colB))]

3.

fwrite(DT)

Error in fwrite(DT) :
Column 2's type is 'complex' - not yet implemented in fwrite.

fread('a\n1+i')
#         a
#    <char>
# 1:    1+i

4.

DT[CJ(a, z)]

Error: Column 1 of by= (1) is type 'complex', not yet supported. Please use the by= argument to specify columns with types that are supported. See NEWS item in v1.12.2 for more information.

Coming from forderv so first this is a repeat of 1

DT[CJ(a, z, sorted = FALSE)]

Error: Type 'complex' not supported by CJ.

5.

DT[ , total := base::sum(z), by = .(a %% 2)]

DT[ , total := base::sum(z), by = .(a %% 2)]
Error in [.data.table(DT, , :=(total_rad, base::sum(z)), by = .(a%%2)) :
Internal error: writeNA passed a vector of type 'complex'

6.

DT[ , sum(z), by = .(a %% 2)]

Error in gsum(z) :
Type 'complex' not supported by GForce sum (gsum). Either add the prefix base::sum(.) or turn off GForce optimization using options(datatable.optimize=1)

7.

DT[DT, on = 'z']

Error in bmerge(i, x, leftcols, rightcols, roll, rollends, nomatch, mult, :
x.z is type complex which is not supported by data.table join

8.

DT[ , coalesce(`is.na<-`(z, 1), 0+0i)]

Error in coalesce(is.na<-(z, 1), 0 + (0+0i)) :
Unsupported type: complex

9.

DT[ , shift(z)]

Error in shift(z) : Unsupported type 'complex'

@MichaelChirico
Copy link
Member Author

@MichaelChirico MichaelChirico commented Jul 10, 2019

I think we could get away with hacking sorting immediately by morphing order(z) into order(Re(z), Im(z)); is there any immediacy to doing this, or is worth wait until going through and doing it properly in C on the CPLXSXP?

@MichaelChirico
Copy link
Member Author

@MichaelChirico MichaelChirico commented Jul 11, 2019

I don't really know where to start on fread... but Hugh's colTypes generalization already works:

class(fread('a\n0+1i', colClasses = c(a='complex'))[[1]])
# [1] "complex"

I'd say this is good enough unless there is some specific demand to speed this up. At a glance it would take significant effort to teach fread to detect & read complex values automatically.

@mattdowle
Copy link
Member

@mattdowle mattdowle commented Jul 19, 2019

Fantastic! Can this be closed now @MichaelChirico ?

@MichaelChirico
Copy link
Member Author

@MichaelChirico MichaelChirico commented Jul 20, 2019

@mattdowle
Copy link
Member

@mattdowle mattdowle commented Jul 20, 2019

Great. I edited the revised news item into the main comment at the top of this issue. It asks for a use case. Is that ok? I'd prefer not to create a new issue for join-on-complex until someone asks for it.

@MichaelChirico
Copy link
Member Author

@MichaelChirico MichaelChirico commented Jul 20, 2019

Yes that's perfect. Thanks for all your help getting the rest of this issue over the finish line!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants