Skip to content
This repository has been archived by the owner on May 5, 2019. It is now read-only.

Stop auto-promoting column-types #30

Closed
wants to merge 46 commits into from
Closed

Stop auto-promoting column-types #30

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
8db2821
add changes
cjprybol Mar 10, 2017
4a939fe
make vcat error more informative
cjprybol Mar 13, 2017
f5a53a1
add docstring for vcat
cjprybol Mar 13, 2017
2c95f13
incorporate edits suggested during review
cjprybol Mar 13, 2017
412ceaa
_unsafe_get -> NullableArrays.unsafe_get
cjprybol Mar 13, 2017
f142df5
Merge branch 'master' into cjp/rebaseretaintype
cjprybol Mar 13, 2017
cc95658
fix new tests from master
cjprybol Mar 13, 2017
06dc914
remove RepeatedVector, StackedVector, unstackdt, meltdt
cjprybol Mar 14, 2017
c4e218e
DataFrames doensn't reshape 2d Arrays -> Vectors so don't do it here
cjprybol Mar 14, 2017
e954226
minor cleanup
cjprybol Mar 14, 2017
ed8a515
change (de)nullify back to copy and cleanup docstrings
cjprybol Mar 15, 2017
1636a0c
NullableArrays.unsafe_get -> compat(unsafe_get)
cjprybol Mar 15, 2017
91233d3
default to NullableArray for joins that may introduce missing data
cjprybol Mar 15, 2017
7462612
align comments
cjprybol Mar 15, 2017
9b65533
lots of edits
cjprybol Mar 15, 2017
b643ff8
tests and no need for compat
cjprybol Mar 15, 2017
4c68452
spacing mistakes
cjprybol Mar 15, 2017
7310681
throw errors on 1-d matrices and change confusing variable name
cjprybol Mar 15, 2017
de280ba
add back check to differentiate scalars from AbstractArrays
cjprybol Mar 15, 2017
88b20ca
save work
cjprybol Mar 16, 2017
be1cacd
save progress, switch to test master
cjprybol Mar 16, 2017
19ffb58
join is ready and tests in place. right join still broken
cjprybol Mar 16, 2017
3f2cd63
fix right join
cjprybol Mar 17, 2017
9c3ad21
update join help message and add note about temp fix
cjprybol Mar 17, 2017
1e7d26e
indentation
cjprybol Mar 17, 2017
e39ba63
changes
cjprybol Mar 17, 2017
04cb9ee
spacing
cjprybol Mar 17, 2017
5d70685
put old unstack back and stabilize types, ordering
cjprybol Mar 18, 2017
7859132
fix bad copy and paste spacing and condense scalar recycling code
cjprybol Mar 18, 2017
6496acf
update vcat error
cjprybol Mar 18, 2017
f47810f
unused function, another test, remove unused variable
cjprybol Mar 18, 2017
259ceef
revert function removal to appease new code failures?
cjprybol Mar 18, 2017
26e87ac
fix v0.5 issue
cjprybol Mar 18, 2017
e0f7982
update vcat testing and change similar_nullable constructor call
cjprybol Mar 18, 2017
d65385e
:Merge branch 'cjp/retaintype' of github.com:cjprybol/DataTables.jl i…
cjprybol Mar 18, 2017
b0c29b4
remove old error message from docstring
cjprybol Mar 18, 2017
95a6f31
and change docstring to doctest
cjprybol Mar 18, 2017
7df712f
change similar_nullable back and fix unrelated copy paste space removal
cjprybol Mar 18, 2017
27da644
add missing rightperm reordering and properly unify hcat! functions
cjprybol Mar 18, 2017
5fa8fa0
accidental spacing changes
cjprybol Mar 18, 2017
a1d58f9
forgot one spacing change
cjprybol Mar 18, 2017
db87443
change deprecations
cjprybol Mar 18, 2017
9c66a1e
add back extra spaces
cjprybol Mar 18, 2017
887346b
bump catarrays version, remove manual resetting of levels in unstack
cjprybol Mar 20, 2017
00c08cc
Merge branch 'master' into cjp/retaintype
cjprybol Mar 24, 2017
020c88e
only use "and" when joining the last estring
cjprybol Mar 24, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions src/abstractdatatable/join.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,17 +3,17 @@
##

# Like similar, but returns a nullable array
similar_nullable{T}(dv::AbstractVector{T}, dims::Union{Int, Tuple{Vararg{Int}}}) =
NullableVector{T}(dims)
similar_nullable{T}(dv::AbstractArray{T}, dims::Union{Int, Tuple{Vararg{Int}}}) =
NullableArray(T, dims)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do NullableArrays not support the proper type parameter convention? I thought that was just DataArrays.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, they do. This is reverting back to the current master syntax. Now that I've added tests for CategoricalArrays in unstack (or maybe just ran a Pkg.update?) I caught a new deprecation warning.

julia> DataTables.similar_nullable(CategoricalArray([1, 1, 1]), 2)
WARNING: NullableCategoricalArray{T}(::Type{T}, dims::Int...; ordered=false) is deprecated, use NullableCategoricalArray{T}(dims, ordered=ordered) instead.
Stacktrace:
 [1] depwarn(::String, ::Symbol) at ./deprecated.jl:64
 [2] #NullableCategoricalArray#137(::Bool, ::Type{T} where T, ::Type{Int64}, ::Int64, ::Vararg{Int64,N} where N) at ./deprecated.jl:51
 [3] CategoricalArrays.NullableCategoricalArray(::Type{Int64}, ::Int64) at ./deprecated.jl:51
 [4] similar_nullable(::CategoricalArrays.CategoricalArray{Int64,1,UInt32}, ::Int64) at /Users/Cameron/.julia/v0.6/DataTables/src/abstractdatatable/join.jl:12
 [5] eval(::Module, ::Any) at ./boot.jl:235
 [6] eval_user_input(::Any, ::Base.REPL.REPLBackend) at ./REPL.jl:66
 [7] macro expansion at ./REPL.jl:97 [inlined]
 [8] (::Base.REPL.##1#2{Base.REPL.REPLBackend})() at ./event.jl:73
while loading no file, in expression starting on line 0
2-element CategoricalArrays.NullableCategoricalArray{Int64,1,UInt32}:
 #NULL
 #NULL

julia> DataTables.similar_nullable(NullableCategoricalArray([1, 1, 1]), 2)
WARNING: NullableCategoricalArray{T}(::Type{T}, dims::Int...; ordered=false) is deprecated, use NullableCategoricalArray{T}(dims, ordered=ordered) instead.
Stacktrace:
 [1] depwarn(::String, ::Symbol) at ./deprecated.jl:64
 [2] #NullableCategoricalArray#137(::Bool, ::Type{T} where T, ::Type{Int64}, ::Int64, ::Vararg{Int64,N} where N) at ./deprecated.jl:51
 [3] CategoricalArrays.NullableCategoricalArray(::Type{Int64}, ::Int64) at ./deprecated.jl:51
 [4] similar_nullable(::CategoricalArrays.NullableCategoricalArray{Int64,1,UInt32}, ::Int64) at /Users/Cameron/.julia/v0.6/DataTables/src/abstractdatatable/join.jl:15
 [5] eval(::Module, ::Any) at ./boot.jl:235
 [6] eval_user_input(::Any, ::Base.REPL.REPLBackend) at ./REPL.jl:66
 [7] macro expansion at ./REPL.jl:97 [inlined]
 [8] (::Base.REPL.##1#2{Base.REPL.REPLBackend})() at ./event.jl:73
while loading no file, in expression starting on line 0
2-element CategoricalArrays.NullableCategoricalArray{Int64,1,UInt32}:
 #NULL
 #NULL

I changed all of these to the new syntax but @nalimilan asked if I would revert and save the changes for another PR, too many changes at once.


similar_nullable{T<:Nullable}(dv::AbstractVector{T}, dims::Union{Int, Tuple{Vararg{Int}}}) =
NullableVector{eltype(T)}(dims)
similar_nullable{T<:Nullable}(dv::AbstractArray{T}, dims::Union{Int, Tuple{Vararg{Int}}}) =
NullableArray(eltype(T), dims)

similar_nullable{T,R}(dv::CategoricalVector{T,R}, dims::Union{Int, Tuple{Vararg{Int}}}) =
NullableCategoricalVector{T}(dims)
similar_nullable{T,R}(dv::CategoricalArray{T,R}, dims::Union{Int, Tuple{Vararg{Int}}}) =
NullableCategoricalArray(T, dims)

similar_nullable{T,R}(dv::NullableCategoricalVector{T,R}, dims::Union{Int, Tuple{Vararg{Int}}}) =
NullableCategoricalVector{T}(dims)
similar_nullable{T,R}(dv::NullableCategoricalArray{T,R}, dims::Union{Int, Tuple{Vararg{Int}}}) =
NullableCategoricalArray(T, dims)

# helper structure for DataTables joining
immutable DataTableJoiner{DT1<:AbstractDataTable, DT2<:AbstractDataTable}
Expand Down
3 changes: 3 additions & 0 deletions src/abstractdatatable/reshape.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -344,6 +344,7 @@ RepeatedVector(parent::AbstractVector, inner::Int, outer::Int)
* `inner` : the numer of times each element is repeated
* `outer` : the numer of times the whole vector is repeated after
expanded by `inner`

`inner` and `outer` have the same meaning as similarly named arguments
to `repeat`.

Expand Down Expand Up @@ -398,6 +399,7 @@ end

"""
A stacked view of a DataTable (long format)

Like `stack` and `melt`, but a view is returned rather than data
copies.

Expand Down Expand Up @@ -432,6 +434,7 @@ The result is a view because the columns are special AbstractVectors
that return indexed views into the original DataTable.

### Examples

```julia
d1 = DataTable(a = repeat([1:3;], inner = [4]),
b = repeat([1:4;], inner = [3]),
Expand Down