excise datasets

FluxML · Nov 11, 2020 · b78cd76 · b78cd76
1 parent e8fe3e6
commit b78cd76
Show file tree

Hide file tree

Showing 14 changed files with 22 additions and 704 deletions.
diff --git a/NEWS.md b/NEWS.md
@@ -1,11 +1,17 @@
-# v0.11.3
-* Added option to set `bias` to [false](https://github.com/FluxML/Flux.jl/pull/1379) to eliminating `bias` from being trained. 
+# Flux Release Notes
 
-# v0.11.2
+## v0.12.0
+
+* Excise datasets in favour of other providers in the julia ecosystem.
+* other new features and bug fixes (see GitHub's releases page)
+* Added option to set `bias` to [false](https://github.com/FluxML/Flux.jl/pull/1379) to eliminating `bias` from being trained.
+
+## v0.11.2
 
 * Adds the [AdaBelief](https://arxiv.org/abs/2010.07468) optimiser.
+* other new features and bug fixes (see GitHub releases page)
 
-# v0.11
+## v0.11
 
 * Moved CUDA compatibility to use [CUDA.jl instead of CuArrays.jl](https://github.com/FluxML/Flux.jl/pull/1204)
 * Add [kaiming initialization](https://arxiv.org/abs/1502.01852) methods: [kaiming_uniform and kaiming_normal](https://github.com/FluxML/Flux.jl/pull/1243)
@@ -29,11 +35,11 @@
 keyword argument. The `Dropout` struct *whose behavior is left unchanged) is the recommended choice for common usage.
 * and many more fixes and additions...
 
-# v0.10.1 - v0.10.4
+## v0.10.1 - v0.10.4
 
 See GitHub's releases.
 
-# v0.10.0
+## v0.10.0
 
 * The default AD engine has switched from [Tracker to Zygote.jl](https://github.com/FluxML/Flux.jl/pull/669)
   - The dependency on Tracker.jl has been removed.
@@ -47,13 +53,13 @@ See GitHub's releases.
 * `@treelike` has been formalised as a [functor](https://github.com/FluxML/Flux.jl/pull/865), with an effective deprecation.
 * `testmode!` is deprecated in favour of [istraining](https://github.com/FluxML/Flux.jl/pull/669)
 
-# v0.9.0
+## v0.9.0
 
 * [Depthwise convolutional layer API changes](https://github.com/FluxML/Flux.jl/pull/756) from `in => mult` channel specification to `in => out` channel specification, and deprecates implicit `out` constructor.
 * New [SkipConnection](https://github.com/FluxML/Flux.jl/pull/446), which can be used to train residual neural network architectures.
 * New [RADAM](https://github.com/FluxML/Flux.jl/pull/842) optimiser.
 
-# v0.8.0
+## v0.8.0
 
 * [Dropout now has a `dims` argument for specifying the unbroadcast dimensions.](https://github.com/FluxML/Flux.jl/pull/563)
 * New [ConvTranspose layer](https://github.com/FluxML/Flux.jl/pull/311).
@@ -77,6 +83,6 @@ AD Changes:
 * Support for [PermuteDimsArray](https://github.com/FluxML/Flux.jl/pull/576).
 * Flux.Tracker is now its [own package](https://github.com/FluxML/Tracker.jl), in preparation for replacing it with Zygote.
 
-# v0.7.0
+## v0.7.0
 
 Despite the heroic efforts of scholars and archeologists, pre-0.7 history is lost to the sands of time.
diff --git a/docs/src/data/dataloader.md b/docs/src/data/dataloader.md
@@ -1,5 +1,6 @@
 # DataLoader
-Flux provides the `DataLoader` type in the `Flux.Data` module to handle iteration over mini-batches of data. 
+
+Flux provides the `DataLoader` type in the `Flux.Data` module to handle iteration over mini-batches of data.
 
 ```@docs
 Flux.Data.DataLoader

diff --git a/docs/src/datasets.md b/docs/src/datasets.md
@@ -1,20 +1,7 @@
 # Datasets
 
-Flux includes several standard machine learning datasets.
+- [MLDatasets.jl](https://github.com/JuliaML/MLDatasets.jl): utility package for accessing common machine learning datasets
+
+
 
-```@docs
-Flux.Data.Iris.features()
-Flux.Data.Iris.labels()
-Flux.Data.MNIST.images()
-Flux.Data.MNIST.labels()
-Flux.Data.FashionMNIST.images()
-Flux.Data.FashionMNIST.labels()
-Flux.Data.CMUDict.phones()
-Flux.Data.CMUDict.symbols()
-Flux.Data.CMUDict.rawdict()
-Flux.Data.CMUDict.cmudict()
-Flux.Data.Sentiment.train()
-Flux.Data.Sentiment.test()
-Flux.Data.Sentiment.dev()
-```
 
diff --git a/docs/src/saving.md b/docs/src/saving.md
@@ -45,7 +45,7 @@ versions of Flux).
 
 In some cases it may be useful to save only the model parameters themselves, and
 rebuild the model architecture in your code. You can use `params(model)` to get
-model parameters. You can also use `data.(params)` to remove tracking.
+model parameters.
 
 ```Julia
 julia> using Flux

diff --git a/docs/src/training/training.md b/docs/src/training/training.md
@@ -49,7 +49,7 @@ Handling all the parameters on a layer by layer basis is explained in the [Layer
 
 ## Datasets
 
-The `data` argument provides a collection of data to train with (usually a set of inputs `x` and target outputs `y`). For example, here's a dummy data set with only one data point:
+The `data` argument of `train!` provides a collection of data to train with (usually a set of inputs `x` and target outputs `y`). For example, here's a dummy dataset with only one data point:
 
 ```julia
 x = rand(784)

diff --git a/src/data/Data.jl b/src/data/Data.jl
@@ -1,56 +1,9 @@
 module Data
 
-import ..Flux
-import SHA
-
 using Random: shuffle!
 using Base: @propagate_inbounds
 
-export CMUDict, cmudict
-
-deps(path...) = joinpath(@__DIR__, "..", "..", "deps", path...)
-
-function download_and_verify(url, path, hash)
-    tmppath = tempname()
-    download(url, tmppath)
-    hash_download = open(tmppath) do f
-        bytes2hex(SHA.sha256(f))
-    end
-    if hash_download !== hash
-        msg  = "Hash Mismatch!\n"
-        msg *= "  Expected sha256:   $hash\n"
-        msg *= "  Calculated sha256: $hash_download"
-        error(msg)
-    end
-    mv(tmppath, path; force=true)
-end
-
-function __init__()
-  mkpath(deps())
-end
-
 include("dataloader.jl")
 export DataLoader
 
-include("mnist.jl")
-export MNIST
-
-include("fashion-mnist.jl")
-export FashionMNIST
-
-include("cmudict.jl")
-using .CMUDict
-
-include("tree.jl")
-include("sentiment.jl")
-using .Sentiment
-
-include("iris.jl")
-export Iris
-
-include("housing.jl")
-export Housing
-
-@deprecate DataLoader(x...; kws...) DataLoader(x; kws...)
-
 end
diff --git a/src/data/cmudict.jl b/src/data/cmudict.jl
diff --git a/src/data/fashion-mnist.jl b/src/data/fashion-mnist.jl