Merge pull request #119 from visr/housekeeping

Housekeeping

.travis.yml

@@ -4,14 +4,9 @@ os:
   - linux
   - osx
 julia:
-  - 0.7
   - 1.0
+  - 1
   - nightly
 notifications:
   email: false
-# uncomment the following lines to override the default test script
-#script:
-#  - if [[ -a .git/shallow ]]; then git fetch --unshallow; fi
-#  - julia -e 'Pkg.clone(pwd()); Pkg.checkout("MemPool"); Pkg.build("Dagger"); Pkg.test("Dagger"; coverage=true)'
-after_success:
-  - julia -e 'cd(Pkg.dir("Dagger")); Pkg.add("Coverage"); using Coverage; Coveralls.submit(Coveralls.process_folder())'
+coveralls: true

Project.toml

@@ -15,6 +15,8 @@ Statistics = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"
 StatsBase = "2913bbd2-ae8a-5f71-8c99-4fb6c76f3a91"
 
 [compat]
+MemPool = "0.2"
+StatsBase = "0.28, 0.29, 0.30, 0.31, 0.32"
 julia = "1.0"
 
 [extras]

README.md

@@ -4,7 +4,7 @@
 
 [![Build Status](https://travis-ci.org/JuliaParallel/Dagger.jl.svg?branch=master)](https://travis-ci.org/JuliaParallel/Dagger.jl) [![Coverage Status](https://coveralls.io/repos/github/JuliaParallel/Dagger.jl/badge.svg?branch=master)](https://coveralls.io/github/JuliaParallel/Dagger.jl?branch=master)
 
-At the core of Dagger.jl is a scheduler heavily inspired by [Dask](http://dask.pydata.org/en/latest/). It can run computations represented as [directed-acyclic-graphs](https://en.wikipedia.org/wiki/Directed_acyclic_graph) (DAGs) efficiently on many Julia worker processes.
+At the core of Dagger.jl is a scheduler heavily inspired by [Dask](https://docs.dask.org/en/latest/). It can run computations represented as [directed-acyclic-graphs](https://en.wikipedia.org/wiki/Directed_acyclic_graph) (DAGs) efficiently on many Julia worker processes.
 
 
 ## Installation
@@ -23,11 +23,11 @@ Once installed, the `Dagger` package can by used by typing
 using Dagger
 ```
 
-The main function for using Dagger is *delayed*
+The main function for using Dagger is `delayed`
 
 `delayed(f; options...)`
 
-Returns a function which when called creates a `Thunk` object representing a call to function `f` with the given arguments. If it is called with other thunks as input, then they form a graph with input nodes directed at the output. The function `f` get the result of the input Thunks.
+It returns a function which when called creates a `Thunk` object representing a call to function `f` with the given arguments. If it is called with other thunks as input, then they form a graph with input nodes directed at the output. The function `f` gets the result of the input Thunks.
 Thunks don't pass keyword argument to the function `f`. Options kwargs... to `delayed` are passed to the scheduler to control its behavior:
 - `get_result::Bool` -- return the actual result to the scheduler instead of `Chunk` objects. Used when `f` explicitly constructs a Chunk or when return value is small (e.g. in case of reduce)
 - `meta::Bool` -- pass the input “Chunk” objects themselves to `f` and not the value contained in them - this is always run on the master process
@@ -40,8 +40,8 @@ Here is a very simple example DAG:
 ```julia
 using Dagger
 
-add1(value) = value+1
-add2(value) = value+2
+add1(value) = value + 1
+add2(value) = value + 2
 combine(a...) = sum(a)
 
 p = delayed(add1)(4)
@@ -80,8 +80,8 @@ compute(top_node)
 
 - First picks the leaf Thunks and distributes them to available workers. Each worker is given at most 1 task at a time. If input to the node is a `Chunk`, then workers which already have the chunk are preferred.
 - When a worker finishes a thunk it will return a `Chunk` object to the scheduler.
-- Once the worker has returned a `Chunk`, scheduler picks the next task for the worker -- this is usually the task the worker immediately made available (if possible). In the small example above, if worker 2 finished `p` it will be given `q` since it will already have the result of `p` which is input to `q`.
-- The scheduler also issues "release" Commands to chunks that are no longer required by nodes in the DAG: for example, when s is computed all of p, q, r are released to free up memory. This can be prevented by passing `persist` or `cache` options to `delayed`.
+- Once the worker has returned a `Chunk`, the scheduler picks the next task for the worker -- this is usually the task the worker immediately made available (if possible). In the small example above, if worker 2 finished `p` it will be given `q` since it will already have the result of `p` which is input to `q`.
+- The scheduler also issues "release" Commands to chunks that are no longer required by nodes in the DAG: for example, when `s` is computed all of `p`, `q`, `r` are released to free up memory. This can be prevented by passing `persist` or `cache` options to `delayed`.
 
 ### Acknowledgements
 

appveyor.yml

@@ -1,8 +1,8 @@
 environment:
   matrix:
-  - julia_version: 0.7
   - julia_version: 1.0
-  - julia_version: latest
+  - julia_version: 1
+  - julia_version: nightly
 
 platform:
   - x86 # 32-bit
@@ -27,4 +27,5 @@ build_script:
   - C:\julia\bin\julia -e "%JL_BUILD_SCRIPT%"
 
 test_script:
-  - C:\julia\bin\julia --check-bounds=yes -e "using Pkg; Pkg.test(\"Dagger\")"
+  - echo "%JL_TEST_SCRIPT%"
+  - C:\julia\bin\julia -e "%JL_TEST_SCRIPT%"

src/lib/block-io.jl

@@ -13,6 +13,7 @@ struct BlockIO <: IO
         seekend(bio)
         try
             while(!eof(bio.s) && (end_byte != read(bio, UInt8))) continue end
+        finally
         end
         position(bio.s)
     end