Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,9 @@ with the ability to add metadata and overload behavior at every level. (For inst

Additionally, [arrow-julia](https://github.com/apache/arrow-julia) provides Julia access to the Apache Arrow format, which is also good for in-memory interprocess communication, but the Awkward Array format is a superset of this format to make it easier to represent intermediate calculations.

## Documenttion
[User Guilde](https://juliahep.github.io/AwkwardArray.jl/dev/)

## Reading and writing the same data type

AwkwardArray.jl is a reimplementation of the concept of Awkward Arrays in Julia, taking advantage of Julia's capabilities. Python's Awkward Array has other backends for sending data to JIT-compiled languages—Numba (CPU and GPU) and C++ (with cppyy and ROOT's RDataFrame)—but as read-only views, owned exclusively by Python, for brief excursions only. Creating new Awkward Arrays in those JIT-compiled languages requires special tools, [ak.ArrayBuilder](https://awkward-array.org/doc/main/reference/generated/ak.ArrayBuilder.html) (discovers data type during iteration) and [LayoutBuilder](https://awkward-array.org/doc/main/user-guide/how-to-use-header-only-layoutbuilder.html) (fills a specified data type; faster).
Expand Down
Binary file added docs/assets/logo-300px.ico
Binary file not shown.
4 changes: 2 additions & 2 deletions docs/make.jl
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,12 @@ makedocs(;
modules=[AwkwardArray],
format = Documenter.HTML(
prettyurls = get(ENV, "CI", nothing) == "true",
assets=String[],
assets=String["img/logo-300px.ico"],
),
pages=[
"Introduction" => "index.md",
"Example Usage" => "exampleusage.md",
"APIs" => "api.md",
"Reference Guide" => "api.md",
"LICENSE" => "LICENSE.md",
],
repo="https://github.com/JuliaHEP/AwkwardArray.jl/blob/{commit}{path}#L{line}",
Expand Down
49 changes: 48 additions & 1 deletion docs/src/api.md
Original file line number Diff line number Diff line change
@@ -1 +1,48 @@
# API
## List of functions

Every `Content` subclass has the following built-in functions:

* `Base.length`
* `Base.size` (1-tuple of `length`)
* `Base.firstindex`, `Base.lastindex` (1-based or inherited from its index)
* `Base.getindex`: select by `Int` (single item), `UnitRange{Int}` (slice), and `Symbol` (record field)
* `Base.iterate`
* `Base.(==)` (equality defined by values: a `ListOffsetArray` and a `ListArray` may be considered the same)
* `Base.push!`
* `Base.append!`
* `Base.show`

They also have the following functions for manipulating and checking structure:

* `AwkwardArray.parameters_of`: gets all parameters
* `AwkwardArray.has_parameter`: returns true if a parameter exists
* `AwkwardArray.get_parameter`: returns a parameter or raises an error
* `AwkwardArray.with_parameter`: returns a copy of this node with a specified parameter
* `AwkwardArray.copy`: shallow-copy of the array, allowing properties to be replaced
* `AwkwardArray.is_valid`: verifies that the structure adheres to Awkward Array's protocol

They have the following functions for filling an array:

* `AwkwardArray.end_list!`: closes off a `ListType` array (`ListOffsetArray`, `ListArray`, or `RegularArray`) in the manner of Python's [ak.ArrayBuilder](https://awkward-array.org/doc/main/reference/generated/ak.ArrayBuilder.html) (no `begin_list` is necessary)
* `AwkwardArray.end_record!`: closes off a `RecordArray`
* `AwkwardArray.end_tuple!`: closes off a `TupleArray`
* `AwkwardArray.push_null!`: pushes a missing value onto `OptionType` arrays (`IndexedOptionArray`, `ByteMaskedArray`, `BitMaskedArray`, or `UnmaskedArray`)
* `AwkwardArray.push_dummy!`: pushes an unspecified value onto the array (used by `ByteMaskedArray` and `BitMaskedArray`, which need to have a placeholder in memory behind each `missing` value)

`RecordArray` and `TupleArray` have the following for selecting fields (as opposed to rows):

* `AwkwardArray.slot`: gets a `RecordArray` or `TupleArray` field, to avoid conflicts with `Base.getindex` for `TupleArrays` (both use integers to select a field)
* `AwkwardArray.Record`: scalar representation of an item from a `RecordArray`
* `AwkwardArray.Tuple`: scalar representation of an item from a `TupleArray` (note: not the same as `Base.Tuple`)

`UnionArray` has the following for dealing with specializations:

* `AwkwardArray.Specialization`: selects a `UnionArray` specialization for `push!`, `append!`, etc.

Finally, all `Content` subclasses can be converted with the following:

* `AwkwardArray.layout_for`: returns an appropriately-nested `Content` type for a given Julia type (`DataType`)
* `AwkwardArray.from_iter`: converts Julia data into an Awkward Array
* `AwkwardArray.to_vector`: converts an Awkward Array into Julia data
* `AwkwardArray.from_buffers`: constructs an Awkward Array from a Form (JSON), length, and buffers for zero-copy passing from Python
* `AwkwardArray.to_buffers`: deconstructs an Awkward Array into a Form (JSON), length, and buffers for zero-copy passing to Python
51 changes: 0 additions & 51 deletions docs/src/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,54 +135,3 @@ String
```

Most applications of `behavior` apply to `RecordArrays` (e.g. [Vector](https://github.com/scikit-hep/vector) in Python).

## List of functions

Every `Content` subclass has the following built-in functions:

* `Base.length`
* `Base.size` (1-tuple of `length`)
* `Base.firstindex`, `Base.lastindex` (1-based or inherited from its index)
* `Base.getindex`: select by `Int` (single item), `UnitRange{Int}` (slice), and `Symbol` (record field)
* `Base.iterate`
* `Base.(==)` (equality defined by values: a `ListOffsetArray` and a `ListArray` may be considered the same)
* `Base.push!`
* `Base.append!`
* `Base.show`

They also have the following functions for manipulating and checking structure:

* `AwkwardArray.parameters_of`: gets all parameters
* `AwkwardArray.has_parameter`: returns true if a parameter exists
* `AwkwardArray.get_parameter`: returns a parameter or raises an error
* `AwkwardArray.with_parameter`: returns a copy of this node with a specified parameter
* `AwkwardArray.copy`: shallow-copy of the array, allowing properties to be replaced
* `AwkwardArray.is_valid`: verifies that the structure adheres to Awkward Array's protocol

They have the following functions for filling an array:

* `AwkwardArray.end_list!`: closes off a `ListType` array (`ListOffsetArray`, `ListArray`, or `RegularArray`) in the manner of Python's [ak.ArrayBuilder](https://awkward-array.org/doc/main/reference/generated/ak.ArrayBuilder.html) (no `begin_list` is necessary)
* `AwkwardArray.end_record!`: closes off a `RecordArray`
* `AwkwardArray.end_tuple!`: closes off a `TupleArray`
* `AwkwardArray.push_null!`: pushes a missing value onto `OptionType` arrays (`IndexedOptionArray`, `ByteMaskedArray`, `BitMaskedArray`, or `UnmaskedArray`)
* `AwkwardArray.push_dummy!`: pushes an unspecified value onto the array (used by `ByteMaskedArray` and `BitMaskedArray`, which need to have a placeholder in memory behind each `missing` value)

`RecordArray` and `TupleArray` have the following for selecting fields (as opposed to rows):

* `AwkwardArray.slot`: gets a `RecordArray` or `TupleArray` field, to avoid conflicts with `Base.getindex` for `TupleArrays` (both use integers to select a field)
* `AwkwardArray.Record`: scalar representation of an item from a `RecordArray`
* `AwkwardArray.Tuple`: scalar representation of an item from a `TupleArray` (note: not the same as `Base.Tuple`)

`UnionArray` has the following for dealing with specializations:

* `AwkwardArray.Specialization`: selects a `UnionArray` specialization for `push!`, `append!`, etc.

Finally, all `Content` subclasses can be converted with the following:

* `AwkwardArray.layout_for`: returns an appropriately-nested `Content` type for a given Julia type (`DataType`)
* `AwkwardArray.from_iter`: converts Julia data into an Awkward Array
* `AwkwardArray.to_vector`: converts an Awkward Array into Julia data
* `AwkwardArray.from_buffers`: constructs an Awkward Array from a Form (JSON), length, and buffers for zero-copy passing from Python
* `AwkwardArray.to_buffers`: deconstructs an Awkward Array into a Form (JSON), length, and buffers for zero-copy passing to Python

_(This will turn into proper documentation, eventually.)_
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jpivarski - the idea is to trim the top README.md file and have full documentation here that is generated as a documentation by Documeter.jl. The full infrastructure is already in place. The dev version of documentation is at https://juliahep.github.io/AwkwardArray.jl/dev/

17 changes: 0 additions & 17 deletions ext/AwkwardPythonCallExt/README.md

This file was deleted.