Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Custom parsers and client functionality #33

Merged
merged 5 commits into from
Jan 12, 2021

Conversation

Arkoniak
Copy link

Contributing guides recommend PR against dev, I am not sure though... Can reopen it vs master.

Despite the size, this PR consists of a relatively small number of features. I am pushing them all since I use them in practice and it would be inconvenient for me to not use them.

Features:

  1. AlphaVantage client. All functions now accept client as an argument and also there is a global client that is used by default. This approach lets the user define credentials not only through the ENV variable, but also read them from the file or by any other means, which can be convenient. Also, the usage of entry field makes it possible to use Alpha Vantage from behind the proxy.
  2. Custom parser. All functions now accept parser as an argument. It can be either AbstractString in this case parser is chosen from the predefined set, or it can be a function or any function like object. This makes possible to use construction like parser = x -> JSON3.read(x.body) and json datatype is parsed by JSON3 instead of JSON. Or, user can apply any parser to process the resulting data, so data can be transformed to DataFrame, StructArray, or any other structure, that the user wants. It is possible to collect some variety of these parsers and add them to the predefined set, thus simplifying function calls.
  3. All tests are wrapped in modules, it helps to isolate all intermediate structures and imports that are necessary for tests.

@ellisvalentiner ellisvalentiner changed the base branch from dev to master January 11, 2021 20:49
@ellisvalentiner
Copy link
Owner

It looks like the dev branch hasn't been updated in over 2 years, so we should probably remove that guidance from the contributing guide.

@Arkoniak I changed the base to master

Copy link
Collaborator

@dm13450 dm13450 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great and a real value add. Thanks for your time on this

src/avclient.jl Outdated Show resolved Hide resolved
@Arkoniak
Copy link
Author

As an additional example, how custom parsers can be used, here is user-sideTimeSeries transformation of the data

using AlphaVantage
using CSV
using TimeSeries
using Tables

taparser(x) = TimeArray(Tables.columntable(CSV.File(x.body)), timestamp = :timestamp)

julia> spy = time_series_daily("SPY", datatype="csv", parser = taparser)
100×5 TimeArray{Float64, 2, Date, Matrix{Float64}} 2020-08-19 to 2021-01-11
│            │ open   │ high     │ low      │ close  │ volume       │
├────────────┼────────┼──────────┼──────────┼────────┼──────────────┤
│ 2020-08-19339.05339.61336.62337.236.8054244e7  │
│ 2020-08-20335.36338.8335.22338.284.2207826e7  │
│ 2020-08-21337.92339.72337.55339.485.5106628e7  │
│ 2020-08-24342.12343.0339.4504342.924.8588662e7  │
│ 2020-08-25343.53344.21342.27344.123.8463381e7

@dm13450 dm13450 merged commit 19c725d into ellisvalentiner:master Jan 12, 2021
@dm13450
Copy link
Collaborator

dm13450 commented Jan 12, 2021

Merged and tested locally, everything passes, so thanks once again!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants