Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Collections item key doesn't start with 0 #112

Closed
manycoding opened this issue Jun 13, 2019 · 4 comments
Closed

Collections item key doesn't start with 0 #112

manycoding opened this issue Jun 13, 2019 · 4 comments
Assignees
Labels
Priority: High Type: Bug Something isn't working
Milestone

Comments

@manycoding
Copy link
Contributor

manycoding commented Jun 13, 2019

start=0 is set as default for Cloud Items puller, however collections have custom indexes, 0 not necessary the first.
Maybe the jobs too.

@manycoding manycoding added Type: Bug Something isn't working Priority: High labels Jun 13, 2019
@manycoding manycoding added this to the 0.3.6 milestone Jun 13, 2019
@manycoding manycoding self-assigned this Jun 13, 2019
@manycoding
Copy link
Contributor Author

This bug raises one question:
is start in Arche needs to be mirrored as our API or not?
Currently, for jobs:

  1. _key repeats index, e.g. 112358/13/21/10 is always 10th element in a job
  2. Thus, start specifies what number of elements to skip
    Collections:
  3. _key is arbitrary, e.g. 10 or 324vds53, it could be or could be not related to an index
  4. Thus, to correctly specify start one needs to know the _key of nth element

So, do we want to user to specify and find out start himself, like:

Arche(start="112358/13/21/100")
Arche(start="asfd32rfas")```
Or better to keep the current way of implying that start is nth element after start.
`Arche(start="100")`

@nerevaryn
Copy link

I think in the case of a normal ordering, we can use the current approach because it doesn't change anything. For the jobs/collections where we have not direct order of items/keys then a person should choose exactly the item number which to start from (not an amount of items to be skipped), because when a user puts something as a value to "start" it means he/she knows where exactly to start from, when to count amount of items before this item to start will be much more difficult task

@manycoding
Copy link
Contributor Author

The current approach doesn't support start for collections.
So I am asking it either 1 or 2 in this library.

@manycoding
Copy link
Contributor Author

So the idea is when somebody specifies start, he means item key and he knows it.
Thus, there should be separation between start and start_index.
On top level, everything will be the same, i.e. start will mean item key.

manycoding added a commit that referenced this issue Jun 17, 2019
* Rewrite Source and add StoreSource mocks

* Separate start_index and start, support for collections, closes #112
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Priority: High Type: Bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants