Facilitate UAST-based tasks on PGA

[smola]

Parent objective: #11
Progress: 0%

• [P0] Create UAST dataset based on PGA consumable by ML team
• [P0] Define a format that can store UAST in a space-efficient way
• [P1] Publish UAST dataset
• [P2] Publish an index for the UAST dataset that DevRel can use to improve download criteria for it

[dennwc]

@smola I think we should add an objective for UAST compression. I have few options in mind to cut the size of the dataset.

[eiso]

@dennwc I am about to invite the @src-d/language-analysis team to the slack channel to start creating their OKR's. This would be a great example of something you can edit/add. OKR's are designed to also be bottom-up, not just top-down in the company.

[bzz]

Just something that we already discussed internal and externally and might be nice to keep in mind - people are excited about ability to query UAST dataset on Google's BigQuery.

[campoy]

@smola, could you explain what you mean by "an index for the UAST dataset"?

Also, I'm wondering whether we could make it so people need to somehow register to download PGA
Do we keep track of the number of downloads already?

[marnovo]

[OKRs Review] #25 removed from parent because too detailed.

[marnovo]

[OKRs Review] Problem/need is not well defined yet to prompt the work for this as an objective. Discarding.

[smola]

@campoy An index you can use to build a selective download tool for different chunks of the dataset. Similar to what we have for PGA.

[campoy]

So basically you can download the UASTs rather than the siva files, @smola?
Cool!

[smola]

@campoy Right. That would be the idea. Note, however, that this is discarded as an engineering objective for this quarter. We'll discuss the viability of this later on though.

[eiso]

@marnovo instead of using discarded, I am closing the issue instead.