Changed spark-api project name to source{d} engine in documentation #110

mcarmonaa · 2017-10-16T10:47:23Z

Closes #66

As first step to renaming, only those references to spark-api in documentation that don't brake links or commands have been changed to source{d} engine
Related to Change project name #66

codecov · 2017-10-16T10:54:31Z

Codecov Report

Merging #110 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #110   +/-   ##
=======================================
  Coverage   90.79%   90.79%           
=======================================
  Files          22       22           
  Lines         619      619           
  Branches       53       53           
=======================================
  Hits          562      562           
  Misses         57       57

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dd8de97...b19f721. Read the comment docs.

erizocosmico · 2017-10-16T11:54:36Z

README.md


 ```bash
 $ spark-shell --packages com.github.src-d:spark-api:master-SNAPSHOT --repositories https://jitpack.io
 ```

-To start using spark-api from the shell you must import everything inside the `tech.sourced.api` package (or, if you prefer, just import `SparkAPI` and `ApiDataFrame` classes):
+To start using source{d} engine from the shell you must import everything inside the `tech.sourced.api` package (or, if you prefer, just import `SparkAPI` and `ApiDataFrame` classes):


we'll have to change this as well when we rename it in the code

mcarmonaa · 2017-10-17T06:58:38Z

Changes in links and code examples have been changed based on rename SparkAPI to Engine #111
For links and jitpack coordinates I supposed the project namespace will be github.com/src-d/engine instead of github.com/src-d/spark-api

mcarmonaa · 2017-10-17T09:13:48Z

Jupyter notebook has been updated to include an UAST extraction example and use the renaming module sourced.engine

mcarmonaa · 2017-10-18T10:45:56Z

MORE CHANGES

Updated notebook and examples to be aligned with the last api changes
Added examples of the new XPath query feature

bzz · 2017-10-18T13:11:37Z

_examples/README.md

@@ -14,6 +14,8 @@ Here you can find a list of annotated *spark-api* examples:

 - [pyspark's shell classifying languages and extracting UASTs](pyspark/pyspark-shell-lang-and-uast.md)

+-[pyspark's shell querying UASTs with XPath](pyspark/pyspark-shell-xpath-query.md)


missing whitespace between - and [ changes the MD formatting

bzz · 2017-10-18T13:15:30Z

_examples/pyspark/pyspark-shell-xpath-query.md

+
+Finally, `extract_tokens()` method will generate a column `tokens` based on the previous generated column `result`.
+
+```python


May be it's better to remove info string python here and in all similar cases, as the code below is not just a Python and gets highlighted very random

What do you think?

Sure, I'll show the code examples just as a plain text

bzz · 2017-10-18T13:42:39Z

👍
@mcarmonaa 9033f80 fixes line number 31 for spark but line 17 for pyspark is still there

mcarmonaa · 2017-10-18T14:40:16Z

@bzz solved

bzz · 2017-10-18T15:30:50Z

_examples/scala/spark-shell-xpath-query.md

+scala> val engine = Engine(spark, "/path/to/siva-files")
+engine: tech.sourced.engine.Engine = tech.sourced.engine.Engine@7e18b9e6
+
+scala> api.getRepositories.getHEAD.getFiles.classifyLanguages.where('lang === "Python").extractUASTs.queryUAST("//*[@roleIdentifier]", "uast", "result").extractTokens("result", "tokens").select('path, 'lang, 'uast, 'tokens).show


api -> engine

bzz · 2017-10-18T16:19:32Z

Great job @mcarmonaa !

One thing I have noticed, while going though all https://github.com/mcarmonaa/spark-api/blob/fix/doc-project-name/_examples/scala/ - is that it's very annoying not to be able to copy the full commands from the docs.

I.e if https://github.com/mcarmonaa/spark-api/blob/fix/doc-project-name/_examples/scala/spark-shell-xpath-query.md#querying-uasts-with-xpath-example we have

$ spark-shell --packages com.github.src-d:engine:master-SNAPSHOT --repositories https://jitpack.io
scala> import tech.sourced.engine._
import tech.sourced.engine._

scala> val engine = Engine(spark, "/path/to/siva-files")
engine: tech.sourced.engine.Engine = tech.sourced.engine.Engine@7e18b9e6

engine.getRepositories.getHEAD.getFiles.classifyLanguages.where('lang === "Python").extractUASTs.queryUAST("//*[@roleIdentifier]", "uast", "result").extractTokens("result", "tokens").select('path, 'lang, 'uast, 'tokens).show
+--------------------+------+-------------+--------------------+
|                path|  lang|         uast|              tokens|
+--------------------+------+-------------+--------------------+
...

but if that would have been

$ spark-shell --packages com.github.src-d:engine:master-SNAPSHOT --repositories https://jitpack.io

import tech.sourced.engine._
val engine = Engine(spark, "/path/to/siva-files")
engine.getRepositories.getHEAD.getFiles.classifyLanguages.where('lang === "Python").extractUASTs.queryUAST("//*[@roleIdentifier]", "uast", "result").extractTokens("result", "tokens").select('path, 'lang, 'uast, 'tokens).show

it would have next advantages:

look better due to syntax hightligh
be copy-able for a user

What do you guys think?

bzz · 2017-10-18T17:29:46Z

README.md


 You can launch our docker container which contains some Notebooks examples just running:

-    docker run --name spark-api-jupyter --rm -it -p 8888:8888 -v $(pwd)/path/to/siva-files:/repositories --link bblfsh:bblfsh srcd/spark-api-jupyter
+    docker run --name engine-jupyter --rm -it -p 8888:8888 -v $(pwd)/path/to/siva-files:/repositories --link bblfsh:bblfsh srcd/engine-jupyter


This needs to be updated: bblfsh is now bblfshd since line 71

erizocosmico · 2017-10-19T07:13:08Z

#111 has been merged, we can merge this as soon as it's ready

mcarmonaa · 2017-10-19T07:13:20Z

LAST CHANGES

Fixed some wrong words.
The examples have been changed to follow @bzz suggestions.

erizocosmico · 2017-10-19T07:23:52Z

Python build fails for latest changes. Also, can you rebase? There were massive changes with the renaming in the code

…enaming engine module

erizocosmico

lgtm, good job!

bzz · 2017-10-19T09:31:08Z

_examples/README.md

@@ -26,6 +28,8 @@ Here you can find a list of annotated *spark-api* examples:

 - [spark-shell classifying languages and extracting UASTs](scala/spark-shell-lang-and-uast.md)

+- [spark-shell querying UASTs with XPath](scala/spark-shell-xpath-query.md)
+
 ### jupyter notebooks

 - [Basic example](notebooks/Basic%2BExample.ipynb)


When rendered, this link gets broken i.e here https://github.com/mcarmonaa/spark-api/tree/fix/doc-project-name/_examples

Is that expected to work only on docSrv or shall it point elsewhere?

bzz · 2017-10-19T09:34:13Z

Looks great, thank you @mcarmonaa !

LGTM sans minor issue from above

erizocosmico approved these changes Oct 16, 2017

View reviewed changes

erizocosmico mentioned this pull request Oct 16, 2017

rename SparkAPI to Engine #111

Merged

erizocosmico approved these changes Oct 17, 2017

View reviewed changes

bzz approved these changes Oct 18, 2017

View reviewed changes

bzz reviewed Oct 18, 2017

View reviewed changes

bzz suggested changes Oct 18, 2017

View reviewed changes

erizocosmico mentioned this pull request Oct 18, 2017

Change Spark API in README to source{d} engine #119

Closed

bzz reviewed Oct 18, 2017

View reviewed changes

erizocosmico mentioned this pull request Oct 19, 2017

Change project name #66

Closed

mcarmonaa added 10 commits October 19, 2017 09:25

changed references from spark-api to sourced engine in the READMEs

aa72828

renaming applied to links and examples

ecfc4de

changed notebook to add uast extraction example and make use of the r…

fffa14d

…enaming engine module

renamed notebook example

ba412ba

updated examples

b7cef9f

added xpath query examples

8f31660

fix missing whitespace in examples' README

b196442

changed examples to show code snippets as plain text

7d337e8

fix scala xpath query example

e24fd37

fix docker command to use bblfshd instead bblfsh in README

3f8fdfd

changed code snippet examples style

b19f721

erizocosmico approved these changes Oct 19, 2017

View reviewed changes

bzz suggested changes Oct 19, 2017

View reviewed changes

bzz approved these changes Oct 19, 2017

View reviewed changes

erizocosmico merged commit 0db9867 into src-d:master Oct 19, 2017

mcarmonaa deleted the fix/doc-project-name branch October 19, 2017 09:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Changed spark-api project name to source{d} engine in documentation #110

Changed spark-api project name to source{d} engine in documentation #110

mcarmonaa commented Oct 16, 2017 •

edited by erizocosmico

codecov bot commented Oct 16, 2017 •

edited

erizocosmico Oct 16, 2017

mcarmonaa commented Oct 17, 2017

mcarmonaa commented Oct 17, 2017

mcarmonaa commented Oct 18, 2017

bzz Oct 18, 2017

bzz Oct 18, 2017 •

edited

mcarmonaa Oct 18, 2017

bzz commented Oct 18, 2017 •

edited

mcarmonaa commented Oct 18, 2017

bzz Oct 18, 2017

bzz commented Oct 18, 2017

bzz Oct 18, 2017

erizocosmico commented Oct 19, 2017

mcarmonaa commented Oct 19, 2017

erizocosmico commented Oct 19, 2017

erizocosmico left a comment

bzz Oct 19, 2017 •

edited

bzz commented Oct 19, 2017

		@@ -14,6 +14,8 @@ Here you can find a list of annotated spark-api examples:

		- [pyspark's shell classifying languages and extracting UASTs](pyspark/pyspark-shell-lang-and-uast.md)

		-[pyspark's shell querying UASTs with XPath](pyspark/pyspark-shell-xpath-query.md)


		Finally, `extract_tokens()` method will generate a column `tokens` based on the previous generated column `result`.

		```python

Changed spark-api project name to source{d} engine in documentation #110

Changed spark-api project name to source{d} engine in documentation #110

Conversation

mcarmonaa commented Oct 16, 2017 • edited by erizocosmico

codecov bot commented Oct 16, 2017 • edited

Codecov Report

erizocosmico Oct 16, 2017

Choose a reason for hiding this comment

mcarmonaa commented Oct 17, 2017

mcarmonaa commented Oct 17, 2017

mcarmonaa commented Oct 18, 2017

bzz Oct 18, 2017

Choose a reason for hiding this comment

bzz Oct 18, 2017 • edited

Choose a reason for hiding this comment

mcarmonaa Oct 18, 2017

Choose a reason for hiding this comment

bzz commented Oct 18, 2017 • edited

mcarmonaa commented Oct 18, 2017

bzz Oct 18, 2017

Choose a reason for hiding this comment

bzz commented Oct 18, 2017

bzz Oct 18, 2017

Choose a reason for hiding this comment

erizocosmico commented Oct 19, 2017

mcarmonaa commented Oct 19, 2017

erizocosmico commented Oct 19, 2017

erizocosmico left a comment

Choose a reason for hiding this comment

bzz Oct 19, 2017 • edited

Choose a reason for hiding this comment

bzz commented Oct 19, 2017

mcarmonaa commented Oct 16, 2017 •

edited by erizocosmico

codecov bot commented Oct 16, 2017 •

edited

bzz Oct 18, 2017 •

edited

bzz commented Oct 18, 2017 •

edited

bzz Oct 19, 2017 •

edited