TINKERPOP-1447 Add some JavaScript intelligence to the documentation so that comments and output are not copied in a copy paste #766

dkuppitz · 2017-12-22T00:25:49Z

https://issues.apache.org/jira/browse/TINKERPOP-1447

Added automatic tab generation for all evaluated code block in all docs. A random screenshot from the generated reference docs (for those who are too lazy to build the docs):

VOTE: +1

jorgebay · 2017-12-22T11:16:23Z

docs/preprocessor/preprocess-file.sh

@@ -139,7 +140,7 @@ if [ ! ${SKIP} ] && [ $(grep -c '^\[gremlin' ${input}) -gt 0 ]; then
  ${lb} awk -f ${AWK_SCRIPTS}/language-variants.awk > ${output}

  ps=(${PIPESTATUS[@]})
-  for i in {0..6}; do
+  for i in {0..7}; do


I'm not able to follow why this is a loop from 0 to 7... can we include a comment or move 7 to a constant with a name?

Ha, it should actually be 0 to 9, as it checks to exit code for each program in the pipe chain. Not sure when that went out of sync, but I'll fix it.

robertdale · 2017-12-27T12:19:50Z

What about for those of us unable to build it?

sh docker/build.sh -d

 * source:   /usr/src/tinkermem/docs/src/recipes/olap-spark-yarn.asciidoc
   target:   /usr/src/tinkermem/target/postprocess-asciidoc/recipes/olap-spark-yarn.asciidoc
   progress: [=======================================================>                                            ] 55%java.io.FileNotFoundException: File file:/tmp/spark-gremlin.zip does not exist
Type ':help' or ':h' for help.
Display stack trace? [yN]pb(104); '----'


Last 10 lines of /usr/src/tinkermem/target/postprocess-asciidoc/recipes/olap-spark-yarn.asciidoc:

	at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2281)
	at org.apache.spark.SparkContext.getOrCreate(SparkContext.scala)
	at org.apache.tinkerpop.gremlin.spark.structure.Spark.create(Spark.java:52)
	at org.apache.tinkerpop.gremlin.spark.structure.Spark.create(Spark.java:60)
	at org.apache.tinkerpop.gremlin.spark.process.computer.SparkGraphComputer.lambda$submitWithExecutor$0(SparkGraphComputer.java:193)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
gremlin> 

xargs: /usr/src/tinkermem/docs/preprocessor/preprocess-file.sh: exited with status 255; aborting

dkuppitz · 2017-12-27T18:22:05Z

Hmm, this was another ticket and should be solved in this branch as I rebased it after the latest release.

robertdale · 2017-12-27T18:27:26Z

@dkuppitz I did see that. There was a dep added to the image. I have that rebuilding now. Will let you know the results.

robertdale · 2017-12-27T20:47:11Z

Just what I was afraid of... the tabs look just like they do in the screenshots. And I don't mean that in a good way 😄

Seriously, this is fantastic. Multi-line works too.

My issues:

I prefer the nice, thin border that was previously used
I'd like to see lighter-weight, less glaring tabs - See also https://thrift.apache.org/ (Example)
actually reading some of the finer details, the WARNING under Mutating the Graph states that 'JAVA' or 'GROOVY' should appear in the code blocks. This doesn't work for me in this PR or 3.2.7 docs. Looks like something introduced in 3.2.6 causes the screen to be super extra wide.
Under SparkGraphComputer -> Loading with BulkLoaderVertexProgram there are multiple tabs that don't seem to work correctly.

It's also interesting in that we can now see what's rendered via console vs. hardcoded. Might need to revisit some of these sections and test for correctness.

BTW, rebuilding the base image did the trick. Alternatively, it looks like I could have just installed zip.

dkuppitz · 2017-12-28T17:56:09Z

@robertdale I like the look :) Do you want to give it a try and tweak the CSS? I just took it from here, changed some colors and that's it. Maybe I just like the current look, because it just worked and I didn't have to mess too much with the CSS stuff.

actually reading some of the finer details, the WARNING under Mutating the Graph states that 'JAVA' or 'GROOVY' should appear in the code blocks. This doesn't work for me in this PR or 3.2.7 docs.

Not related to this PR, but you're right. That seems to be related to an AsciiDoc update as the blocks no longer react on mouse events. I guess we should just change the WARNING text accordingly.

robertdale · 2017-12-29T16:56:13Z

@dkuppitz my css skills are close to nil. what little i did play with it, i'm not sure it looks any better. and in the course of doing so, I've grown accustomed to it.

In any case, one outstanding issue it the Loading with BulkLoaderVertexProgram tabs. The docker image uses awk 4.0.1. The result is that the tab sequence is 4,1,2,3. I'm not entirely sure what the cause is. It works correctly in awk 4.1.

robertdale · 2017-12-31T16:09:41Z

@dkuppitz I pushed a CTR fix for invisibility to tp32/master. This fixes the screen from being rendered to 9999 pixels wide. It didn't help the code block language popup though.

robertdale · 2018-01-03T12:19:55Z

In the 'Console' tab, muli-line statements work. They have the \ continuation after the callout. However, on the 'language' tab, these seem to be lost thus copy-paste multi-line statements do not work.

robertdale · 2018-01-03T19:14:23Z

Excellent!
VOTE +1

spmallette · 2018-01-04T23:41:23Z

I saw some problems using docker above to generate docs....is that still an issue? I'm getting this:

 * source:   /usr/src/tinkerpop/docs/src/reference/gremlin-variants.asciidoc
   target:   /usr/src/tinkerpop/target/postprocess-asciidoc/reference/gremlin-variants.asciidoc
   progress: [==============================================>                                                     ] 46%ImportError: No module named six in <script> at line number 1
Type ':help' or ':h' for help.
Display stack trace? [yN]jython.eval('from gremlin_python.process.traversal import *')
groovysh_parse: 2: expecting EOF, found ')' @ line 2, column 21.
   GraphSONVersion.V2_0).create().createMapper()
                       ^

1 error
Type ':help' or ':h' for help.
Display stack trace? [yN]bytecode = mapper.readValue(jython.eval('GraphSONWriter().writeObject(j)').toString(), Bytecode.class)
groovysh_parse: 2: expecting EOF, found ')' @ line 2, column 4.
   s()) f.deleteDir()
      ^

1 error
Type ':help' or ':h' for help.
Display stack trace? [yN]f = new File('/tmp/tinkergraph.kryo')
groovysh_parse: 2: unexpected char: ''' @ line 2, column 42.
   of a lambda in Gremlin-Python.'
                                 ^

1 error
Type ':help' or ':h' for help.
Display stack trace? [yN]pb(277); '<2> The default lambda language is currently Gremlin-Python.'
groovysh_parse: 2: unexpected token: , @ line 2, column 1.
   , the strategy is'
   ^

1 error
Type ':help' or ':h' for help.
Display stack trace? [yN]pb(411); 'encoded in the Gremlin.Net bytecode and transmitted to the Gremlin traversal machine for re-construction machine-side.'


Last 10 lines of /usr/src/tinkerpop/target/postprocess-asciidoc/reference/gremlin-variants.asciidoc:

* `Traversal.toList()`
* `Traversal.toSet()`
* `Traversal.iterate()`

=== Gremlin-Python Sugar

Python supports meta-programming and operator overloading. There are three uses of these techniques in Gremlin-Python that
makes traversals a bit more concise.

[source,python]

xargs: /usr/src/tinkerpop/docs/preprocessor/preprocess-file.sh: exited with status 255; aborting

I get the same error before and after deleting docker images.

robertdale · 2018-01-04T23:51:31Z

Try rebuilding the docker images using master.

robertdale · 2018-01-04T23:59:07Z

Hmm.. there doesn't seem to be much difference. Are you deleting both the hadoop and base images?

spmallette · 2018-01-05T00:10:23Z

i deleted both when i did it, but i didn't rebuild from master - i built from this branch i guess

dkuppitz · 2018-01-05T19:53:38Z

I rebuilt my containers using this branch and docker/build.sh -d just finished gremlin-variants.asciidoc.

spmallette · 2018-01-06T20:21:33Z

tried again using images from this branch and died the same way. so weird when stuff like this happens with docker.

robertdale · 2018-01-08T15:13:48Z

@spmallette did you save your console log?

spmallette · 2018-01-08T18:17:38Z

don't have any of that stuff from my last attempt. what are you looking for exactly? i could fire it up again....

robertdale · 2018-01-09T23:58:28Z

@spmallette delete all of your images and then rebuild everything with docker/build.sh -d |tee /tmp/console.log and send me that output file. I wonder if it's choking on some python download. I've had that problem before. I just rebuilt everything from scratch and it still works. (I think I use master to build the image because I can override the maven url in .travis.install-maven.sh)

spmallette · 2018-01-18T23:46:40Z

I'm posting this to all open PRs of current relevance - this PR needs to be rebased against the branch it is targeted against given a broken python dependency that was published to pypi a day or so ago. I've pushed a fix on tp32 and master at this point and David Brown is working on getting an issue raised with the project that initiated the problem.

spmallette · 2018-03-09T21:16:17Z

wow - thanks to @robertdale with 34f924e because i can finally generate these docs with docker. i guess the question now is whether we rush to immediately look to merge this or push off until next release. i kinda feel like we should do the latter and not be hasty. I'd like some time to put in some other code in certain places and play around with it a bit before we get this on to a release branch and i don't know that i will have time for such things between now and first week of april when 3.2.8/3.3.2 are going out the door. any thoughts to the contrary?

dkuppitz · 2018-03-11T16:26:03Z

I have no problem pushing it off. It's unlikely that any other PR is going to affect the changes in this PR, so rebasing should be a no-brainer at any time.

spmallette · 2018-04-25T10:34:33Z

I was hoping to see us merge this for next release, but now that we have to release this patch because of the groovy bug, there won't be a lot of time to do much with the cool changes you have here. I was hoping we'd be able to see at least some GLV code translation in place for an actual release. Maybe that doesn't matter, in which case I guess we can merge this. Your call in that respect.

A question though about the tab system...the tabs look good with two options as shown in your screenshot, but we have Console, Groovy, Java (maybe we wouldn't do that), Python, Javascript and .NET - how does it look when we have all those? Does the tab system scale that way?

dkuppitz · 2018-04-25T14:16:26Z

It does scale to up to 4 tabs. Should we ever have more, we only need to tweak the maxTabs property and regenerate the CSS.

spmallette · 2018-05-09T17:25:46Z

I just generated docs locally on this branch and i didn't get the tabs - it showed radio buttons. something wrong with the stylesheet after rebase maybe?

dkuppitz · 2018-05-10T16:00:33Z

Worked well for me.

spmallette · 2018-05-10T16:07:18Z

i guess i will try again.............................................................................................................................

spmallette · 2018-05-10T19:44:22Z

Ok - the issue is related to how the tabs.css is in there:

<link rel="stylesheet" href="/docs/3.2.8-SNAPSHOT/stylesheets/tabs.css" />

if that were a relative url like everything else, we'd have it.

spmallette · 2018-05-12T00:33:30Z

nice @dkuppitz - i can finally VOTE +1 🥇

better merge it through fast before someone sees a problem! 😈

…ciidoc files.

jorgebay reviewed Dec 22, 2017

View reviewed changes

asfgit force-pushed the TINKERPOP-1447 branch from 07cb6a3 to bac969c Compare December 22, 2017 17:02

asfgit force-pushed the TINKERPOP-1447 branch 2 times, most recently from bca37a8 to 6c0b14c Compare January 24, 2018 21:14

asfgit force-pushed the TINKERPOP-1447 branch from 6c0b14c to 605c069 Compare March 9, 2018 15:35

asfgit force-pushed the TINKERPOP-1447 branch from 605c069 to 91f2f9b Compare April 26, 2018 00:03

dkuppitz changed the base branch from tp32 to tp33 April 26, 2018 00:03

TINKERPOP-1447 Added automatic tab generation for code snippets in as…

906f244

…ciidoc files.

asfgit force-pushed the TINKERPOP-1447 branch from 49726b8 to 906f244 Compare May 12, 2018 01:10

asfgit merged commit 906f244 into tp33 May 12, 2018

asfgit deleted the TINKERPOP-1447 branch May 19, 2018 09:28

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TINKERPOP-1447 Add some JavaScript intelligence to the documentation so that comments and output are not copied in a copy paste #766

TINKERPOP-1447 Add some JavaScript intelligence to the documentation so that comments and output are not copied in a copy paste #766

dkuppitz commented Dec 22, 2017

jorgebay Dec 22, 2017

dkuppitz Dec 22, 2017

robertdale commented Dec 27, 2017

dkuppitz commented Dec 27, 2017

robertdale commented Dec 27, 2017

robertdale commented Dec 27, 2017

dkuppitz commented Dec 28, 2017 •

edited

Loading

robertdale commented Dec 29, 2017

robertdale commented Dec 31, 2017

robertdale commented Jan 3, 2018

robertdale commented Jan 3, 2018

spmallette commented Jan 4, 2018

robertdale commented Jan 4, 2018

robertdale commented Jan 4, 2018

spmallette commented Jan 5, 2018

dkuppitz commented Jan 5, 2018

spmallette commented Jan 6, 2018

robertdale commented Jan 8, 2018

spmallette commented Jan 8, 2018

robertdale commented Jan 9, 2018

spmallette commented Jan 18, 2018

spmallette commented Mar 9, 2018

dkuppitz commented Mar 11, 2018

spmallette commented Apr 25, 2018

dkuppitz commented Apr 25, 2018

spmallette commented May 9, 2018

dkuppitz commented May 10, 2018

spmallette commented May 10, 2018

spmallette commented May 10, 2018

spmallette commented May 12, 2018

TINKERPOP-1447 Add some JavaScript intelligence to the documentation so that comments and output are not copied in a copy paste #766

TINKERPOP-1447 Add some JavaScript intelligence to the documentation so that comments and output are not copied in a copy paste #766

Conversation

dkuppitz commented Dec 22, 2017

jorgebay Dec 22, 2017

Choose a reason for hiding this comment

dkuppitz Dec 22, 2017

Choose a reason for hiding this comment

robertdale commented Dec 27, 2017

dkuppitz commented Dec 27, 2017

robertdale commented Dec 27, 2017

robertdale commented Dec 27, 2017

dkuppitz commented Dec 28, 2017 • edited Loading

robertdale commented Dec 29, 2017

robertdale commented Dec 31, 2017

robertdale commented Jan 3, 2018

robertdale commented Jan 3, 2018

spmallette commented Jan 4, 2018

robertdale commented Jan 4, 2018

robertdale commented Jan 4, 2018

spmallette commented Jan 5, 2018

dkuppitz commented Jan 5, 2018

spmallette commented Jan 6, 2018

robertdale commented Jan 8, 2018

spmallette commented Jan 8, 2018

robertdale commented Jan 9, 2018

spmallette commented Jan 18, 2018

spmallette commented Mar 9, 2018

dkuppitz commented Mar 11, 2018

spmallette commented Apr 25, 2018

dkuppitz commented Apr 25, 2018

spmallette commented May 9, 2018

dkuppitz commented May 10, 2018

spmallette commented May 10, 2018

spmallette commented May 10, 2018

spmallette commented May 12, 2018

dkuppitz commented Dec 28, 2017 •

edited

Loading