Skip to content
This repository

json doc generator now generates _index.json as it creates files. #3696

Open
wants to merge 4 commits into from

5 participants

Deepak Octavian Damiean Isaac Z. Schlueter Nodejs Jenkins Ben Noordhuis
Deepak
iapain commented

generate.js now accepts --output=<file> which tools/doc/json.js uses to write the output. If output file is missing then it executes callback. This was required to get the directory for new _index.json. Ideally, I left it for you to decide what to call this _index.json. There is already an index.json file which should not be generated at all. Additionally it sets title as first heading node text if applicable.

TODO

  • Add exclude files to avoid generation of index.json
  • Dependency map in Makefile (if subchapters like structure is required).
Deepak iapain Added output file option to gentest.
json doc generator now accepts output file as well.
json doc generator now generates index file automatically.
5152bf5
Deepak iapain closed this
Deepak iapain reopened this
Deepak
iapain commented

Damn I cannot attach it to the issue. It's for #3668

Octavian Damiean

What's up with this pull request? Is it going to get merged?

Deepak
iapain commented

It still needs some API related decisions from @isaacs. He urged me to write this patch but probably too busy with other stuff.

Octavian Damiean

I reckoned, just wanted to bump this a little.

Isaac Z. Schlueter
Collaborator

The index should be written to index.json, not _index.json.

Also, it's a bit odd having url be a an object. Maybe it should be just html: 'foo.html', json: 'foo.json' or something?

Deepak

@isaacs The reason I kept _index.json was index.markdown translates into index.json but I guess it's no longer required. In last commit I have renamed it to index.json. Moreover url.html, url.json is now just html, json as you suggested.

Nodejs Jenkins
Collaborator

Can one of the admins verify this patch?

Octavian Damiean

Indeed, it would be awesome to get this in, I don't quite understand what's taking eight months to review ...

Ben Noordhuis

Yeah, because we're all just twiddling our thumbs while raking in the big paychecks...

Look at the number of open issues and pull requests, now realize that there are only two or three people working on node.js full time.

If you want to do something constructive, start triaging bug reports and reviewing PRs.

Octavian Damiean

@bnoordhuis I didn't say that you guys weren't doing something but eight months for such a minor PR?

I'm not actively working on Node.js, I have other projects I'm actively contributing to.

Deepak

I must agree with @mainerror being a contributor it's very discouraging. I tried to bring it up few times on IRC to @isaacs in vain.

@bnoordhuis I totally respect what you guys are doing it's pretty cool and cutting edge but eventually it's core team who needs to organize and expand if it's required.

tools/doc/json.js
@@ -146,9 +159,62 @@ function doJSON(input, filename, cb) {
146 159 finishSection(current, stack[stack.length - 1]);
147 160 }
148 161
149   - return cb(null, root)
  162 + if (outfile) {
  163 + writeOututToFile(root, filename, outfile, indexfile, writeToIndexFile);
2
Isaac Z. Schlueter Collaborator
isaacs added a note

Typo? Should this be writeOutputToFile?

Deepak
iapain added a note

Sorry that's a typo. Will rectify in next commit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
tools/doc/json.js
((4 lines not shown))
149   - return cb(null, root)
  162 + if (outfile) {
  163 + writeOututToFile(root, filename, outfile, indexfile, writeToIndexFile);
  164 + }
  165 + else {
  166 + return cb(null, root)
  167 + }
  168 +}
  169 +
  170 +// write output object to outfile
  171 +function writeOututToFile(obj, sourcefile, outfile, indexfile, cb) {
  172 + fs.writeFile(outfile, JSON.stringify(obj, null, 2), function(err) {
  173 + if(err) {
  174 + throw new Error('error saving file - '+ err);
  175 + }
  176 + cb(obj, sourcefile, path.join(path.dirname(outfile), indexfile));
2
Isaac Z. Schlueter Collaborator
isaacs added a note

API here is a bit weird. cb functions should pass err as their first argument. Better to just call fs.writeFile with the supplied cb, than to force a throw.

Deepak
iapain added a note

I tried to keep the way it worked with console.log version. I remember there it threw an error as well but I can see you're right and in this setup it doesn't make any sense to throw an error. I will clean up this in a moment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Isaac Z. Schlueter
Collaborator

In general, it is better to light candles than to curse darkness. @mainerror, if you have comments on this patch, then share them. We welcome reviews from all parties. Though it does take some core team member to actually land it, you can help things along by providing feedback.

@iapain I'm a bit confused about what happens if just one of the files changes. Ie, if you've generated the index.json, and then we change doc/api/url.markdown, for instance. Will it generate a new index.json containing all of the previous files as well? How does it know to do index.json last?

Deepak

@isaacs I'm not tracking anything, it just regenerates index.json. Should it keep track for changes?

Deepak

@isaacs changes checked-in. I think now I got it what you were trying to say in previous comment. Actually writeToIndexFile checks if there is already an index.json file. It also check (in a way) that if it's proper index.json and then it update or create proper chapter entry into main json object which is written back to index.json

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Showing 4 unique commits by 2 authors.

Jul 12, 2012
Deepak iapain Added output file option to gentest.
json doc generator now accepts output file as well.
json doc generator now generates index file automatically.
5152bf5
Oct 03, 2012
Deepak iapain Now json doc generator writes index.json file.\n Renamed url.html and…
… url.json to html and json
24b1c2a
Mar 14, 2013
Deepak iapain Fixed some typos. `writeOutputToFile` does not raise and exception an…
…ymore.
64c39ba
Deepak iapain Merge branch 'master' into newdoc a5544bd
This page is out of date. Refresh to see the latest.

Showing 3 changed files with 76 additions and 5 deletions. Show diff stats Hide diff stats

  1. +1 1  Makefile
  2. +4 1 tools/doc/generate.js
  3. +71 3 tools/doc/json.js
2  Makefile
@@ -176,7 +176,7 @@ out/doc/%: doc/%
176 176 cp -r $< $@
177 177
178 178 out/doc/api/%.json: doc/api/%.markdown node
179   - out/Release/node tools/doc/generate.js --format=json $< > $@
  179 + out/Release/node tools/doc/generate.js --format=json $< --output=$@
180 180
181 181 out/doc/api/%.html: doc/api/%.markdown node
182 182 out/Release/node tools/doc/generate.js --format=html --template=doc/template.html $< > $@
5 tools/doc/generate.js
@@ -31,6 +31,7 @@ var args = process.argv.slice(2);
31 31 var format = 'json';
32 32 var template = null;
33 33 var inputFile = null;
  34 +var outputFile = null;
34 35
35 36 args.forEach(function (arg) {
36 37 if (!arg.match(/^\-\-/)) {
@@ -39,6 +40,8 @@ args.forEach(function (arg) {
39 40 format = arg.replace(/^\-\-format=/, '');
40 41 } else if (arg.match(/^\-\-template=/)) {
41 42 template = arg.replace(/^\-\-template=/, '');
  43 + } else if (arg.match(/^\-\-output=/)) {
  44 + outputFile = arg.replace(/^\-\-output=/, '');
42 45 }
43 46 })
44 47
@@ -100,7 +103,7 @@ function next(er, input) {
100 103 if (er) throw er;
101 104 switch (format) {
102 105 case 'json':
103   - require('./json.js')(input, inputFile, function(er, obj) {
  106 + require('./json.js')(input, inputFile, outputFile, function(er, obj) {
104 107 console.log(JSON.stringify(obj, null, 2));
105 108 if (er) throw er;
106 109 });
74 tools/doc/json.js
@@ -23,16 +23,25 @@ module.exports = doJSON;
23 23
24 24 // Take the lexed input, and return a JSON-encoded object
25 25 // A module looks like this: https://gist.github.com/1777387
26   -
  26 +var fs = require('fs');
  27 +var path = require('path');
27 28 var marked = require('marked');
28 29
29   -function doJSON(input, filename, cb) {
  30 +function doJSON(input, filename, outfile, cb) {
30 31 var root = {source: filename};
  32 + // Default index file
  33 + var indexfile = "index.json";
31 34 var stack = [root];
32 35 var depth = 0;
33 36 var current = root;
34 37 var state = null;
35 38 var lexed = marked.lexer(input);
  39 +
  40 + // If outfile is same as index file do nothing
  41 + if (outfile && outfile.slice(-indexfile.length) === indexfile) {
  42 + return;
  43 + }
  44 +
36 45 lexed.forEach(function (tok) {
37 46 var type = tok.type;
38 47 var text = tok.text;
@@ -55,6 +64,10 @@ function doJSON(input, filename, cb) {
55 64 return cb(new Error('Inappropriate heading level\n'+
56 65 JSON.stringify(tok)));
57 66 }
  67 + // set first heading to title
  68 + if (current && !current.title) {
  69 + current.title = tok.text;
  70 + }
58 71
59 72 // Sometimes we have two headings with a single
60 73 // blob of description. Treat as a clone.
@@ -146,9 +159,64 @@ function doJSON(input, filename, cb) {
146 159 finishSection(current, stack[stack.length - 1]);
147 160 }
148 161
149   - return cb(null, root)
  162 + if (outfile) {
  163 + writeOutputToFile(root, filename, outfile, indexfile, writeToIndexFile);
  164 + }
  165 + else {
  166 + return cb(null, root)
  167 + }
150 168 }
151 169
  170 +// write output object to outfile
  171 +function writeOutputToFile(obj, sourcefile, outfile, indexfile, cb) {
  172 + fs.writeFile(outfile, JSON.stringify(obj, null, 2), function(err) {
  173 + cb(err, obj, sourcefile, path.join(path.dirname(outfile), indexfile));
  174 + });
  175 +}
  176 +
  177 +// make an entry into index file
  178 +function writeToIndexFile(err, root, sourcefile, outfile) {
  179 + // check if there was an error writing file
  180 + if (err) {
  181 + throw new Error('error writing file - '+ e);
  182 + }
  183 +
  184 + // default type of an index
  185 + var obj = {"type":"index"};
  186 + var entry = {"source":sourcefile};
  187 +
  188 + // check if indexfile already exists
  189 + if (fs.existsSync(outfile)) {
  190 + var data = fs.readFileSync(outfile);
  191 + try {
  192 + obj = JSON.parse(data.toString());
  193 + }
  194 + catch(e) {
  195 + throw new Error('invalid json data - '+ e);
  196 + }
  197 + }
  198 + // check if index file is valid
  199 + if (obj.type !== "index") {
  200 + throw new Error('invalid index file - '+ outfile);
  201 + }
  202 + // construct an entry object
  203 + entry.title = root.title;
  204 + entry.html = path.basename(sourcefile).replace(/\.(markdown|md)/i, ".html");
  205 + entry.json = entry.html.replace(/\.html/i, ".json");
  206 +
  207 + // append mode
  208 + if (obj.chapters && typeof obj.chapters === "object") {
  209 + obj.chapters.push(entry);
  210 + }
  211 + else {
  212 + obj.chapters = [entry];
  213 + }
  214 + fs.writeFile(outfile, JSON.stringify(obj, null, 2), function(err) {
  215 + if(err) {
  216 + throw new Error('error saving file - '+ err);
  217 + }
  218 + });
  219 +}
152 220
153 221 // go from something like this:
154 222 // [ { type: 'list_item_start' },

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.