This repository has been archived by the owner on Oct 15, 2022. It is now read-only.
Scala: New Scala Docs Fathead #704
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add a new Scala Documentation fathead. Resolves duckduckgo#692
Scala DocumentationDescription: Scala Reference Example Query: scala implicitConversions, scala FallbackArrayBuilding, scala stringContext Tab Name: About Source: http://www.scala-lang.org/api/2.12.1/ These are the important fields from the IA page. Please check these for errors or missing information and update the IA page This is an automated message which will be updated as changes are made to the IA page |
|
@pjhampton Please review |
|
@sam09 The ID of the IA and the directory in |
moollaza
suggested changes
Jan 26, 2017
lib/fathead/scala/parse.py
Outdated
| data[12] = url | ||
| if abstract == None: | ||
| abstract = "No data found" | ||
| abstract = abstract.replace("\n", "\\n").replace("\t","\\t") |
There was a problem hiding this comment.
@sam09 we can't have escaped tabs, please use 2 or 4 spaces instead of \\t
lib/fathead/scala/parse.py
Outdated
| data[1] = 'A' | ||
| data[12] = url | ||
| if abstract == None: | ||
| abstract = "No data found" |
There was a problem hiding this comment.
@sam09 if we have no data to show we should skip the article -- no point in telling the user that we have nothing to show them 👍
lib/fathead/scala/parse.py
Outdated
|
|
||
| url = BASE_URL + filename.replace("./download/api/", "") #title_link.find("a").get('href') | ||
|
|
||
| article_text = soup.find_all("div", attrs={"class" : "comment"})[0].text |
There was a problem hiding this comment.
@sam09 we should pull out code blocks from the docs and put them into <pre><code> so we can display them properly. Likewise and text content needs to be put inside a <p>. That way we can closely mimic the display in the docs 👍
Otherwise we're showing code as plaintext without formatting which looks ugly and is hard to read
There was a problem hiding this comment.
For example: http://scala-lang.org/files/archive/nightly/docs/library/index.html#scala.Dynamic
We can have:
<section>
<p>{description text}</p>
<pre>
<code>{code}</code>
</pre>
</section>
|
@sam09 this is a great start! I've updated your IA page so subsequent changes to your output.txt will be autoinstalled to Beta for you to test yourself 👍 |
moollaza
suggested changes
Jan 31, 2017
lib/fathead/scala/triggers.txt
Outdated
| @@ -0,0 +1,2 @@ | |||
| scala | |||
| Scala | |||
There was a problem hiding this comment.
@sam09 triggers aren't case sensitive, so all you need is scala 👍
lib/fathead/scala/output.txt
Outdated
| Tuple20 A <section class="prog_container"><p>A tuple of 20 elements; the canonical representation of a scala.Product20.\n</p></section> https://www.scala-lang.org/api/current/scala/Tuple20.html | ||
| deprecatedName A <section class="prog_container"><p>An annotation that designates the name of the parameter to which it is\napplied as deprecated. Using that name in a named argument generates\na deprecation warning.</p><p>For instance, evaluating the code below in the Scala interpreter</p><pre>def inc(x: Int, @deprecatedName('y) n: Int): Int = x + n\ninc(1, y = 2)</pre><p>will produce the following output:</p><pre>warning: there were 1 deprecation warnings; re-run with -deprecation for details\nres0: Int = 3</pre></section> https://www.scala-lang.org/api/current/scala/deprecatedName.html | ||
| throws A <section class="prog_container"><p>Annotation for specifying the exceptions thrown by a method.\nFor example:</p><pre>class Reader(fname: String) {\nprivate val in = new BufferedReader(new FileReader(fname))\n@throws[IOException]("if the file doesn't exist")\ndef read() = in.read()\n}</pre></section> https://www.scala-lang.org/api/current/scala/throws.html | ||
| language A <section class="prog_container"><p> The scala.language object controls the language features available to the programmer, as proposed in the\n SIP-18 document.</p><p> Each of these features has to be explicitly imported into the current scope to become available:</p><pre>import language.postfixOps // or language._\nList(1, 2, 3) reverse</pre><p> The language features are:</p><pre> dynamics enables defining calls rewriting using the Dynamic trait postfixOps enables postfix operators reflectiveCalls enables using structural typesimplicitConversions enables defining implicit methods and members higherKinds enables writing higher-kinded types existentials enables writing existential types experimental contains newer features that have not yet been tested in production\n</pre></section> https://www.scala-lang.org/api/current/scala/language$.html |
There was a problem hiding this comment.
@sam09 it looks like we're turning a <ul> into a <pre><code> here?
https://www.scala-lang.org/api/current/scala/language$.html
We can put the <ul> into our <section> and it will work. If you add the class prog__ul to the <ul> then very soon, it will also display bullets. Right now our site CSS disables bullets on <ul>, but I'm adjusting that 👍
|
@moollaza @pjhampton Sorry for the delay.. Was quite busy. For adding lists as |
|
@sam09 just one minor fix and this should be good for merge 👍 |
|
@moollaza I don't get what you mean by the minor fix. Sorry 😄 |
moollaza
suggested changes
Feb 7, 2017
lib/fathead/scala/parse.py
Outdated
| article_text = "" | ||
| for i in article.children: | ||
| if i.name == "pre" or i.name == "code": | ||
| article_text += "<pre>" + extarct_article_text(i) + "</pre>" |
There was a problem hiding this comment.
@sam09 please use <code> for inline code.
For larger code blocks that should have the full width of the IA container, use <pre><code>text...</code></pre>
There was a problem hiding this comment.
i.e. This: https://beta.duckduckgo.com/?q=UninitializedFieldError&ia=about&iax=1 should probably have an inline code bloc, so the code isn't on its own line, breaking up the sentence 👍
There was a problem hiding this comment.
@moollaza That would mean that when a code appears inside a paragraph it should be inline and otherwise it should break the page.
There was a problem hiding this comment.
@sam09 yes, exactly.
Inline snippets should be <code> to keep the flow of the text, but change the look of it.
<pre><code> is used to make a code block, which our front end will also apply syntax highlighting to
Gah, sorry! -- My PR review didn't post -- please take a look now. |
|
@sam09 Thanks! This LGTM 👍 Excellent work! 💯 |
moollaza
approved these changes
Feb 7, 2017
|
@sam09 this is now merged! It should be live on DuckDuckGo within one week (or during the next release) and we'll be sure to notify you when it happens. After your Instant Answer goes live for the World to use, you will be able to see traffic information about your IA as well as receive feedback directly from users. Please refer to the maintenance guide and docs to keep up with potential changes to your Instant Answer. Thank you again for your help! |
moollaza
reviewed
Feb 8, 2017
| def get_all_files(): | ||
| doc_files = [] | ||
|
|
||
| for (path, dirs, files) in os.walk(BASE_DIRECTORY): |
There was a problem hiding this comment.
@sam09 any reason we're not grabbing all the files in all the subdirectories?
Seems like we're only scratching the surface of the Scala API!
There was a problem hiding this comment.
@moollaza I actually used only scala directory while developing and forgot to add other directories while making the pull 😄
There was a problem hiding this comment.
I think we could create a new issue for it
mayank
added a commit
to mayank/zeroclickinfo-fathead
that referenced
this pull request
Feb 17, 2017
* Removed Django_built_ins (duckduckgo#677) * adding more coverage to the php functions (duckduckgo#678) * adding debugging with some class reading changes * library update for php parsing * updating code for enable/disable debugging * fix for repeated functions ⛳ * updated parsers, also updated readme, easy debugging some whitespace changes some more and more and more whitespace changes some more and more and more whitespace changes some more and more and more whitespace changes adding more coverage to the php functions * adding debugging with some class reading changes , updated readme new parser with improved coverage removed some code that was not in use whitepsace corrections in code a small indentation fix arrrhhhvi parse.js a whitespace * added colored console, disabled debugging * more coverage for "function" keyword * fixed the urls for more info * adding classes as categories * output.txt generated with results * Update PULL_REQUEST_TEMPLATE.md * Jquery: Add article aliases(redirects) for append,after,before (duckduckgo#610) (duckduckgo#679) * Add append,after,before redirects (duckduckgo#610) * Add append,after,before redirects (Fixed Pull request issues) (duckduckgo#610) * Ending period on disambiguation should be optional * jQuery Fathead: Add arguments type & description, Fixes duckduckgo#319 (duckduckgo#682) * Add argument desc * Completed arg descriptions * Exclude xi:include argsDescription * Remove whitespace * Fix args desc parse issue * Change argsdesc from code to unordered list * Format arguments name & type * Improove content formatting * Update output.txt * Final changes * Fix sub-heading and remove ill formatted redirects * Remove whitespace * Perl 6 Documentation: Fix the parser so that it generates correct links (duckduckgo#680) * Perl6: Use https instead of http * Perl6: Fix broken links * Perl6: Update output.txt * New SQLAlchemy Fathead Fixes duckduckgo#462 (duckduckgo#483) * New fathead for SQLAlchemy Fixes duckduckgo#462 * corrected unwanted redundancy in output.txt * Corrected improperly escaped '\n's * Added IAs for sqlalchemy events, internals and exceptions without disambiguation entries * Added disambiguation entries and handled a few corner cases * removed unnecessary '\t's, changed indentation and removed inline styles * Adding examples to all the topics of numpy. (duckduckgo#688) * Added Examples Template * Added examples for all topics of numpy * Typo fixed * Removed Extra spans from examples * using highlight-python to get examples * Remove extra new line characters and correct docmentation * Removed spaces * JavaScript: New NPM CLI Fathead (duckduckgo#694) * First commit * Truncated fetch.sh * Scraping Home Page and indentation fixed * Bugs fixed * Final changes * First commit * Removed unnecessary data * Indentation fixed and redirects added * Final changes * Order of desc and code changed * Changes made * STY: Fix PEP8 issues * Remove incorrect comment * Fix escaping issues in numpy fathead examples. * Perl 6 Documentation: Fix parse.pl so that it can handle the latest html structure. (duckduckgo#726) * Perl 6 Documentation: Modify the way to parse so that it can handle the latest (i.e. 2017.01) html structure * Perl 6 Documentation: Update output.txt corresponding with parse.pl's one * Perl 6 Documentation: Fix code blocks so that they don't contain escaped characters * Perl 6 Documentation: Update output.txt corresponding with parse.pl's one * C++ Fathead: Add language keywords (duckduckgo#626) * Initial Commit: Keywords Added * Fixed misspell * File Name changed * Removed Blanked lines * add "boost" trigger word * Java: New JMeter Fathead (duckduckgo#700) * Fixing conflicts * Code review and adding trigger file * Formatting updates * jmeter fathead inital commit * test commit to trigger build * Updating directory name for dax to pickup * Code review updates * Updating triggers * Wrapping up details inside <p> tags * JavaScript: New ESLint Fathead (duckduckgo#690) * Push initial version. * Test fetch + parse - Add output.txt - Tune download folder structure. * Code Add parse.sh * >Code: Process all categories instead of one. - Apply forEach on categories. - Polish tests. * Documentation: References, Intro, Testing. * Doc: Move notes to readme.md. * Test: Update current output.txt. * Infrastructure: Rename 'javascript_eslint/' to 'eslint/'. * Infrastructure: Adapt package.json - Name. * Add TXT for trigger words. * Add TXT for trigger words. * Code: Abstract layout - Add markdown-js. - Test inline code. * Testing: Run fetch + parse. * Infrastructure: 'parse.h' - Add **npm install** before calling 'parse.js'. * Update trigger_words.txt Add common search quotes. * +x permissions to fetch and parse sh * Code: Format summary text: - Start with uppercase letter. - End with dot. * Cosmetics: Code analysis. * Run fetch + parse. * JavaScript: New react native fathead (duckduckgo#713) * Download docs for the current stable release. * Parsing of docs * Module title and intro text. * Identifying prop sections and method sections. * Property signature, paragraph and url. * Methods section parsed. * Output result to output file. * Add parse.sh and output_statistics.awk. * Add output.txt for testing on beta. * Fix issues listed in review * Methods or Props with no information other than the signature should not add an empty <p></p> to output. * Method names should not be classified in classes. The module name is the class. * Fix erroneous prop_name * Some of the documentations pages are not of expected format. * Really sort on first field. * Handle disambiguations. * Add redirect.py that generates disambiguations. * Add requirements.txt that specifies requirements for this fathead. * Javascript: New Webpack Fathead (duckduckgo#720) * first steps of the fetching script * getting all the hrefs * cleaning up the file, and adding some comments * a working fetch script * placing it all under the same dir * rename the package, and adding some info * adding readme * adding node version * a very first beginning of parse.js * fixing node version dependency * adding a new line to the end of each entry * adding parse.sh to parse all the files in the download dir * bugfix: not trying to iterate when there are no more tags * small cleanup * adding counter * escaping strings * fixing a bug in the escaping * loading the pages with the original urls (and not file://) * fixing counter zero-based bug * refactoring: increasing readability * bugfix: not escaping links inside paragraphs * adding output.txt * removing empty lines * strip out a tags * Python: New Arrow Fathead (duckduckgo#669) (duckduckgo#723) * New Arrow Fathead (duckduckgo#669) * Fixed wrong formatting of abstract in output.txt The paragraph and code are supposed to be on the same level * Python: New Scipy fathead (duckduckgo#731) * scipy: Add fetcher scripts. * scipy: parser for scipy fathead. * scipy: Add parser runner * scipy: Add parser python requirements * scipy: Add a README file * scipy: Add trigger words file * scipy: Add fathead output * adding redirector script, and making redirect for . seperated values (duckduckgo#743) * JavaScript: New Lodash Fathead (duckduckgo#719) * Initial Commit * second commit * nothing huge * nothing major * nothing major * fixed fetch.sh * Nothing major * nothing much * added some redirects * fixed redirects * added more redirects * added methods.txt, some redirects, and formatted the code a bit * one /t was missing * updated txt with an extra tab * added formatting for abstract * removed extra /n in abstract formatting * Changed summary from text to html to count it <br> in <p> * stripped anchor tags * Added <code> tags in anchor tags title * fixed arguments and trigger words.txt * Scala: New Scala Docs Fathead (duckduckgo#704) * Initial Commit Add a new Scala Documentation fathead. Resolves duckduckgo#692 * Remove Debug Statements * Add requirements.txt * Add output.txt File * Fix minor issues * ADd triggers.txt * Add formatting to answers * Add recursive parsing of lists * Fix issue with inline code tag * sqlalchemy: Added requirements.txt and changed headings in output.txt (duckduckgo#703) * Added requirements.txt and changed header's <span> to <pre><code> * Corrected paragraph,list and table displays * MDN CSS: Add missing documentation articles (duckduckgo#702) * Rename --* to its proper name custom properties (--*). * Sort tags.txt. * Remove parameter from scale3d(). * Remove :full-screen. There is :fullscreen already. * Add creating of more redirects. Use a variant of title found in <code> part of description paragraph to create redirects. * Handle more than one alternate titles. * Make writing of redirects lazy. Write redirects only after all the articles have been written so that all redirects that have the same title as an article are dropped. The logic being that an article has a higher precedence than a redirect and thus a redirect should not prevent an article from being written. Thus, redirects get to use a different hash from the articles hash when keeping track of titles. * Check for duplication in create_article function. This ensures whenever create_article is called outside the parse loop we do not end up with duplicate titles in output.txt. * Fix comment indentation. * Add parsing of Values and Units. * Create redirects for all versions of :not(). * Remove !important. It is missing in https://developer.mozilla.org/en-US/docs/Web/CSS/Reference * tree-cell is not covered by Mozilla CSS reference. * Fix custom-ident typo. * Lowercase title in create_article. In some cases the title is not lowercased beforehand. * Replace <indent> with ident which is covered. * Add parsing of units for @font-feature-values. * Use exact element name to create link if it is present. This makes the article link scroll down to the exact element in the page more accurately than using the parent id. * No need for redirects for titles with ' * Appearance does not appear on mdn css reference. * Background-position-{x, y} are not on mdn css reference. * Bleed is not no mdn css reference. * border-start is not in mdn css reference. * Fix contrast typo. * fit is not covered in mdn css reference. * font-height is not covered by mdn css reference. * format is not covered by mdn css reference. * fr is not covered by mdn css reference. * grid-span is not covered by mdn css reference. * mark is not covered by mdn css reference. * Fix scroll-behavior typo. * SkewZ() is not covered by mdn css reference. * wrap-mode is not covered by mdn css reference. * Stop Mojo::IOLoop gracefully. See http://mojolicious.org/perldoc/Mojo/IOLoop#stop_gracefully * Delete keywords requiring extra processing or with 404 pages. See https://gist.github.com/hchienjo/c61adddbe63641a7b3c9de20a9995422 * Add output.txt. * Remove parsing of values data. This messes up with some of the articles as the titles collide. * Fetch only links in /docs/Web/CSS/ path. * Limit accepted alternate titles. * Parsing of #Feature_value_blocks is not harmful. * Parse values for font-variant-alternate. This is because it contains entries that are in tags.txt like: 1. swash() 2. ornaments() 3. annotation() * Remove "using css" from titles. * <counter> actually points to https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Lists_and_Counters/Using_CSS_counters * <flex> is converedd by flex. * mozilla.org/en-US/docs/Web/CSS/ident leads to a 404 error. * Add output.txt. * tighten rules on alternate titles. * Drop incorrect *.warning redirect. * Remove useless comment. * Limit creation of alternate titles for redirects. * Add output.txt. * No css snippet should be a redirect entry title. * Add output.txt. * Ensure removing brackets from titles preserves meaning. For instance :-moz-locale-dir(ltr) should be :-moz-locale-dir ltr and not :-moz-locale-dirltr. * Add output.txt. * New Six Library Fathead duckduckgo#453 (duckduckgo#532) * added Readme and fetch * added parse file * added class to store info about python_six lib * added class to parse the python six library content and added output.txt file * added new template and fixed issues * updated output.txt * added trigger words * if code section is not present then skip that in the abstraction * fixed issue of newline in abstarction * fixed indention issue * renamed the directory to match the id * change the trigger words * New Python Pip Fathead (duckduckgo#465) * parser,fetch and requirements added * Prelim pip fathead completed * Made required changes * SOme changes to parser * Fixed import file error. Also reduced output size to display only relevant information * Fixed some parsing errors * Serving pages to command <pip command> * Updated code to handle formatting error * added trigger words file and updated code * corrected few errors * New Flask Fathead (duckduckgo#459) * Added flask fathead * Added flask fathead * Parsed API documentation Flask * Escaped html symbols and newlines * Escaped html symbols and newlines * Fixed HTML class for closing quote * Fixed HTML class for closing quote * Fixes duckduckgo#751 Improves Redirects * Fixes duckduckgo#751 Changes done * Javascript: New Electron Fathead (duckduckgo#721) * First commit * Truncated fetch.sh * Scraping Home Page and indentation fixed * Bugs fixed * Final changes * First commit - Electron branch * Removed unnecessary data * Modified code * Indentation fixed * Final changes * Fixes duckduckgo#757 Updates docs and directory structure * Fixes duckduckgo#757: Some final clean ups * Added trigger_words.txt * added angular javascript trigger along with angular js :duckduckgo#755 * changes done as requested * updated parse.sh and readme * Fixed minor indentation issue. * Sort and make sure entries are unique. * Fix issue with parsing the newest docs. * Apparently, the docs could not be parsed anymore. This was solved by checking that the elements used were really bs4.element.Tag elements and actually had the .name etc properties. * Java: New Java Fathead (Continue duckduckgo#642) (duckduckgo#774) * Added method coverage data for the Java fathead * Added methods to output.txt * Removed classname from several functions * Fixed formatting for output and coverage * Added classesto methods in methods.txt and updated file * Coverage data and output are sorted if they exist * Added sorted methods.txt * Stripped methods of brackets and parameters * Added sorted version of methods.txt * Added a space between method description and url * Added formatted method parameters to methods.txt and output.txt * Added method output and coverage without parameters * Changed method_url find parameter from "href" to "a" * Removed test_parse.py due to it being a redundant test * Added tests for checking the output is correct in output.txt and method coverage * Removed pipe to uniq -u and added sort -u instead * Removed test cases for method parameter formatting * Fixed spacing in method coverage so that sort -u recognizes duplicates * Removed output.txt from .gitignore * Added -f option to sort in parse.sh * Excluded methods without parameters from being appended twice to output.txt * Fixed method_url parsing and reverted method extraction function * Fixed formatting issues on cover and output so that sort -u recognizes duplicates * Added <section> to method descriptions and an additional tab * Remove output.txt after beta upload * Python: New PEP Fathead (duckduckgo#627) * Added PEP Fathead. * Removed data.url file. * Tab replacement now 4 spaces. * Updated trigger words. * Adding output file. * Updated title. * Updated with redirects. * Removed meta.txt file. * Removed 'PEP' trigger word. * Updated parser from review. * Update Parser and redirects. * Updating output.txt file for beta server. * Updated output.txt * Updated Parser for new lines. * Updated parser. * Patched Parser flush bug. * Fixes spacing issue. * Refactored parser. * Updated parser to include newlines in code blocks. * Updated parser. * Updated parser to handle <p> * JavaScript: New Vuejs Fathead (duckduckgo#776) * Initial commit * added trigger words.txt * added more redirects * fixed redirect function * fixed abstract title * fixed formatting of code in abstract * fixed trigger words, changed order of abstraction * Removed unnecessary redirects * Python Coverage Fathead (duckduckgo#738) * inital commit to repo * fixes incorrect file write * adds trigger words * updates README to install requirements * corrects inconsistency of method docstrings * removes blank line in triggerwords, adds output.txt * removes minor version spec from pip install doc line * removes original site URL and text * changes html cleanup to preserve some tags * removes debugging print line, swaps <cite> tags with <code> tags * Ocaml: New OPAM fathead. (duckduckgo#775) * Updated stale OPAM fathead. * Added trigger words. * Updated parser to deal with version. * Updated parser. Updated output.txt * npm CLI: Added Categories (duckduckgo#754) * Added list of commands for npm cli * Removed npm.txt * Updated from npm-cli.txt * Delete npm-cli.txt * Revert "Swift Docs: Readme updated and url changed from 2.2 to 3.1" (duckduckgo#790)
mayank
added a commit
to mayank/zeroclickinfo-fathead
that referenced
this pull request
Feb 18, 2017
* Removed Django_built_ins (duckduckgo#677) * adding more coverage to the php functions (duckduckgo#678) * adding debugging with some class reading changes * library update for php parsing * updating code for enable/disable debugging * fix for repeated functions ⛳ * updated parsers, also updated readme, easy debugging some whitespace changes some more and more and more whitespace changes some more and more and more whitespace changes some more and more and more whitespace changes adding more coverage to the php functions * adding debugging with some class reading changes , updated readme new parser with improved coverage removed some code that was not in use whitepsace corrections in code a small indentation fix arrrhhhvi parse.js a whitespace * added colored console, disabled debugging * more coverage for "function" keyword * fixed the urls for more info * adding classes as categories * output.txt generated with results * Update PULL_REQUEST_TEMPLATE.md * Jquery: Add article aliases(redirects) for append,after,before (duckduckgo#610) (duckduckgo#679) * Add append,after,before redirects (duckduckgo#610) * Add append,after,before redirects (Fixed Pull request issues) (duckduckgo#610) * Ending period on disambiguation should be optional * jQuery Fathead: Add arguments type & description, Fixes duckduckgo#319 (duckduckgo#682) * Add argument desc * Completed arg descriptions * Exclude xi:include argsDescription * Remove whitespace * Fix args desc parse issue * Change argsdesc from code to unordered list * Format arguments name & type * Improove content formatting * Update output.txt * Final changes * Fix sub-heading and remove ill formatted redirects * Remove whitespace * Perl 6 Documentation: Fix the parser so that it generates correct links (duckduckgo#680) * Perl6: Use https instead of http * Perl6: Fix broken links * Perl6: Update output.txt * New SQLAlchemy Fathead Fixes duckduckgo#462 (duckduckgo#483) * New fathead for SQLAlchemy Fixes duckduckgo#462 * corrected unwanted redundancy in output.txt * Corrected improperly escaped '\n's * Added IAs for sqlalchemy events, internals and exceptions without disambiguation entries * Added disambiguation entries and handled a few corner cases * removed unnecessary '\t's, changed indentation and removed inline styles * Adding examples to all the topics of numpy. (duckduckgo#688) * Added Examples Template * Added examples for all topics of numpy * Typo fixed * Removed Extra spans from examples * using highlight-python to get examples * Remove extra new line characters and correct docmentation * Removed spaces * JavaScript: New NPM CLI Fathead (duckduckgo#694) * First commit * Truncated fetch.sh * Scraping Home Page and indentation fixed * Bugs fixed * Final changes * First commit * Removed unnecessary data * Indentation fixed and redirects added * Final changes * Order of desc and code changed * Changes made * STY: Fix PEP8 issues * Remove incorrect comment * Fix escaping issues in numpy fathead examples. * Perl 6 Documentation: Fix parse.pl so that it can handle the latest html structure. (duckduckgo#726) * Perl 6 Documentation: Modify the way to parse so that it can handle the latest (i.e. 2017.01) html structure * Perl 6 Documentation: Update output.txt corresponding with parse.pl's one * Perl 6 Documentation: Fix code blocks so that they don't contain escaped characters * Perl 6 Documentation: Update output.txt corresponding with parse.pl's one * C++ Fathead: Add language keywords (duckduckgo#626) * Initial Commit: Keywords Added * Fixed misspell * File Name changed * Removed Blanked lines * add "boost" trigger word * Java: New JMeter Fathead (duckduckgo#700) * Fixing conflicts * Code review and adding trigger file * Formatting updates * jmeter fathead inital commit * test commit to trigger build * Updating directory name for dax to pickup * Code review updates * Updating triggers * Wrapping up details inside <p> tags * JavaScript: New ESLint Fathead (duckduckgo#690) * Push initial version. * Test fetch + parse - Add output.txt - Tune download folder structure. * Code Add parse.sh * >Code: Process all categories instead of one. - Apply forEach on categories. - Polish tests. * Documentation: References, Intro, Testing. * Doc: Move notes to readme.md. * Test: Update current output.txt. * Infrastructure: Rename 'javascript_eslint/' to 'eslint/'. * Infrastructure: Adapt package.json - Name. * Add TXT for trigger words. * Add TXT for trigger words. * Code: Abstract layout - Add markdown-js. - Test inline code. * Testing: Run fetch + parse. * Infrastructure: 'parse.h' - Add **npm install** before calling 'parse.js'. * Update trigger_words.txt Add common search quotes. * +x permissions to fetch and parse sh * Code: Format summary text: - Start with uppercase letter. - End with dot. * Cosmetics: Code analysis. * Run fetch + parse. * JavaScript: New react native fathead (duckduckgo#713) * Download docs for the current stable release. * Parsing of docs * Module title and intro text. * Identifying prop sections and method sections. * Property signature, paragraph and url. * Methods section parsed. * Output result to output file. * Add parse.sh and output_statistics.awk. * Add output.txt for testing on beta. * Fix issues listed in review * Methods or Props with no information other than the signature should not add an empty <p></p> to output. * Method names should not be classified in classes. The module name is the class. * Fix erroneous prop_name * Some of the documentations pages are not of expected format. * Really sort on first field. * Handle disambiguations. * Add redirect.py that generates disambiguations. * Add requirements.txt that specifies requirements for this fathead. * Javascript: New Webpack Fathead (duckduckgo#720) * first steps of the fetching script * getting all the hrefs * cleaning up the file, and adding some comments * a working fetch script * placing it all under the same dir * rename the package, and adding some info * adding readme * adding node version * a very first beginning of parse.js * fixing node version dependency * adding a new line to the end of each entry * adding parse.sh to parse all the files in the download dir * bugfix: not trying to iterate when there are no more tags * small cleanup * adding counter * escaping strings * fixing a bug in the escaping * loading the pages with the original urls (and not file://) * fixing counter zero-based bug * refactoring: increasing readability * bugfix: not escaping links inside paragraphs * adding output.txt * removing empty lines * strip out a tags * Python: New Arrow Fathead (duckduckgo#669) (duckduckgo#723) * New Arrow Fathead (duckduckgo#669) * Fixed wrong formatting of abstract in output.txt The paragraph and code are supposed to be on the same level * Python: New Scipy fathead (duckduckgo#731) * scipy: Add fetcher scripts. * scipy: parser for scipy fathead. * scipy: Add parser runner * scipy: Add parser python requirements * scipy: Add a README file * scipy: Add trigger words file * scipy: Add fathead output * adding redirector script, and making redirect for . seperated values (duckduckgo#743) * JavaScript: New Lodash Fathead (duckduckgo#719) * Initial Commit * second commit * nothing huge * nothing major * nothing major * fixed fetch.sh * Nothing major * nothing much * added some redirects * fixed redirects * added more redirects * added methods.txt, some redirects, and formatted the code a bit * one /t was missing * updated txt with an extra tab * added formatting for abstract * removed extra /n in abstract formatting * Changed summary from text to html to count it <br> in <p> * stripped anchor tags * Added <code> tags in anchor tags title * fixed arguments and trigger words.txt * Scala: New Scala Docs Fathead (duckduckgo#704) * Initial Commit Add a new Scala Documentation fathead. Resolves duckduckgo#692 * Remove Debug Statements * Add requirements.txt * Add output.txt File * Fix minor issues * ADd triggers.txt * Add formatting to answers * Add recursive parsing of lists * Fix issue with inline code tag * sqlalchemy: Added requirements.txt and changed headings in output.txt (duckduckgo#703) * Added requirements.txt and changed header's <span> to <pre><code> * Corrected paragraph,list and table displays * MDN CSS: Add missing documentation articles (duckduckgo#702) * Rename --* to its proper name custom properties (--*). * Sort tags.txt. * Remove parameter from scale3d(). * Remove :full-screen. There is :fullscreen already. * Add creating of more redirects. Use a variant of title found in <code> part of description paragraph to create redirects. * Handle more than one alternate titles. * Make writing of redirects lazy. Write redirects only after all the articles have been written so that all redirects that have the same title as an article are dropped. The logic being that an article has a higher precedence than a redirect and thus a redirect should not prevent an article from being written. Thus, redirects get to use a different hash from the articles hash when keeping track of titles. * Check for duplication in create_article function. This ensures whenever create_article is called outside the parse loop we do not end up with duplicate titles in output.txt. * Fix comment indentation. * Add parsing of Values and Units. * Create redirects for all versions of :not(). * Remove !important. It is missing in https://developer.mozilla.org/en-US/docs/Web/CSS/Reference * tree-cell is not covered by Mozilla CSS reference. * Fix custom-ident typo. * Lowercase title in create_article. In some cases the title is not lowercased beforehand. * Replace <indent> with ident which is covered. * Add parsing of units for @font-feature-values. * Use exact element name to create link if it is present. This makes the article link scroll down to the exact element in the page more accurately than using the parent id. * No need for redirects for titles with ' * Appearance does not appear on mdn css reference. * Background-position-{x, y} are not on mdn css reference. * Bleed is not no mdn css reference. * border-start is not in mdn css reference. * Fix contrast typo. * fit is not covered in mdn css reference. * font-height is not covered by mdn css reference. * format is not covered by mdn css reference. * fr is not covered by mdn css reference. * grid-span is not covered by mdn css reference. * mark is not covered by mdn css reference. * Fix scroll-behavior typo. * SkewZ() is not covered by mdn css reference. * wrap-mode is not covered by mdn css reference. * Stop Mojo::IOLoop gracefully. See http://mojolicious.org/perldoc/Mojo/IOLoop#stop_gracefully * Delete keywords requiring extra processing or with 404 pages. See https://gist.github.com/hchienjo/c61adddbe63641a7b3c9de20a9995422 * Add output.txt. * Remove parsing of values data. This messes up with some of the articles as the titles collide. * Fetch only links in /docs/Web/CSS/ path. * Limit accepted alternate titles. * Parsing of #Feature_value_blocks is not harmful. * Parse values for font-variant-alternate. This is because it contains entries that are in tags.txt like: 1. swash() 2. ornaments() 3. annotation() * Remove "using css" from titles. * <counter> actually points to https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Lists_and_Counters/Using_CSS_counters * <flex> is converedd by flex. * mozilla.org/en-US/docs/Web/CSS/ident leads to a 404 error. * Add output.txt. * tighten rules on alternate titles. * Drop incorrect *.warning redirect. * Remove useless comment. * Limit creation of alternate titles for redirects. * Add output.txt. * No css snippet should be a redirect entry title. * Add output.txt. * Ensure removing brackets from titles preserves meaning. For instance :-moz-locale-dir(ltr) should be :-moz-locale-dir ltr and not :-moz-locale-dirltr. * Add output.txt. * New Six Library Fathead duckduckgo#453 (duckduckgo#532) * added Readme and fetch * added parse file * added class to store info about python_six lib * added class to parse the python six library content and added output.txt file * added new template and fixed issues * updated output.txt * added trigger words * if code section is not present then skip that in the abstraction * fixed issue of newline in abstarction * fixed indention issue * renamed the directory to match the id * change the trigger words * New Python Pip Fathead (duckduckgo#465) * parser,fetch and requirements added * Prelim pip fathead completed * Made required changes * SOme changes to parser * Fixed import file error. Also reduced output size to display only relevant information * Fixed some parsing errors * Serving pages to command <pip command> * Updated code to handle formatting error * added trigger words file and updated code * corrected few errors * New Flask Fathead (duckduckgo#459) * Added flask fathead * Added flask fathead * Parsed API documentation Flask * Escaped html symbols and newlines * Escaped html symbols and newlines * Fixed HTML class for closing quote * Fixed HTML class for closing quote * Fixes duckduckgo#751 Improves Redirects * Fixes duckduckgo#751 Changes done * Javascript: New Electron Fathead (duckduckgo#721) * First commit * Truncated fetch.sh * Scraping Home Page and indentation fixed * Bugs fixed * Final changes * First commit - Electron branch * Removed unnecessary data * Modified code * Indentation fixed * Final changes * Fixes duckduckgo#757 Updates docs and directory structure * Fixes duckduckgo#757: Some final clean ups * Added trigger_words.txt * added angular javascript trigger along with angular js :duckduckgo#755 * changes done as requested * updated parse.sh and readme * Fixed minor indentation issue. * Sort and make sure entries are unique. * Fix issue with parsing the newest docs. * Apparently, the docs could not be parsed anymore. This was solved by checking that the elements used were really bs4.element.Tag elements and actually had the .name etc properties. * Java: New Java Fathead (Continue duckduckgo#642) (duckduckgo#774) * Added method coverage data for the Java fathead * Added methods to output.txt * Removed classname from several functions * Fixed formatting for output and coverage * Added classesto methods in methods.txt and updated file * Coverage data and output are sorted if they exist * Added sorted methods.txt * Stripped methods of brackets and parameters * Added sorted version of methods.txt * Added a space between method description and url * Added formatted method parameters to methods.txt and output.txt * Added method output and coverage without parameters * Changed method_url find parameter from "href" to "a" * Removed test_parse.py due to it being a redundant test * Added tests for checking the output is correct in output.txt and method coverage * Removed pipe to uniq -u and added sort -u instead * Removed test cases for method parameter formatting * Fixed spacing in method coverage so that sort -u recognizes duplicates * Removed output.txt from .gitignore * Added -f option to sort in parse.sh * Excluded methods without parameters from being appended twice to output.txt * Fixed method_url parsing and reverted method extraction function * Fixed formatting issues on cover and output so that sort -u recognizes duplicates * Added <section> to method descriptions and an additional tab * Remove output.txt after beta upload * Python: New PEP Fathead (duckduckgo#627) * Added PEP Fathead. * Removed data.url file. * Tab replacement now 4 spaces. * Updated trigger words. * Adding output file. * Updated title. * Updated with redirects. * Removed meta.txt file. * Removed 'PEP' trigger word. * Updated parser from review. * Update Parser and redirects. * Updating output.txt file for beta server. * Updated output.txt * Updated Parser for new lines. * Updated parser. * Patched Parser flush bug. * Fixes spacing issue. * Refactored parser. * Updated parser to include newlines in code blocks. * Updated parser. * Updated parser to handle <p> * JavaScript: New Vuejs Fathead (duckduckgo#776) * Initial commit * added trigger words.txt * added more redirects * fixed redirect function * fixed abstract title * fixed formatting of code in abstract * fixed trigger words, changed order of abstraction * Removed unnecessary redirects * Python Coverage Fathead (duckduckgo#738) * inital commit to repo * fixes incorrect file write * adds trigger words * updates README to install requirements * corrects inconsistency of method docstrings * removes blank line in triggerwords, adds output.txt * removes minor version spec from pip install doc line * removes original site URL and text * changes html cleanup to preserve some tags * removes debugging print line, swaps <cite> tags with <code> tags * Ocaml: New OPAM fathead. (duckduckgo#775) * Updated stale OPAM fathead. * Added trigger words. * Updated parser to deal with version. * Updated parser. Updated output.txt * npm CLI: Added Categories (duckduckgo#754) * Added list of commands for npm cli * Removed npm.txt * Updated from npm-cli.txt * Delete npm-cli.txt * Revert "Swift Docs: Readme updated and url changed from 2.2 to 3.1" (duckduckgo#790)
moollaza
pushed a commit
that referenced
this pull request
Mar 8, 2017
* adding debugging with some class reading changes * library update for php parsing * updating code for enable/disable debugging * fix for repeated functions ⛳ * updated parsers, also updated readme, easy debugging some whitespace changes some more and more and more whitespace changes some more and more and more whitespace changes some more and more and more whitespace changes adding more coverage to the php functions * adding debugging with some class reading changes , updated readme new parser with improved coverage removed some code that was not in use whitepsace corrections in code a small indentation fix arrrhhhvi parse.js a whitespace * added colored console, disabled debugging * more coverage for "function" keyword * fixed the urls for more info * adding classes as categories * output.txt generated with results * Merging (#2) * Removed Django_built_ins (#677) * adding more coverage to the php functions (#678) * adding debugging with some class reading changes * library update for php parsing * updating code for enable/disable debugging * fix for repeated functions ⛳ * updated parsers, also updated readme, easy debugging some whitespace changes some more and more and more whitespace changes some more and more and more whitespace changes some more and more and more whitespace changes adding more coverage to the php functions * adding debugging with some class reading changes , updated readme new parser with improved coverage removed some code that was not in use whitepsace corrections in code a small indentation fix arrrhhhvi parse.js a whitespace * added colored console, disabled debugging * more coverage for "function" keyword * fixed the urls for more info * adding classes as categories * output.txt generated with results * Update PULL_REQUEST_TEMPLATE.md * Jquery: Add article aliases(redirects) for append,after,before (#610) (#679) * Add append,after,before redirects (#610) * Add append,after,before redirects (Fixed Pull request issues) (#610) * Ending period on disambiguation should be optional * jQuery Fathead: Add arguments type & description, Fixes #319 (#682) * Add argument desc * Completed arg descriptions * Exclude xi:include argsDescription * Remove whitespace * Fix args desc parse issue * Change argsdesc from code to unordered list * Format arguments name & type * Improove content formatting * Update output.txt * Final changes * Fix sub-heading and remove ill formatted redirects * Remove whitespace * Perl 6 Documentation: Fix the parser so that it generates correct links (#680) * Perl6: Use https instead of http * Perl6: Fix broken links * Perl6: Update output.txt * New SQLAlchemy Fathead Fixes #462 (#483) * New fathead for SQLAlchemy Fixes #462 * corrected unwanted redundancy in output.txt * Corrected improperly escaped '\n's * Added IAs for sqlalchemy events, internals and exceptions without disambiguation entries * Added disambiguation entries and handled a few corner cases * removed unnecessary '\t's, changed indentation and removed inline styles * Adding examples to all the topics of numpy. (#688) * Added Examples Template * Added examples for all topics of numpy * Typo fixed * Removed Extra spans from examples * using highlight-python to get examples * Remove extra new line characters and correct docmentation * Removed spaces * JavaScript: New NPM CLI Fathead (#694) * First commit * Truncated fetch.sh * Scraping Home Page and indentation fixed * Bugs fixed * Final changes * First commit * Removed unnecessary data * Indentation fixed and redirects added * Final changes * Order of desc and code changed * Changes made * STY: Fix PEP8 issues * Remove incorrect comment * Fix escaping issues in numpy fathead examples. * Perl 6 Documentation: Fix parse.pl so that it can handle the latest html structure. (#726) * Perl 6 Documentation: Modify the way to parse so that it can handle the latest (i.e. 2017.01) html structure * Perl 6 Documentation: Update output.txt corresponding with parse.pl's one * Perl 6 Documentation: Fix code blocks so that they don't contain escaped characters * Perl 6 Documentation: Update output.txt corresponding with parse.pl's one * C++ Fathead: Add language keywords (#626) * Initial Commit: Keywords Added * Fixed misspell * File Name changed * Removed Blanked lines * add "boost" trigger word * Java: New JMeter Fathead (#700) * Fixing conflicts * Code review and adding trigger file * Formatting updates * jmeter fathead inital commit * test commit to trigger build * Updating directory name for dax to pickup * Code review updates * Updating triggers * Wrapping up details inside <p> tags * JavaScript: New ESLint Fathead (#690) * Push initial version. * Test fetch + parse - Add output.txt - Tune download folder structure. * Code Add parse.sh * >Code: Process all categories instead of one. - Apply forEach on categories. - Polish tests. * Documentation: References, Intro, Testing. * Doc: Move notes to readme.md. * Test: Update current output.txt. * Infrastructure: Rename 'javascript_eslint/' to 'eslint/'. * Infrastructure: Adapt package.json - Name. * Add TXT for trigger words. * Add TXT for trigger words. * Code: Abstract layout - Add markdown-js. - Test inline code. * Testing: Run fetch + parse. * Infrastructure: 'parse.h' - Add **npm install** before calling 'parse.js'. * Update trigger_words.txt Add common search quotes. * +x permissions to fetch and parse sh * Code: Format summary text: - Start with uppercase letter. - End with dot. * Cosmetics: Code analysis. * Run fetch + parse. * JavaScript: New react native fathead (#713) * Download docs for the current stable release. * Parsing of docs * Module title and intro text. * Identifying prop sections and method sections. * Property signature, paragraph and url. * Methods section parsed. * Output result to output file. * Add parse.sh and output_statistics.awk. * Add output.txt for testing on beta. * Fix issues listed in review * Methods or Props with no information other than the signature should not add an empty <p></p> to output. * Method names should not be classified in classes. The module name is the class. * Fix erroneous prop_name * Some of the documentations pages are not of expected format. * Really sort on first field. * Handle disambiguations. * Add redirect.py that generates disambiguations. * Add requirements.txt that specifies requirements for this fathead. * Javascript: New Webpack Fathead (#720) * first steps of the fetching script * getting all the hrefs * cleaning up the file, and adding some comments * a working fetch script * placing it all under the same dir * rename the package, and adding some info * adding readme * adding node version * a very first beginning of parse.js * fixing node version dependency * adding a new line to the end of each entry * adding parse.sh to parse all the files in the download dir * bugfix: not trying to iterate when there are no more tags * small cleanup * adding counter * escaping strings * fixing a bug in the escaping * loading the pages with the original urls (and not file://) * fixing counter zero-based bug * refactoring: increasing readability * bugfix: not escaping links inside paragraphs * adding output.txt * removing empty lines * strip out a tags * Python: New Arrow Fathead (#669) (#723) * New Arrow Fathead (#669) * Fixed wrong formatting of abstract in output.txt The paragraph and code are supposed to be on the same level * Python: New Scipy fathead (#731) * scipy: Add fetcher scripts. * scipy: parser for scipy fathead. * scipy: Add parser runner * scipy: Add parser python requirements * scipy: Add a README file * scipy: Add trigger words file * scipy: Add fathead output * adding redirector script, and making redirect for . seperated values (#743) * JavaScript: New Lodash Fathead (#719) * Initial Commit * second commit * nothing huge * nothing major * nothing major * fixed fetch.sh * Nothing major * nothing much * added some redirects * fixed redirects * added more redirects * added methods.txt, some redirects, and formatted the code a bit * one /t was missing * updated txt with an extra tab * added formatting for abstract * removed extra /n in abstract formatting * Changed summary from text to html to count it <br> in <p> * stripped anchor tags * Added <code> tags in anchor tags title * fixed arguments and trigger words.txt * Scala: New Scala Docs Fathead (#704) * Initial Commit Add a new Scala Documentation fathead. Resolves #692 * Remove Debug Statements * Add requirements.txt * Add output.txt File * Fix minor issues * ADd triggers.txt * Add formatting to answers * Add recursive parsing of lists * Fix issue with inline code tag * sqlalchemy: Added requirements.txt and changed headings in output.txt (#703) * Added requirements.txt and changed header's <span> to <pre><code> * Corrected paragraph,list and table displays * MDN CSS: Add missing documentation articles (#702) * Rename --* to its proper name custom properties (--*). * Sort tags.txt. * Remove parameter from scale3d(). * Remove :full-screen. There is :fullscreen already. * Add creating of more redirects. Use a variant of title found in <code> part of description paragraph to create redirects. * Handle more than one alternate titles. * Make writing of redirects lazy. Write redirects only after all the articles have been written so that all redirects that have the same title as an article are dropped. The logic being that an article has a higher precedence than a redirect and thus a redirect should not prevent an article from being written. Thus, redirects get to use a different hash from the articles hash when keeping track of titles. * Check for duplication in create_article function. This ensures whenever create_article is called outside the parse loop we do not end up with duplicate titles in output.txt. * Fix comment indentation. * Add parsing of Values and Units. * Create redirects for all versions of :not(). * Remove !important. It is missing in https://developer.mozilla.org/en-US/docs/Web/CSS/Reference * tree-cell is not covered by Mozilla CSS reference. * Fix custom-ident typo. * Lowercase title in create_article. In some cases the title is not lowercased beforehand. * Replace <indent> with ident which is covered. * Add parsing of units for @font-feature-values. * Use exact element name to create link if it is present. This makes the article link scroll down to the exact element in the page more accurately than using the parent id. * No need for redirects for titles with ' * Appearance does not appear on mdn css reference. * Background-position-{x, y} are not on mdn css reference. * Bleed is not no mdn css reference. * border-start is not in mdn css reference. * Fix contrast typo. * fit is not covered in mdn css reference. * font-height is not covered by mdn css reference. * format is not covered by mdn css reference. * fr is not covered by mdn css reference. * grid-span is not covered by mdn css reference. * mark is not covered by mdn css reference. * Fix scroll-behavior typo. * SkewZ() is not covered by mdn css reference. * wrap-mode is not covered by mdn css reference. * Stop Mojo::IOLoop gracefully. See http://mojolicious.org/perldoc/Mojo/IOLoop#stop_gracefully * Delete keywords requiring extra processing or with 404 pages. See https://gist.github.com/hchienjo/c61adddbe63641a7b3c9de20a9995422 * Add output.txt. * Remove parsing of values data. This messes up with some of the articles as the titles collide. * Fetch only links in /docs/Web/CSS/ path. * Limit accepted alternate titles. * Parsing of #Feature_value_blocks is not harmful. * Parse values for font-variant-alternate. This is because it contains entries that are in tags.txt like: 1. swash() 2. ornaments() 3. annotation() * Remove "using css" from titles. * <counter> actually points to https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Lists_and_Counters/Using_CSS_counters * <flex> is converedd by flex. * mozilla.org/en-US/docs/Web/CSS/ident leads to a 404 error. * Add output.txt. * tighten rules on alternate titles. * Drop incorrect *.warning redirect. * Remove useless comment. * Limit creation of alternate titles for redirects. * Add output.txt. * No css snippet should be a redirect entry title. * Add output.txt. * Ensure removing brackets from titles preserves meaning. For instance :-moz-locale-dir(ltr) should be :-moz-locale-dir ltr and not :-moz-locale-dirltr. * Add output.txt. * New Six Library Fathead #453 (#532) * added Readme and fetch * added parse file * added class to store info about python_six lib * added class to parse the python six library content and added output.txt file * added new template and fixed issues * updated output.txt * added trigger words * if code section is not present then skip that in the abstraction * fixed issue of newline in abstarction * fixed indention issue * renamed the directory to match the id * change the trigger words * New Python Pip Fathead (#465) * parser,fetch and requirements added * Prelim pip fathead completed * Made required changes * SOme changes to parser * Fixed import file error. Also reduced output size to display only relevant information * Fixed some parsing errors * Serving pages to command <pip command> * Updated code to handle formatting error * added trigger words file and updated code * corrected few errors * New Flask Fathead (#459) * Added flask fathead * Added flask fathead * Parsed API documentation Flask * Escaped html symbols and newlines * Escaped html symbols and newlines * Fixed HTML class for closing quote * Fixed HTML class for closing quote * Fixes #751 Improves Redirects * Fixes #751 Changes done * Javascript: New Electron Fathead (#721) * First commit * Truncated fetch.sh * Scraping Home Page and indentation fixed * Bugs fixed * Final changes * First commit - Electron branch * Removed unnecessary data * Modified code * Indentation fixed * Final changes * Fixes #757 Updates docs and directory structure * Fixes #757: Some final clean ups * Added trigger_words.txt * added angular javascript trigger along with angular js :#755 * changes done as requested * updated parse.sh and readme * Fixed minor indentation issue. * Sort and make sure entries are unique. * Fix issue with parsing the newest docs. * Apparently, the docs could not be parsed anymore. This was solved by checking that the elements used were really bs4.element.Tag elements and actually had the .name etc properties. * Java: New Java Fathead (Continue #642) (#774) * Added method coverage data for the Java fathead * Added methods to output.txt * Removed classname from several functions * Fixed formatting for output and coverage * Added classesto methods in methods.txt and updated file * Coverage data and output are sorted if they exist * Added sorted methods.txt * Stripped methods of brackets and parameters * Added sorted version of methods.txt * Added a space between method description and url * Added formatted method parameters to methods.txt and output.txt * Added method output and coverage without parameters * Changed method_url find parameter from "href" to "a" * Removed test_parse.py due to it being a redundant test * Added tests for checking the output is correct in output.txt and method coverage * Removed pipe to uniq -u and added sort -u instead * Removed test cases for method parameter formatting * Fixed spacing in method coverage so that sort -u recognizes duplicates * Removed output.txt from .gitignore * Added -f option to sort in parse.sh * Excluded methods without parameters from being appended twice to output.txt * Fixed method_url parsing and reverted method extraction function * Fixed formatting issues on cover and output so that sort -u recognizes duplicates * Added <section> to method descriptions and an additional tab * Remove output.txt after beta upload * Python: New PEP Fathead (#627) * Added PEP Fathead. * Removed data.url file. * Tab replacement now 4 spaces. * Updated trigger words. * Adding output file. * Updated title. * Updated with redirects. * Removed meta.txt file. * Removed 'PEP' trigger word. * Updated parser from review. * Update Parser and redirects. * Updating output.txt file for beta server. * Updated output.txt * Updated Parser for new lines. * Updated parser. * Patched Parser flush bug. * Fixes spacing issue. * Refactored parser. * Updated parser to include newlines in code blocks. * Updated parser. * Updated parser to handle <p> * JavaScript: New Vuejs Fathead (#776) * Initial commit * added trigger words.txt * added more redirects * fixed redirect function * fixed abstract title * fixed formatting of code in abstract * fixed trigger words, changed order of abstraction * Removed unnecessary redirects * Python Coverage Fathead (#738) * inital commit to repo * fixes incorrect file write * adds trigger words * updates README to install requirements * corrects inconsistency of method docstrings * removes blank line in triggerwords, adds output.txt * removes minor version spec from pip install doc line * removes original site URL and text * changes html cleanup to preserve some tags * removes debugging print line, swaps <cite> tags with <code> tags * Ocaml: New OPAM fathead. (#775) * Updated stale OPAM fathead. * Added trigger words. * Updated parser to deal with version. * Updated parser. Updated output.txt * npm CLI: Added Categories (#754) * Added list of commands for npm cli * Removed npm.txt * Updated from npm-cli.txt * Delete npm-cli.txt * Revert "Swift Docs: Readme updated and url changed from 2.2 to 3.1" (#790) * fetching the API from docs * created fathead as required [meteor] * Yo! (#3) * Removed Django_built_ins (#677) * adding more coverage to the php functions (#678) * adding debugging with some class reading changes * library update for php parsing * updating code for enable/disable debugging * fix for repeated functions ⛳ * updated parsers, also updated readme, easy debugging some whitespace changes some more and more and more whitespace changes some more and more and more whitespace changes some more and more and more whitespace changes adding more coverage to the php functions * adding debugging with some class reading changes , updated readme new parser with improved coverage removed some code that was not in use whitepsace corrections in code a small indentation fix arrrhhhvi parse.js a whitespace * added colored console, disabled debugging * more coverage for "function" keyword * fixed the urls for more info * adding classes as categories * output.txt generated with results * Update PULL_REQUEST_TEMPLATE.md * Jquery: Add article aliases(redirects) for append,after,before (#610) (#679) * Add append,after,before redirects (#610) * Add append,after,before redirects (Fixed Pull request issues) (#610) * Ending period on disambiguation should be optional * jQuery Fathead: Add arguments type & description, Fixes #319 (#682) * Add argument desc * Completed arg descriptions * Exclude xi:include argsDescription * Remove whitespace * Fix args desc parse issue * Change argsdesc from code to unordered list * Format arguments name & type * Improove content formatting * Update output.txt * Final changes * Fix sub-heading and remove ill formatted redirects * Remove whitespace * Perl 6 Documentation: Fix the parser so that it generates correct links (#680) * Perl6: Use https instead of http * Perl6: Fix broken links * Perl6: Update output.txt * New SQLAlchemy Fathead Fixes #462 (#483) * New fathead for SQLAlchemy Fixes #462 * corrected unwanted redundancy in output.txt * Corrected improperly escaped '\n's * Added IAs for sqlalchemy events, internals and exceptions without disambiguation entries * Added disambiguation entries and handled a few corner cases * removed unnecessary '\t's, changed indentation and removed inline styles * Adding examples to all the topics of numpy. (#688) * Added Examples Template * Added examples for all topics of numpy * Typo fixed * Removed Extra spans from examples * using highlight-python to get examples * Remove extra new line characters and correct docmentation * Removed spaces * JavaScript: New NPM CLI Fathead (#694) * First commit * Truncated fetch.sh * Scraping Home Page and indentation fixed * Bugs fixed * Final changes * First commit * Removed unnecessary data * Indentation fixed and redirects added * Final changes * Order of desc and code changed * Changes made * STY: Fix PEP8 issues * Remove incorrect comment * Fix escaping issues in numpy fathead examples. * Perl 6 Documentation: Fix parse.pl so that it can handle the latest html structure. (#726) * Perl 6 Documentation: Modify the way to parse so that it can handle the latest (i.e. 2017.01) html structure * Perl 6 Documentation: Update output.txt corresponding with parse.pl's one * Perl 6 Documentation: Fix code blocks so that they don't contain escaped characters * Perl 6 Documentation: Update output.txt corresponding with parse.pl's one * C++ Fathead: Add language keywords (#626) * Initial Commit: Keywords Added * Fixed misspell * File Name changed * Removed Blanked lines * add "boost" trigger word * Java: New JMeter Fathead (#700) * Fixing conflicts * Code review and adding trigger file * Formatting updates * jmeter fathead inital commit * test commit to trigger build * Updating directory name for dax to pickup * Code review updates * Updating triggers * Wrapping up details inside <p> tags * JavaScript: New ESLint Fathead (#690) * Push initial version. * Test fetch + parse - Add output.txt - Tune download folder structure. * Code Add parse.sh * >Code: Process all categories instead of one. - Apply forEach on categories. - Polish tests. * Documentation: References, Intro, Testing. * Doc: Move notes to readme.md. * Test: Update current output.txt. * Infrastructure: Rename 'javascript_eslint/' to 'eslint/'. * Infrastructure: Adapt package.json - Name. * Add TXT for trigger words. * Add TXT for trigger words. * Code: Abstract layout - Add markdown-js. - Test inline code. * Testing: Run fetch + parse. * Infrastructure: 'parse.h' - Add **npm install** before calling 'parse.js'. * Update trigger_words.txt Add common search quotes. * +x permissions to fetch and parse sh * Code: Format summary text: - Start with uppercase letter. - End with dot. * Cosmetics: Code analysis. * Run fetch + parse. * JavaScript: New react native fathead (#713) * Download docs for the current stable release. * Parsing of docs * Module title and intro text. * Identifying prop sections and method sections. * Property signature, paragraph and url. * Methods section parsed. * Output result to output file. * Add parse.sh and output_statistics.awk. * Add output.txt for testing on beta. * Fix issues listed in review * Methods or Props with no information other than the signature should not add an empty <p></p> to output. * Method names should not be classified in classes. The module name is the class. * Fix erroneous prop_name * Some of the documentations pages are not of expected format. * Really sort on first field. * Handle disambiguations. * Add redirect.py that generates disambiguations. * Add requirements.txt that specifies requirements for this fathead. * Javascript: New Webpack Fathead (#720) * first steps of the fetching script * getting all the hrefs * cleaning up the file, and adding some comments * a working fetch script * placing it all under the same dir * rename the package, and adding some info * adding readme * adding node version * a very first beginning of parse.js * fixing node version dependency * adding a new line to the end of each entry * adding parse.sh to parse all the files in the download dir * bugfix: not trying to iterate when there are no more tags * small cleanup * adding counter * escaping strings * fixing a bug in the escaping * loading the pages with the original urls (and not file://) * fixing counter zero-based bug * refactoring: increasing readability * bugfix: not escaping links inside paragraphs * adding output.txt * removing empty lines * strip out a tags * Python: New Arrow Fathead (#669) (#723) * New Arrow Fathead (#669) * Fixed wrong formatting of abstract in output.txt The paragraph and code are supposed to be on the same level * Python: New Scipy fathead (#731) * scipy: Add fetcher scripts. * scipy: parser for scipy fathead. * scipy: Add parser runner * scipy: Add parser python requirements * scipy: Add a README file * scipy: Add trigger words file * scipy: Add fathead output * adding redirector script, and making redirect for . seperated values (#743) * JavaScript: New Lodash Fathead (#719) * Initial Commit * second commit * nothing huge * nothing major * nothing major * fixed fetch.sh * Nothing major * nothing much * added some redirects * fixed redirects * added more redirects * added methods.txt, some redirects, and formatted the code a bit * one /t was missing * updated txt with an extra tab * added formatting for abstract * removed extra /n in abstract formatting * Changed summary from text to html to count it <br> in <p> * stripped anchor tags * Added <code> tags in anchor tags title * fixed arguments and trigger words.txt * Scala: New Scala Docs Fathead (#704) * Initial Commit Add a new Scala Documentation fathead. Resolves #692 * Remove Debug Statements * Add requirements.txt * Add output.txt File * Fix minor issues * ADd triggers.txt * Add formatting to answers * Add recursive parsing of lists * Fix issue with inline code tag * sqlalchemy: Added requirements.txt and changed headings in output.txt (#703) * Added requirements.txt and changed header's <span> to <pre><code> * Corrected paragraph,list and table displays * MDN CSS: Add missing documentation articles (#702) * Rename --* to its proper name custom properties (--*). * Sort tags.txt. * Remove parameter from scale3d(). * Remove :full-screen. There is :fullscreen already. * Add creating of more redirects. Use a variant of title found in <code> part of description paragraph to create redirects. * Handle more than one alternate titles. * Make writing of redirects lazy. Write redirects only after all the articles have been written so that all redirects that have the same title as an article are dropped. The logic being that an article has a higher precedence than a redirect and thus a redirect should not prevent an article from being written. Thus, redirects get to use a different hash from the articles hash when keeping track of titles. * Check for duplication in create_article function. This ensures whenever create_article is called outside the parse loop we do not end up with duplicate titles in output.txt. * Fix comment indentation. * Add parsing of Values and Units. * Create redirects for all versions of :not(). * Remove !important. It is missing in https://developer.mozilla.org/en-US/docs/Web/CSS/Reference * tree-cell is not covered by Mozilla CSS reference. * Fix custom-ident typo. * Lowercase title in create_article. In some cases the title is not lowercased beforehand. * Replace <indent> with ident which is covered. * Add parsing of units for @font-feature-values. * Use exact element name to create link if it is present. This makes the article link scroll down to the exact element in the page more accurately than using the parent id. * No need for redirects for titles with ' * Appearance does not appear on mdn css reference. * Background-position-{x, y} are not on mdn css reference. * Bleed is not no mdn css reference. * border-start is not in mdn css reference. * Fix contrast typo. * fit is not covered in mdn css reference. * font-height is not covered by mdn css reference. * format is not covered by mdn css reference. * fr is not covered by mdn css reference. * grid-span is not covered by mdn css reference. * mark is not covered by mdn css reference. * Fix scroll-behavior typo. * SkewZ() is not covered by mdn css reference. * wrap-mode is not covered by mdn css reference. * Stop Mojo::IOLoop gracefully. See http://mojolicious.org/perldoc/Mojo/IOLoop#stop_gracefully * Delete keywords requiring extra processing or with 404 pages. See https://gist.github.com/hchienjo/c61adddbe63641a7b3c9de20a9995422 * Add output.txt. * Remove parsing of values data. This messes up with some of the articles as the titles collide. * Fetch only links in /docs/Web/CSS/ path. * Limit accepted alternate titles. * Parsing of #Feature_value_blocks is not harmful. * Parse values for font-variant-alternate. This is because it contains entries that are in tags.txt like: 1. swash() 2. ornaments() 3. annotation() * Remove "using css" from titles. * <counter> actually points to https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Lists_and_Counters/Using_CSS_counters * <flex> is converedd by flex. * mozilla.org/en-US/docs/Web/CSS/ident leads to a 404 error. * Add output.txt. * tighten rules on alternate titles. * Drop incorrect *.warning redirect. * Remove useless comment. * Limit creation of alternate titles for redirects. * Add output.txt. * No css snippet should be a redirect entry title. * Add output.txt. * Ensure removing brackets from titles preserves meaning. For instance :-moz-locale-dir(ltr) should be :-moz-locale-dir ltr and not :-moz-locale-dirltr. * Add output.txt. * New Six Library Fathead #453 (#532) * added Readme and fetch * added parse file * added class to store info about python_six lib * added class to parse the python six library content and added output.txt file * added new template and fixed issues * updated output.txt * added trigger words * if code section is not present then skip that in the abstraction * fixed issue of newline in abstarction * fixed indention issue * renamed the directory to match the id * change the trigger words * New Python Pip Fathead (#465) * parser,fetch and requirements added * Prelim pip fathead completed * Made required changes * SOme changes to parser * Fixed import file error. Also reduced output size to display only relevant information * Fixed some parsing errors * Serving pages to command <pip command> * Updated code to handle formatting error * added trigger words file and updated code * corrected few errors * New Flask Fathead (#459) * Added flask fathead * Added flask fathead * Parsed API documentation Flask * Escaped html symbols and newlines * Escaped html symbols and newlines * Fixed HTML class for closing quote * Fixed HTML class for closing quote * Fixes #751 Improves Redirects * Fixes #751 Changes done * Javascript: New Electron Fathead (#721) * First commit * Truncated fetch.sh * Scraping Home Page and indentation fixed * Bugs fixed * Final changes * First commit - Electron branch * Removed unnecessary data * Modified code * Indentation fixed * Final changes * Fixes #757 Updates docs and directory structure * Fixes #757: Some final clean ups * Added trigger_words.txt * added angular javascript trigger along with angular js :#755 * changes done as requested * updated parse.sh and readme * Fixed minor indentation issue. * Sort and make sure entries are unique. * Fix issue with parsing the newest docs. * Apparently, the docs could not be parsed anymore. This was solved by checking that the elements used were really bs4.element.Tag elements and actually had the .name etc properties. * Java: New Java Fathead (Continue #642) (#774) * Added method coverage data for the Java fathead * Added methods to output.txt * Removed classname from several functions * Fixed formatting for output and coverage * Added classesto methods in methods.txt and updated file * Coverage data and output are sorted if they exist * Added sorted methods.txt * Stripped methods of brackets and parameters * Added sorted version of methods.txt * Added a space between method description and url * Added formatted method parameters to methods.txt and output.txt * Added method output and coverage without parameters * Changed method_url find parameter from "href" to "a" * Removed test_parse.py due to it being a redundant test * Added tests for checking the output is correct in output.txt and method coverage * Removed pipe to uniq -u and added sort -u instead * Removed test cases for method parameter formatting * Fixed spacing in method coverage so that sort -u recognizes duplicates * Removed output.txt from .gitignore * Added -f option to sort in parse.sh * Excluded methods without parameters from being appended twice to output.txt * Fixed method_url parsing and reverted method extraction function * Fixed formatting issues on cover and output so that sort -u recognizes duplicates * Added <section> to method descriptions and an additional tab * Remove output.txt after beta upload * Python: New PEP Fathead (#627) * Added PEP Fathead. * Removed data.url file. * Tab replacement now 4 spaces. * Updated trigger words. * Adding output file. * Updated title. * Updated with redirects. * Removed meta.txt file. * Removed 'PEP' trigger word. * Updated parser from review. * Update Parser and redirects. * Updating output.txt file for beta server. * Updated output.txt * Updated Parser for new lines. * Updated parser. * Patched Parser flush bug. * Fixes spacing issue. * Refactored parser. * Updated parser to include newlines in code blocks. * Updated parser. * Updated parser to handle <p> * JavaScript: New Vuejs Fathead (#776) * Initial commit * added trigger words.txt * added more redirects * fixed redirect function * fixed abstract title * fixed formatting of code in abstract * fixed trigger words, changed order of abstraction * Removed unnecessary redirects * Python Coverage Fathead (#738) * inital commit to repo * fixes incorrect file write * adds trigger words * updates README to install requirements * corrects inconsistency of method docstrings * removes blank line in triggerwords, adds output.txt * removes minor version spec from pip install doc line * removes original site URL and text * changes html cleanup to preserve some tags * removes debugging print line, swaps <cite> tags with <code> tags * Ocaml: New OPAM fathead. (#775) * Updated stale OPAM fathead. * Added trigger words. * Updated parser to deal with version. * Updated parser. Updated output.txt * npm CLI: Added Categories (#754) * Added list of commands for npm cli * Removed npm.txt * Updated from npm-cli.txt * Delete npm-cli.txt * Revert "Swift Docs: Readme updated and url changed from 2.2 to 3.1" (#790) * better code, fixed documentation link
danidee10
pushed a commit
to danidee10/zeroclickinfo-fathead
that referenced
this pull request
Mar 30, 2017
* adding debugging with some class reading changes * library update for php parsing * updating code for enable/disable debugging * fix for repeated functions ⛳ * updated parsers, also updated readme, easy debugging some whitespace changes some more and more and more whitespace changes some more and more and more whitespace changes some more and more and more whitespace changes adding more coverage to the php functions * adding debugging with some class reading changes , updated readme new parser with improved coverage removed some code that was not in use whitepsace corrections in code a small indentation fix arrrhhhvi parse.js a whitespace * added colored console, disabled debugging * more coverage for "function" keyword * fixed the urls for more info * adding classes as categories * output.txt generated with results * Merging (duckduckgo#2) * Removed Django_built_ins (duckduckgo#677) * adding more coverage to the php functions (duckduckgo#678) * adding debugging with some class reading changes * library update for php parsing * updating code for enable/disable debugging * fix for repeated functions ⛳ * updated parsers, also updated readme, easy debugging some whitespace changes some more and more and more whitespace changes some more and more and more whitespace changes some more and more and more whitespace changes adding more coverage to the php functions * adding debugging with some class reading changes , updated readme new parser with improved coverage removed some code that was not in use whitepsace corrections in code a small indentation fix arrrhhhvi parse.js a whitespace * added colored console, disabled debugging * more coverage for "function" keyword * fixed the urls for more info * adding classes as categories * output.txt generated with results * Update PULL_REQUEST_TEMPLATE.md * Jquery: Add article aliases(redirects) for append,after,before (duckduckgo#610) (duckduckgo#679) * Add append,after,before redirects (duckduckgo#610) * Add append,after,before redirects (Fixed Pull request issues) (duckduckgo#610) * Ending period on disambiguation should be optional * jQuery Fathead: Add arguments type & description, Fixes duckduckgo#319 (duckduckgo#682) * Add argument desc * Completed arg descriptions * Exclude xi:include argsDescription * Remove whitespace * Fix args desc parse issue * Change argsdesc from code to unordered list * Format arguments name & type * Improove content formatting * Update output.txt * Final changes * Fix sub-heading and remove ill formatted redirects * Remove whitespace * Perl 6 Documentation: Fix the parser so that it generates correct links (duckduckgo#680) * Perl6: Use https instead of http * Perl6: Fix broken links * Perl6: Update output.txt * New SQLAlchemy Fathead Fixes duckduckgo#462 (duckduckgo#483) * New fathead for SQLAlchemy Fixes duckduckgo#462 * corrected unwanted redundancy in output.txt * Corrected improperly escaped '\n's * Added IAs for sqlalchemy events, internals and exceptions without disambiguation entries * Added disambiguation entries and handled a few corner cases * removed unnecessary '\t's, changed indentation and removed inline styles * Adding examples to all the topics of numpy. (duckduckgo#688) * Added Examples Template * Added examples for all topics of numpy * Typo fixed * Removed Extra spans from examples * using highlight-python to get examples * Remove extra new line characters and correct docmentation * Removed spaces * JavaScript: New NPM CLI Fathead (duckduckgo#694) * First commit * Truncated fetch.sh * Scraping Home Page and indentation fixed * Bugs fixed * Final changes * First commit * Removed unnecessary data * Indentation fixed and redirects added * Final changes * Order of desc and code changed * Changes made * STY: Fix PEP8 issues * Remove incorrect comment * Fix escaping issues in numpy fathead examples. * Perl 6 Documentation: Fix parse.pl so that it can handle the latest html structure. (duckduckgo#726) * Perl 6 Documentation: Modify the way to parse so that it can handle the latest (i.e. 2017.01) html structure * Perl 6 Documentation: Update output.txt corresponding with parse.pl's one * Perl 6 Documentation: Fix code blocks so that they don't contain escaped characters * Perl 6 Documentation: Update output.txt corresponding with parse.pl's one * C++ Fathead: Add language keywords (duckduckgo#626) * Initial Commit: Keywords Added * Fixed misspell * File Name changed * Removed Blanked lines * add "boost" trigger word * Java: New JMeter Fathead (duckduckgo#700) * Fixing conflicts * Code review and adding trigger file * Formatting updates * jmeter fathead inital commit * test commit to trigger build * Updating directory name for dax to pickup * Code review updates * Updating triggers * Wrapping up details inside <p> tags * JavaScript: New ESLint Fathead (duckduckgo#690) * Push initial version. * Test fetch + parse - Add output.txt - Tune download folder structure. * Code Add parse.sh * >Code: Process all categories instead of one. - Apply forEach on categories. - Polish tests. * Documentation: References, Intro, Testing. * Doc: Move notes to readme.md. * Test: Update current output.txt. * Infrastructure: Rename 'javascript_eslint/' to 'eslint/'. * Infrastructure: Adapt package.json - Name. * Add TXT for trigger words. * Add TXT for trigger words. * Code: Abstract layout - Add markdown-js. - Test inline code. * Testing: Run fetch + parse. * Infrastructure: 'parse.h' - Add **npm install** before calling 'parse.js'. * Update trigger_words.txt Add common search quotes. * +x permissions to fetch and parse sh * Code: Format summary text: - Start with uppercase letter. - End with dot. * Cosmetics: Code analysis. * Run fetch + parse. * JavaScript: New react native fathead (duckduckgo#713) * Download docs for the current stable release. * Parsing of docs * Module title and intro text. * Identifying prop sections and method sections. * Property signature, paragraph and url. * Methods section parsed. * Output result to output file. * Add parse.sh and output_statistics.awk. * Add output.txt for testing on beta. * Fix issues listed in review * Methods or Props with no information other than the signature should not add an empty <p></p> to output. * Method names should not be classified in classes. The module name is the class. * Fix erroneous prop_name * Some of the documentations pages are not of expected format. * Really sort on first field. * Handle disambiguations. * Add redirect.py that generates disambiguations. * Add requirements.txt that specifies requirements for this fathead. * Javascript: New Webpack Fathead (duckduckgo#720) * first steps of the fetching script * getting all the hrefs * cleaning up the file, and adding some comments * a working fetch script * placing it all under the same dir * rename the package, and adding some info * adding readme * adding node version * a very first beginning of parse.js * fixing node version dependency * adding a new line to the end of each entry * adding parse.sh to parse all the files in the download dir * bugfix: not trying to iterate when there are no more tags * small cleanup * adding counter * escaping strings * fixing a bug in the escaping * loading the pages with the original urls (and not file://) * fixing counter zero-based bug * refactoring: increasing readability * bugfix: not escaping links inside paragraphs * adding output.txt * removing empty lines * strip out a tags * Python: New Arrow Fathead (duckduckgo#669) (duckduckgo#723) * New Arrow Fathead (duckduckgo#669) * Fixed wrong formatting of abstract in output.txt The paragraph and code are supposed to be on the same level * Python: New Scipy fathead (duckduckgo#731) * scipy: Add fetcher scripts. * scipy: parser for scipy fathead. * scipy: Add parser runner * scipy: Add parser python requirements * scipy: Add a README file * scipy: Add trigger words file * scipy: Add fathead output * adding redirector script, and making redirect for . seperated values (duckduckgo#743) * JavaScript: New Lodash Fathead (duckduckgo#719) * Initial Commit * second commit * nothing huge * nothing major * nothing major * fixed fetch.sh * Nothing major * nothing much * added some redirects * fixed redirects * added more redirects * added methods.txt, some redirects, and formatted the code a bit * one /t was missing * updated txt with an extra tab * added formatting for abstract * removed extra /n in abstract formatting * Changed summary from text to html to count it <br> in <p> * stripped anchor tags * Added <code> tags in anchor tags title * fixed arguments and trigger words.txt * Scala: New Scala Docs Fathead (duckduckgo#704) * Initial Commit Add a new Scala Documentation fathead. Resolves duckduckgo#692 * Remove Debug Statements * Add requirements.txt * Add output.txt File * Fix minor issues * ADd triggers.txt * Add formatting to answers * Add recursive parsing of lists * Fix issue with inline code tag * sqlalchemy: Added requirements.txt and changed headings in output.txt (duckduckgo#703) * Added requirements.txt and changed header's <span> to <pre><code> * Corrected paragraph,list and table displays * MDN CSS: Add missing documentation articles (duckduckgo#702) * Rename --* to its proper name custom properties (--*). * Sort tags.txt. * Remove parameter from scale3d(). * Remove :full-screen. There is :fullscreen already. * Add creating of more redirects. Use a variant of title found in <code> part of description paragraph to create redirects. * Handle more than one alternate titles. * Make writing of redirects lazy. Write redirects only after all the articles have been written so that all redirects that have the same title as an article are dropped. The logic being that an article has a higher precedence than a redirect and thus a redirect should not prevent an article from being written. Thus, redirects get to use a different hash from the articles hash when keeping track of titles. * Check for duplication in create_article function. This ensures whenever create_article is called outside the parse loop we do not end up with duplicate titles in output.txt. * Fix comment indentation. * Add parsing of Values and Units. * Create redirects for all versions of :not(). * Remove !important. It is missing in https://developer.mozilla.org/en-US/docs/Web/CSS/Reference * tree-cell is not covered by Mozilla CSS reference. * Fix custom-ident typo. * Lowercase title in create_article. In some cases the title is not lowercased beforehand. * Replace <indent> with ident which is covered. * Add parsing of units for @font-feature-values. * Use exact element name to create link if it is present. This makes the article link scroll down to the exact element in the page more accurately than using the parent id. * No need for redirects for titles with ' * Appearance does not appear on mdn css reference. * Background-position-{x, y} are not on mdn css reference. * Bleed is not no mdn css reference. * border-start is not in mdn css reference. * Fix contrast typo. * fit is not covered in mdn css reference. * font-height is not covered by mdn css reference. * format is not covered by mdn css reference. * fr is not covered by mdn css reference. * grid-span is not covered by mdn css reference. * mark is not covered by mdn css reference. * Fix scroll-behavior typo. * SkewZ() is not covered by mdn css reference. * wrap-mode is not covered by mdn css reference. * Stop Mojo::IOLoop gracefully. See http://mojolicious.org/perldoc/Mojo/IOLoop#stop_gracefully * Delete keywords requiring extra processing or with 404 pages. See https://gist.github.com/hchienjo/c61adddbe63641a7b3c9de20a9995422 * Add output.txt. * Remove parsing of values data. This messes up with some of the articles as the titles collide. * Fetch only links in /docs/Web/CSS/ path. * Limit accepted alternate titles. * Parsing of #Feature_value_blocks is not harmful. * Parse values for font-variant-alternate. This is because it contains entries that are in tags.txt like: 1. swash() 2. ornaments() 3. annotation() * Remove "using css" from titles. * <counter> actually points to https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Lists_and_Counters/Using_CSS_counters * <flex> is converedd by flex. * mozilla.org/en-US/docs/Web/CSS/ident leads to a 404 error. * Add output.txt. * tighten rules on alternate titles. * Drop incorrect *.warning redirect. * Remove useless comment. * Limit creation of alternate titles for redirects. * Add output.txt. * No css snippet should be a redirect entry title. * Add output.txt. * Ensure removing brackets from titles preserves meaning. For instance :-moz-locale-dir(ltr) should be :-moz-locale-dir ltr and not :-moz-locale-dirltr. * Add output.txt. * New Six Library Fathead duckduckgo#453 (duckduckgo#532) * added Readme and fetch * added parse file * added class to store info about python_six lib * added class to parse the python six library content and added output.txt file * added new template and fixed issues * updated output.txt * added trigger words * if code section is not present then skip that in the abstraction * fixed issue of newline in abstarction * fixed indention issue * renamed the directory to match the id * change the trigger words * New Python Pip Fathead (duckduckgo#465) * parser,fetch and requirements added * Prelim pip fathead completed * Made required changes * SOme changes to parser * Fixed import file error. Also reduced output size to display only relevant information * Fixed some parsing errors * Serving pages to command <pip command> * Updated code to handle formatting error * added trigger words file and updated code * corrected few errors * New Flask Fathead (duckduckgo#459) * Added flask fathead * Added flask fathead * Parsed API documentation Flask * Escaped html symbols and newlines * Escaped html symbols and newlines * Fixed HTML class for closing quote * Fixed HTML class for closing quote * Fixes duckduckgo#751 Improves Redirects * Fixes duckduckgo#751 Changes done * Javascript: New Electron Fathead (duckduckgo#721) * First commit * Truncated fetch.sh * Scraping Home Page and indentation fixed * Bugs fixed * Final changes * First commit - Electron branch * Removed unnecessary data * Modified code * Indentation fixed * Final changes * Fixes duckduckgo#757 Updates docs and directory structure * Fixes duckduckgo#757: Some final clean ups * Added trigger_words.txt * added angular javascript trigger along with angular js :duckduckgo#755 * changes done as requested * updated parse.sh and readme * Fixed minor indentation issue. * Sort and make sure entries are unique. * Fix issue with parsing the newest docs. * Apparently, the docs could not be parsed anymore. This was solved by checking that the elements used were really bs4.element.Tag elements and actually had the .name etc properties. * Java: New Java Fathead (Continue duckduckgo#642) (duckduckgo#774) * Added method coverage data for the Java fathead * Added methods to output.txt * Removed classname from several functions * Fixed formatting for output and coverage * Added classesto methods in methods.txt and updated file * Coverage data and output are sorted if they exist * Added sorted methods.txt * Stripped methods of brackets and parameters * Added sorted version of methods.txt * Added a space between method description and url * Added formatted method parameters to methods.txt and output.txt * Added method output and coverage without parameters * Changed method_url find parameter from "href" to "a" * Removed test_parse.py due to it being a redundant test * Added tests for checking the output is correct in output.txt and method coverage * Removed pipe to uniq -u and added sort -u instead * Removed test cases for method parameter formatting * Fixed spacing in method coverage so that sort -u recognizes duplicates * Removed output.txt from .gitignore * Added -f option to sort in parse.sh * Excluded methods without parameters from being appended twice to output.txt * Fixed method_url parsing and reverted method extraction function * Fixed formatting issues on cover and output so that sort -u recognizes duplicates * Added <section> to method descriptions and an additional tab * Remove output.txt after beta upload * Python: New PEP Fathead (duckduckgo#627) * Added PEP Fathead. * Removed data.url file. * Tab replacement now 4 spaces. * Updated trigger words. * Adding output file. * Updated title. * Updated with redirects. * Removed meta.txt file. * Removed 'PEP' trigger word. * Updated parser from review. * Update Parser and redirects. * Updating output.txt file for beta server. * Updated output.txt * Updated Parser for new lines. * Updated parser. * Patched Parser flush bug. * Fixes spacing issue. * Refactored parser. * Updated parser to include newlines in code blocks. * Updated parser. * Updated parser to handle <p> * JavaScript: New Vuejs Fathead (duckduckgo#776) * Initial commit * added trigger words.txt * added more redirects * fixed redirect function * fixed abstract title * fixed formatting of code in abstract * fixed trigger words, changed order of abstraction * Removed unnecessary redirects * Python Coverage Fathead (duckduckgo#738) * inital commit to repo * fixes incorrect file write * adds trigger words * updates README to install requirements * corrects inconsistency of method docstrings * removes blank line in triggerwords, adds output.txt * removes minor version spec from pip install doc line * removes original site URL and text * changes html cleanup to preserve some tags * removes debugging print line, swaps <cite> tags with <code> tags * Ocaml: New OPAM fathead. (duckduckgo#775) * Updated stale OPAM fathead. * Added trigger words. * Updated parser to deal with version. * Updated parser. Updated output.txt * npm CLI: Added Categories (duckduckgo#754) * Added list of commands for npm cli * Removed npm.txt * Updated from npm-cli.txt * Delete npm-cli.txt * Revert "Swift Docs: Readme updated and url changed from 2.2 to 3.1" (duckduckgo#790) * fetching the API from docs * created fathead as required [meteor] * Yo! (duckduckgo#3) * Removed Django_built_ins (duckduckgo#677) * adding more coverage to the php functions (duckduckgo#678) * adding debugging with some class reading changes * library update for php parsing * updating code for enable/disable debugging * fix for repeated functions ⛳ * updated parsers, also updated readme, easy debugging some whitespace changes some more and more and more whitespace changes some more and more and more whitespace changes some more and more and more whitespace changes adding more coverage to the php functions * adding debugging with some class reading changes , updated readme new parser with improved coverage removed some code that was not in use whitepsace corrections in code a small indentation fix arrrhhhvi parse.js a whitespace * added colored console, disabled debugging * more coverage for "function" keyword * fixed the urls for more info * adding classes as categories * output.txt generated with results * Update PULL_REQUEST_TEMPLATE.md * Jquery: Add article aliases(redirects) for append,after,before (duckduckgo#610) (duckduckgo#679) * Add append,after,before redirects (duckduckgo#610) * Add append,after,before redirects (Fixed Pull request issues) (duckduckgo#610) * Ending period on disambiguation should be optional * jQuery Fathead: Add arguments type & description, Fixes duckduckgo#319 (duckduckgo#682) * Add argument desc * Completed arg descriptions * Exclude xi:include argsDescription * Remove whitespace * Fix args desc parse issue * Change argsdesc from code to unordered list * Format arguments name & type * Improove content formatting * Update output.txt * Final changes * Fix sub-heading and remove ill formatted redirects * Remove whitespace * Perl 6 Documentation: Fix the parser so that it generates correct links (duckduckgo#680) * Perl6: Use https instead of http * Perl6: Fix broken links * Perl6: Update output.txt * New SQLAlchemy Fathead Fixes duckduckgo#462 (duckduckgo#483) * New fathead for SQLAlchemy Fixes duckduckgo#462 * corrected unwanted redundancy in output.txt * Corrected improperly escaped '\n's * Added IAs for sqlalchemy events, internals and exceptions without disambiguation entries * Added disambiguation entries and handled a few corner cases * removed unnecessary '\t's, changed indentation and removed inline styles * Adding examples to all the topics of numpy. (duckduckgo#688) * Added Examples Template * Added examples for all topics of numpy * Typo fixed * Removed Extra spans from examples * using highlight-python to get examples * Remove extra new line characters and correct docmentation * Removed spaces * JavaScript: New NPM CLI Fathead (duckduckgo#694) * First commit * Truncated fetch.sh * Scraping Home Page and indentation fixed * Bugs fixed * Final changes * First commit * Removed unnecessary data * Indentation fixed and redirects added * Final changes * Order of desc and code changed * Changes made * STY: Fix PEP8 issues * Remove incorrect comment * Fix escaping issues in numpy fathead examples. * Perl 6 Documentation: Fix parse.pl so that it can handle the latest html structure. (duckduckgo#726) * Perl 6 Documentation: Modify the way to parse so that it can handle the latest (i.e. 2017.01) html structure * Perl 6 Documentation: Update output.txt corresponding with parse.pl's one * Perl 6 Documentation: Fix code blocks so that they don't contain escaped characters * Perl 6 Documentation: Update output.txt corresponding with parse.pl's one * C++ Fathead: Add language keywords (duckduckgo#626) * Initial Commit: Keywords Added * Fixed misspell * File Name changed * Removed Blanked lines * add "boost" trigger word * Java: New JMeter Fathead (duckduckgo#700) * Fixing conflicts * Code review and adding trigger file * Formatting updates * jmeter fathead inital commit * test commit to trigger build * Updating directory name for dax to pickup * Code review updates * Updating triggers * Wrapping up details inside <p> tags * JavaScript: New ESLint Fathead (duckduckgo#690) * Push initial version. * Test fetch + parse - Add output.txt - Tune download folder structure. * Code Add parse.sh * >Code: Process all categories instead of one. - Apply forEach on categories. - Polish tests. * Documentation: References, Intro, Testing. * Doc: Move notes to readme.md. * Test: Update current output.txt. * Infrastructure: Rename 'javascript_eslint/' to 'eslint/'. * Infrastructure: Adapt package.json - Name. * Add TXT for trigger words. * Add TXT for trigger words. * Code: Abstract layout - Add markdown-js. - Test inline code. * Testing: Run fetch + parse. * Infrastructure: 'parse.h' - Add **npm install** before calling 'parse.js'. * Update trigger_words.txt Add common search quotes. * +x permissions to fetch and parse sh * Code: Format summary text: - Start with uppercase letter. - End with dot. * Cosmetics: Code analysis. * Run fetch + parse. * JavaScript: New react native fathead (duckduckgo#713) * Download docs for the current stable release. * Parsing of docs * Module title and intro text. * Identifying prop sections and method sections. * Property signature, paragraph and url. * Methods section parsed. * Output result to output file. * Add parse.sh and output_statistics.awk. * Add output.txt for testing on beta. * Fix issues listed in review * Methods or Props with no information other than the signature should not add an empty <p></p> to output. * Method names should not be classified in classes. The module name is the class. * Fix erroneous prop_name * Some of the documentations pages are not of expected format. * Really sort on first field. * Handle disambiguations. * Add redirect.py that generates disambiguations. * Add requirements.txt that specifies requirements for this fathead. * Javascript: New Webpack Fathead (duckduckgo#720) * first steps of the fetching script * getting all the hrefs * cleaning up the file, and adding some comments * a working fetch script * placing it all under the same dir * rename the package, and adding some info * adding readme * adding node version * a very first beginning of parse.js * fixing node version dependency * adding a new line to the end of each entry * adding parse.sh to parse all the files in the download dir * bugfix: not trying to iterate when there are no more tags * small cleanup * adding counter * escaping strings * fixing a bug in the escaping * loading the pages with the original urls (and not file://) * fixing counter zero-based bug * refactoring: increasing readability * bugfix: not escaping links inside paragraphs * adding output.txt * removing empty lines * strip out a tags * Python: New Arrow Fathead (duckduckgo#669) (duckduckgo#723) * New Arrow Fathead (duckduckgo#669) * Fixed wrong formatting of abstract in output.txt The paragraph and code are supposed to be on the same level * Python: New Scipy fathead (duckduckgo#731) * scipy: Add fetcher scripts. * scipy: parser for scipy fathead. * scipy: Add parser runner * scipy: Add parser python requirements * scipy: Add a README file * scipy: Add trigger words file * scipy: Add fathead output * adding redirector script, and making redirect for . seperated values (duckduckgo#743) * JavaScript: New Lodash Fathead (duckduckgo#719) * Initial Commit * second commit * nothing huge * nothing major * nothing major * fixed fetch.sh * Nothing major * nothing much * added some redirects * fixed redirects * added more redirects * added methods.txt, some redirects, and formatted the code a bit * one /t was missing * updated txt with an extra tab * added formatting for abstract * removed extra /n in abstract formatting * Changed summary from text to html to count it <br> in <p> * stripped anchor tags * Added <code> tags in anchor tags title * fixed arguments and trigger words.txt * Scala: New Scala Docs Fathead (duckduckgo#704) * Initial Commit Add a new Scala Documentation fathead. Resolves duckduckgo#692 * Remove Debug Statements * Add requirements.txt * Add output.txt File * Fix minor issues * ADd triggers.txt * Add formatting to answers * Add recursive parsing of lists * Fix issue with inline code tag * sqlalchemy: Added requirements.txt and changed headings in output.txt (duckduckgo#703) * Added requirements.txt and changed header's <span> to <pre><code> * Corrected paragraph,list and table displays * MDN CSS: Add missing documentation articles (duckduckgo#702) * Rename --* to its proper name custom properties (--*). * Sort tags.txt. * Remove parameter from scale3d(). * Remove :full-screen. There is :fullscreen already. * Add creating of more redirects. Use a variant of title found in <code> part of description paragraph to create redirects. * Handle more than one alternate titles. * Make writing of redirects lazy. Write redirects only after all the articles have been written so that all redirects that have the same title as an article are dropped. The logic being that an article has a higher precedence than a redirect and thus a redirect should not prevent an article from being written. Thus, redirects get to use a different hash from the articles hash when keeping track of titles. * Check for duplication in create_article function. This ensures whenever create_article is called outside the parse loop we do not end up with duplicate titles in output.txt. * Fix comment indentation. * Add parsing of Values and Units. * Create redirects for all versions of :not(). * Remove !important. It is missing in https://developer.mozilla.org/en-US/docs/Web/CSS/Reference * tree-cell is not covered by Mozilla CSS reference. * Fix custom-ident typo. * Lowercase title in create_article. In some cases the title is not lowercased beforehand. * Replace <indent> with ident which is covered. * Add parsing of units for @font-feature-values. * Use exact element name to create link if it is present. This makes the article link scroll down to the exact element in the page more accurately than using the parent id. * No need for redirects for titles with ' * Appearance does not appear on mdn css reference. * Background-position-{x, y} are not on mdn css reference. * Bleed is not no mdn css reference. * border-start is not in mdn css reference. * Fix contrast typo. * fit is not covered in mdn css reference. * font-height is not covered by mdn css reference. * format is not covered by mdn css reference. * fr is not covered by mdn css reference. * grid-span is not covered by mdn css reference. * mark is not covered by mdn css reference. * Fix scroll-behavior typo. * SkewZ() is not covered by mdn css reference. * wrap-mode is not covered by mdn css reference. * Stop Mojo::IOLoop gracefully. See http://mojolicious.org/perldoc/Mojo/IOLoop#stop_gracefully * Delete keywords requiring extra processing or with 404 pages. See https://gist.github.com/hchienjo/c61adddbe63641a7b3c9de20a9995422 * Add output.txt. * Remove parsing of values data. This messes up with some of the articles as the titles collide. * Fetch only links in /docs/Web/CSS/ path. * Limit accepted alternate titles. * Parsing of #Feature_value_blocks is not harmful. * Parse values for font-variant-alternate. This is because it contains entries that are in tags.txt like: 1. swash() 2. ornaments() 3. annotation() * Remove "using css" from titles. * <counter> actually points to https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Lists_and_Counters/Using_CSS_counters * <flex> is converedd by flex. * mozilla.org/en-US/docs/Web/CSS/ident leads to a 404 error. * Add output.txt. * tighten rules on alternate titles. * Drop incorrect *.warning redirect. * Remove useless comment. * Limit creation of alternate titles for redirects. * Add output.txt. * No css snippet should be a redirect entry title. * Add output.txt. * Ensure removing brackets from titles preserves meaning. For instance :-moz-locale-dir(ltr) should be :-moz-locale-dir ltr and not :-moz-locale-dirltr. * Add output.txt. * New Six Library Fathead duckduckgo#453 (duckduckgo#532) * added Readme and fetch * added parse file * added class to store info about python_six lib * added class to parse the python six library content and added output.txt file * added new template and fixed issues * updated output.txt * added trigger words * if code section is not present then skip that in the abstraction * fixed issue of newline in abstarction * fixed indention issue * renamed the directory to match the id * change the trigger words * New Python Pip Fathead (duckduckgo#465) * parser,fetch and requirements added * Prelim pip fathead completed * Made required changes * SOme changes to parser * Fixed import file error. Also reduced output size to display only relevant information * Fixed some parsing errors * Serving pages to command <pip command> * Updated code to handle formatting error * added trigger words file and updated code * corrected few errors * New Flask Fathead (duckduckgo#459) * Added flask fathead * Added flask fathead * Parsed API documentation Flask * Escaped html symbols and newlines * Escaped html symbols and newlines * Fixed HTML class for closing quote * Fixed HTML class for closing quote * Fixes duckduckgo#751 Improves Redirects * Fixes duckduckgo#751 Changes done * Javascript: New Electron Fathead (duckduckgo#721) * First commit * Truncated fetch.sh * Scraping Home Page and indentation fixed * Bugs fixed * Final changes * First commit - Electron branch * Removed unnecessary data * Modified code * Indentation fixed * Final changes * Fixes duckduckgo#757 Updates docs and directory structure * Fixes duckduckgo#757: Some final clean ups * Added trigger_words.txt * added angular javascript trigger along with angular js :duckduckgo#755 * changes done as requested * updated parse.sh and readme * Fixed minor indentation issue. * Sort and make sure entries are unique. * Fix issue with parsing the newest docs. * Apparently, the docs could not be parsed anymore. This was solved by checking that the elements used were really bs4.element.Tag elements and actually had the .name etc properties. * Java: New Java Fathead (Continue duckduckgo#642) (duckduckgo#774) * Added method coverage data for the Java fathead * Added methods to output.txt * Removed classname from several functions * Fixed formatting for output and coverage * Added classesto methods in methods.txt and updated file * Coverage data and output are sorted if they exist * Added sorted methods.txt * Stripped methods of brackets and parameters * Added sorted version of methods.txt * Added a space between method description and url * Added formatted method parameters to methods.txt and output.txt * Added method output and coverage without parameters * Changed method_url find parameter from "href" to "a" * Removed test_parse.py due to it being a redundant test * Added tests for checking the output is correct in output.txt and method coverage * Removed pipe to uniq -u and added sort -u instead * Removed test cases for method parameter formatting * Fixed spacing in method coverage so that sort -u recognizes duplicates * Removed output.txt from .gitignore * Added -f option to sort in parse.sh * Excluded methods without parameters from being appended twice to output.txt * Fixed method_url parsing and reverted method extraction function * Fixed formatting issues on cover and output so that sort -u recognizes duplicates * Added <section> to method descriptions and an additional tab * Remove output.txt after beta upload * Python: New PEP Fathead (duckduckgo#627) * Added PEP Fathead. * Removed data.url file. * Tab replacement now 4 spaces. * Updated trigger words. * Adding output file. * Updated title. * Updated with redirects. * Removed meta.txt file. * Removed 'PEP' trigger word. * Updated parser from review. * Update Parser and redirects. * Updating output.txt file for beta server. * Updated output.txt * Updated Parser for new lines. * Updated parser. * Patched Parser flush bug. * Fixes spacing issue. * Refactored parser. * Updated parser to include newlines in code blocks. * Updated parser. * Updated parser to handle <p> * JavaScript: New Vuejs Fathead (duckduckgo#776) * Initial commit * added trigger words.txt * added more redirects * fixed redirect function * fixed abstract title * fixed formatting of code in abstract * fixed trigger words, changed order of abstraction * Removed unnecessary redirects * Python Coverage Fathead (duckduckgo#738) * inital commit to repo * fixes incorrect file write * adds trigger words * updates README to install requirements * corrects inconsistency of method docstrings * removes blank line in triggerwords, adds output.txt * removes minor version spec from pip install doc line * removes original site URL and text * changes html cleanup to preserve some tags * removes debugging print line, swaps <cite> tags with <code> tags * Ocaml: New OPAM fathead. (duckduckgo#775) * Updated stale OPAM fathead. * Added trigger words. * Updated parser to deal with version. * Updated parser. Updated output.txt * npm CLI: Added Categories (duckduckgo#754) * Added list of commands for npm cli * Removed npm.txt * Updated from npm-cli.txt * Delete npm-cli.txt * Revert "Swift Docs: Readme updated and url changed from 2.2 to 3.1" (duckduckgo#790) * better code, fixed documentation link
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of new Instant Answer, or changes
New Scala Fathead
Fixes: #692
People to notify
@pjhampton
Testing & Review
To be completed by Language Leader (or DDG Staff) when reviewing Pull Request
Pull Request
Instant Answer Page (for new Instant Answers)
Code
$ duckpan test <fathead_id>)Pull Request Review Guidelines: https://docs.duckduckhack.com/programming-mission/pr-review.html
Instant Answer Page: https://duck.co/ia/view/scala