Merge e80b7d4 into f207996

FRSOURCE · Oct 26, 2020 · f58ca89 · f58ca89
2 parents f207996 + e80b7d4
commit f58ca89
Show file tree

Hide file tree

Showing 13 changed files with 736 additions and 343 deletions.
diff --git a/README.md b/README.md
@@ -13,9 +13,17 @@
 The fastest ([see benchmarks](#benchmarks)) CLI & Node wrapper around [javascript replace](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/replace) which allows on-the-fly replacing (with or without changing input files), [globbing](https://en.wikipedia.org/wiki/Glob_(programming)), [piping](https://en.wikipedia.org/wiki/Pipeline_(Unix)) and many more!
 
 * [:scroll: Installation](#scroll-installation)
-* [:books: Node API usage](#books-node-api-usage)
-* [:keyboard: CLI usage](#keyboard-cli-usage)
-* [:mag_right: Examples](#mag_right-examples)
+* [:books: Node API](#books-node-api)
+* [:keyboard: CLI API](#keyboard-cli-api)
+* [:mag_right: Example usage](#mag_right-example-usage)
+  * [1. Replace all `a` occurences with `b` from given `foo.js` file and return result / write result to the console](#1-replace-all-a-occurences-with-b-from-given-foojs-file-and-return-result--write-result-to-the-console)
+  * [2. Replace all `a` occurences with `b` from given `foo.js` and save result to the `foo_replaced.js`](#2-replace-all-a-occurences-with-b-from-given-foojs-and-save-result-to-the-foo_replacedjs)
+  * [3. Replace all `a` occurences with `b` from given array of files and save result to the `foo_replaced.js` using default `\n` as result-joining string](#3-replace-all-a-occurences-with-b-from-given-array-of-files-and-save-result-to-the-foo_replacedjs-using-default-n-as-result-joining-string)
+  * [4. Replace all `a` occurences with `b` from all `.js` files in `foo` directory and save result to the `foo_replaced.js` using `\n/////\n` as result-joining string](#4-replace-all-a-occurences-with-b-from-all-js-files-in-foo-directory-and-save-result-to-the-foo_replacedjs-using-nn-as-result-joining-string)
+  * [5. Replace all `a` occurences with `b` in given content string `abcd` and save result to the `foo_replaced.js`](#5-replace-all-a-occurences-with-b-in-given-content-string-abcd-and-save-result-to-the-foo_replacedjs)
+  * [6. Replace all `a` occurences with `b` from piped stream and save it to the output file](#6-replace-all-a-occurences-with-b-from-piped-stream-and-save-it-to-the-output-file)
+  * [7. Replace all `a` occurences with `b` from piped stream and pass it through `stdout` stream to the `<next-command>`](#7-replace-all-a-occurences-with-b-from-piped-stream-and-pass-it-through-stdout-stream-to-the-next-command)
+  * [8. Both pipe & options styles can be mixed together, here - getting input from `-i` argument and passing output down the stream to the `<next-command>`](#8-both-pipe--options-styles-can-be-mixed-together-here---getting-input-from--i-argument-and-passing-output-down-the-stream-to-the-next-command)
 * [:chart_with_upwards_trend: Benchmarks](#chart_with_upwards_trend-benchmarks)
 
 ## :scroll: Installation
@@ -33,33 +41,42 @@ npm install @frsource/frs-replace
 or download
 [zipped from `@frsource/frs-replace` releases](https://github.com/FRSource/frs-replace/releases)
 
-## :books: Node API usage
+## :books: Node API
 
-@frsource/frs-replace package provides 2 methods for synchronous / asynchronous (with promise and ES6 `async`/`await` syntax support) usage:
+@frsource/frs-replace package provides 2 methods: for synchronous or for asynchronous (with promise and ES6 `async`/`await` syntax support) usage:
 
 ```javascript
 const FRSReplace = require('@frsource/frs-replace');
+// or
+import * as FRSReplace from '@frsource/frs-replace';
 
 FRSReplace.sync({/* options */})
 FRSReplace.async({/* options */})
+
+// you might also want to import methods separately (e.g. import only one of them):
+const FRSReplaceSync = require('@frsource/frs-replace/sync');
+const FRSReplaceAsync = require('@frsource/frs-replace/async');
+// or
+import { sync, async } from '@frsource/frs-replace';
 ```
 
 Where `/* options */` is an object containing:
-> Note: remember that you need to provide some input for @frsource/frs-replace to work, so one of the parameters: input or content is **required**
+> Note: remember that you need to provide some input for @frsource/frs-replace to work, so **one of parameters: input or content is required**
 
 | Option | Type | Default | Description |
   | --- | --- | --- | --- |
-  | input | string or string[] | *undefined* | Path to files or [fast-glob](https://github.com/mrmlnc/fast-glob) pattern pointing to files to be read & replaced from. If multiple files specified results will be joined using `inputJoinString` option's value) |
+  | input | string or string[] | *undefined* | Path to files or [fast-glob](https://github.com/mrmlnc/fast-glob) pattern pointing to files to be read & replaced from. If multiple files specified results will be joined using `outputJoinString` option's value) |
   | inputReadOptions | string or object | utf8 | Options which are passed directly to the [readFileSync method](https://nodejs.org/api/fs.html#fs_fs_readfilesync_path_options) when reading input file |
   | inputGlobOptions | object | *undefined* | Options which are passed directly to the [fast-glob package](https://github.com/mrmlnc/fast-glob#options-1) when resolving glob patterns |
-  | inputJoinString | string | \n | String used when joining multiple files, passed directly to [javascript join](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/join#Syntax) |
   | content    | string | *undefined* | Content to be replaced (takes precedence over file input) |
-  | regex    | string or [RegExp Object](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/RegExp#Syntax)| *-* | Used as a first argument of [javascript replace](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/replace#Syntax) |
+  | strategy   | "join" or "flatten" or "preserve-structure" | "join" | Output file generation strategy. *"join"* - joins all input files and outputs them as a single file using path passed as: *"output"*. *"preserve-structure"* - saves all files to the *"output"* directory keeping relative directory structure.*"flatten"* - same as *"preserve-structure"* but flattens the directory structure  |
+  | needle    | string or [RegExp Object](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/RegExp#Syntax)| *-* | Used as a first argument of [javascript replace](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/replace#Syntax) |
   | replacement  | string | *-* | Passed as a second argument to [javascript replace](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/replace#Syntax) |
   | output | string | *undefined* | Path of an output file |
-  | outputWriteOptions  | string or object | utf8 | Passed as options argument of [write's .sync](https://www.npmjs.com/package/write#sync) |
+  | outputWriteOptions  | string or object | "utf8" | Passed as options argument of [write's .sync](https://www.npmjs.com/package/write#sync) |
+  | outputJoinString | string | \n | String used when joining multiple files, passed directly to [javascript join](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/join#Syntax) |
 
-## :keyboard: CLI usage
+## :keyboard: CLI API
 
 ```bash
 frs-replace <regex> <replacement> [options]
@@ -80,24 +97,25 @@ frs-replace <regex> <replacement> [options]
 
   | Option | Type | Default | Description |
   | --- | --- | --- | --- |
-  |&#8209;i, &#8209;&#8209;input       | string or string[] | *-* | Path to files or [fast-glob](https://github.com/mrmlnc/fast-glob) pattern pointing to files to be read & replaced from. If multiple files specified results will be joined using `inputJoinString` option's value) |
+  |&#8209;i, &#8209;&#8209;input       | string or string[] | *-* | Path to files or [fast-glob](https://github.com/mrmlnc/fast-glob) pattern pointing to files to be read & replaced from. If multiple files specified results will be joined using `outputJoinString` option's value) |
   | &#8209;&#8209;i-read-opts | string or object | utf8 | Options which are passed directly to the [readFileSync method](https://nodejs.org/api/fs.html#fs_fs_readfilesync_path_options) when reading input file |
   | &#8209;&#8209;i-glob-opts | object | *undefined* | Options which are passed directly to the [fast-glob package](https://github.com/mrmlnc/fast-glob#options-1) when resolving glob patterns |
-  | &#8209;&#8209;i-join-str | string | \n | Used when joining multiple files, passed directly to [javascript join](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/join#Syntax) |
   | &#8209;o, &#8209;&#8209;output     | string | *-* | Output file name/path (replaces the file if it already exists and creates any intermediate directories if they don't already exist) |
   | &#8209;&#8209;o-write-opts | string or object | utf8 | Passed as options argument of [write's .sync](https://www.npmjs.com/package/write#sync) |
-  | &#8209;f, &#8209;&#8209;flags | combination of *gim* flags | g | [RegExp](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/RegExp#Syntax) flags |
+  | &#8209;&#8209;o-join-str | string | \n | Used when joining multiple files, passed directly to [javascript join](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/join#Syntax) |
   | &#8209;c, &#8209;&#8209;content    | string | *-* | Content to be replaced (takes precedence over stream & file input) |
-  | &#8209;&#8209;stdout  | boolean | true if piped input present, false otherwise | Force sending output on stdout |
+  | &#8209;s, &#8209;&#8209;strategy   | "join" or "flatten" or "preserve-structure" | "join" | Output file generation strategy. *"join"* - joins all input files and outputs them as a single file using path passed as: *"output"*. *"preserve-structure"* - saves all files to the *"output"* directory keeping relative directory structure.*"flatten"* - same as *"preserve-structure"* but flattens the directory structure  |
+  | &#8209;f, &#8209;&#8209;flags | combination of *gim* flags | g | [RegExp](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/RegExp#Syntax) flags |
+  | &#8209;&#8209;stdout  | boolean | *true* if piped input present, *false* otherwise | Force sending output on stdout |
   | &#8209;r, &#8209;&#8209;replace&#8209;fn | boolean | false | Treat replacement argument as path to file containing replacement function |
   | &#8209;h, &#8209;&#8209;help  | boolean | *-* | Show help |
   | &#8209;v, &#8209;&#8209;version  | boolean | *-* | Show version number |
 
-## :mag_right: Examples
+## :mag_right: Example usage
 
 > Note: While most of examples are using synchronous API method in all cases `.async` is applicable as well.
 
-### 1. Replace all `a` occurences with `b` from given `foo.js` file and return result / write result to console
+### 1. Replace all `a` occurences with `b` from given `foo.js` file and return result / write result to the console
 
 <details><summary>Click to expand</summary>
 
@@ -184,13 +202,13 @@ const result = require('@frsource/frs-replace').sync({
 #### 3.2 CLI
 
 ```bash
-frs-replace a b -i foo.js foo2.js -o foo_replaced.js --i-join-str "\n/////\n"
+frs-replace a b -i foo.js foo2.js -o foo_replaced.js
 ```
 
 or
 
 ```bash
-frs-replace a b -i foo.js -i foo2.js -o foo_replaced.js --i-join-str "\n/////\n"
+frs-replace a b -i foo.js -i foo2.js -o foo_replaced.js
 ```
 
 > Note: Arrays can be passed under single flag-entry as a space-separated list *or* under same flag repeated multiple times (all values will be concatenated into single array using, details - [yargs array notation](https://github.com/yargs/yargs/blob/master/docs/tricks.md#arrays)).
@@ -208,15 +226,15 @@ const result = require('@frsource/frs-replace').sync({
   input           : 'foo/*.js',
   regex           : new RegExp('a', 'g'),
   replacement     : 'b',
-  inputJoinString : '\n/////\n',
+  outputJoinString : '\n/////\n',
   output          : 'foo_replaced.js'
 })
 ```
 
 #### 4.2 CLI
 
 ```bash
-frs-replace a b -i foo/*.js -o foo_replaced.js --i-join-str "\n/////\n"
+frs-replace a b -i foo/*.js -o foo_replaced.js --o-join-str "\n/////\n"
 ```
 
 </details>
@@ -276,20 +294,20 @@ frs-replace a b -i foo.js | <next-command>
 
 | Library (best&nbsp;bolded) | Execution time [s] | Difference percentage (comparing&nbsp;to&nbsp;best&nbsp;time) |
 | --- | --- | --- |
-| frs-replace async | 0.01546155 | 84.3014% |
-| **frs-replace sync** | 0.00838927 | 0.0000% |
-| replace-in-file | 0.02182254 | 160.1244% |
+| frs-replace async | 0.01362554 | 34.9580% |
+| **frs-replace sync** | 0.01009613 | 0.0000% |
+| replace-in-file | 0.02028758 | 100.9440% |
 | replace async | *N/A* | *N/A* |
-| replace sync | 0.04994809 | 495.3804% |
+| replace sync | 0.05186623 | 413.7238% |
 | replace-string | *N/A* | *N/A* |
 
 ### input & replacement as strings [1000 iterations x 100 repetitions]
 
 | Library (best&nbsp;bolded) | Execution time [s] | Difference percentage (comparing&nbsp;to&nbsp;best&nbsp;time) |
 | --- | --- | --- |
-| frs-replace async | 0.00003655 | 17.4625% |
-| **frs-replace sync** | 0.00003112 | 0.0000% |
+| frs-replace async | 0.00023470 | 472.5367% |
+| frs-replace sync | 0.00004455 | 8.6850% |
 | replace-in-file | *N/A* | *N/A* |
 | replace async | *N/A* | *N/A* |
 | replace sync | *N/A* | *N/A* |
-| replace-string | 0.00003231 | 3.8462% |
+| **replace-string** | 0.00004099 | 0.0000% |
diff --git a/async.js b/async.js
@@ -0,0 +1 @@
+module.exports = require('./src/sync')
diff --git a/benchmark/multiple-file-replace.benchmark.test.js b/benchmark/multiple-file-replace.benchmark.test.js
@@ -5,7 +5,8 @@ const fs = require('fs')
 const perfy = require('perfy')
 const glob = require('fast-glob')
 
-const FRSreplace = require('../src/replace')
+const FRSreplaceSync = require('../src/sync')
+const FRSreplaceAsync = require('../src/async')
 const replace = require('replace')
 const replaceInFile = require('replace-in-file')
 const replaceString = require('replace-string')
@@ -22,7 +23,7 @@ const tmpPrefixes = {
 const defaults = {
   inputReadOptions: 'utf8',
   outputWriteOptions: 'utf8',
-  inputJoinString: '\n'
+  outputJoinString: '\n'
 }
 const repetitionsNo = 100000
 const iterationsNo = 1000
@@ -109,14 +110,14 @@ tap.teardown(() => {
   fs.writeFileSync('./README.md', readmeContent.replace(/(##\s:chart_with_upwards_trend:\sBenchmarks)[\s\S]*?(?:$|(?:\s##\s))/, `$1\n\n> Tested on Node ${process.version}.\n${perfyResults}`))
 })
 
-tap.test(`input as glob pattern [${inputFilesNo} files x ${iterationsNo} iterations x ${repetitionsNo / iterationsNo} repetitions]`, async ct => {
-  const results = await multipleTests(ct, [
+tap.test(`input as glob pattern [${inputFilesNo} files x ${iterationsNo} iterations x ${repetitionsNo / iterationsNo} repetitions]`, async t => {
+  const results = await multipleTests(t, [
     {
-      fn: () => FRSreplace.async(testInput.FRSReplace),
+      fn: () => FRSreplaceAsync(testInput.FRSReplace),
       before: () => (testInput.FRSReplace.input = `${dir}/${tmpPrefixes.input}*`)
     },
     {
-      fn: () => FRSreplace.sync(testInput.FRSReplace),
+      fn: () => FRSreplaceSync(testInput.FRSReplace),
       before: () => (testInput.FRSReplace.input = `${dir}/${tmpPrefixes.input}*`)
     },
     {
@@ -133,26 +134,26 @@ tap.test(`input as glob pattern [${inputFilesNo} files x ${iterationsNo} iterati
     undefined
   ])
 
-  const result = outputPerfy(ct, results, results.slice().sort(sortByNumberVariable('fullNanoseconds'))[0])
+  const result = outputPerfy(t, results, results.slice().sort(sortByNumberVariable('fullNanoseconds'))[0])
   const sortedResults = result.results.slice().sort(sortByNumberVariable('avgTime'))
 
-  ct.is((sortedResults[0].name.indexOf('frs-replace sync') !== -1 || (sortedResults[1].name.indexOf('frs-replace sync') !== -1 && sortedResults[1].avgPercentageDifference < 5)), true, 'frs-replace sync should be the fastest or second, but at most with 5% difference to best')
-  ct.is(sortedResults[0].name.indexOf('frs-replace async') !== -1 || sortedResults[1].name.indexOf('frs-replace async') !== -1, true, 'frs-replace async should be the fastest or second')
+  t.is((sortedResults[0].name.indexOf('frs-replace sync') !== -1 || (sortedResults[1].name.indexOf('frs-replace sync') !== -1 && sortedResults[1].avgPercentageDifference < 5)), true, 'frs-replace sync should be the fastest or second, but at most with 5% difference to best')
+  t.is(sortedResults[0].name.indexOf('frs-replace async') !== -1 || sortedResults[1].name.indexOf('frs-replace async') !== -1, true, 'frs-replace async should be the fastest or second')
 
-  ct.end()
+  t.end()
 })
 
-tap.test(`input & replacement as strings [${iterationsNo} iterations x ${repetitionsNo / iterationsNo} repetitions]`, async ct => {
-  const results = await multipleTests(ct, [
+tap.test(`input & replacement as strings [${iterationsNo} iterations x ${repetitionsNo / iterationsNo} repetitions]`, async t => {
+  const results = await multipleTests(t, [
     {
-      fn: () => FRSreplace.async(testInput.FRSReplace),
+      fn: () => FRSreplaceAsync(testInput.FRSReplace),
       before: () => {
         testInput.FRSReplace.regex = regex.source
         testInput.FRSReplace.content = content
       }
     },
     {
-      fn: () => FRSreplace.sync(testInput.FRSReplace),
+      fn: () => FRSreplaceSync(testInput.FRSReplace),
       before: () => {
         testInput.FRSReplace.regex = regex.source
         testInput.FRSReplace.content = content
@@ -164,12 +165,12 @@ tap.test(`input & replacement as strings [${iterationsNo} iterations x ${repetit
     { fn: () => replaceString(content, regex.source, replacement) }
   ])
 
-  const result = outputPerfy(ct, results, results.slice().sort(sortByNumberVariable('fullNanoseconds'))[0])
+  const result = outputPerfy(t, results, results.slice().sort(sortByNumberVariable('fullNanoseconds'))[0])
   const sortedResults = result.results.slice().sort(sortByNumberVariable('avgTime'))
 
-  ct.is((sortedResults[0].name.indexOf('frs-replace') !== -1 || (sortedResults[1].name.indexOf('frs-replace') !== -1 && sortedResults[1].avgPercentageDifference < 10)), true, 'frs-replace should be the fastest or second, but at most with 10% difference to best')
+  t.is((sortedResults[0].name.indexOf('frs-replace') !== -1 || (sortedResults[1].name.indexOf('frs-replace') !== -1 && sortedResults[1].avgPercentageDifference < 20)), true, 'frs-replace should be the fastest or second, but at most with 20% difference to the best')
 
-  ct.end()
+  t.end()
 })
 
 function outputPerfy (t, testResults, best) {
@@ -248,7 +249,7 @@ async function multipleTests (t, testCfgs, n, iterations) {
     const { v: testCfg, i: index } = testCfgs[k]
     const prevResult = results[index]
     const libName = testedLibraries[index]
-    await t.test(`${t.name} - ${libName}`, async ct => {
+    await t.test(`${t.name} - ${libName}`, async t => {
       for (let i = 0; i < n; ++i) {
         testCfg.before && testCfg.before()
         const result = await singleTest(libName, testCfg.fn, iterations)
@@ -264,7 +265,7 @@ async function multipleTests (t, testCfgs, n, iterations) {
           }
         }
       }
-      ct.end()
+      t.end()
     })
   }