v0.2.0 Pre-Release

README.md

@@ -8,12 +8,15 @@
 - All transformations support multibyte characters.
 
 ```
-Usage of Password Transformation Tool (ptt) version (0.1.3):
+Usage of Password Transformation Tool (ptt) version (0.2.0):
 
 ptt [options] [...]
 Accepts standard input and/or additonal arguments.
 
 Options:
+  -b    Bypass map creation and use stdout as primary output.
+  -d int
+        Enable debug mode with verbosity levels [0-2].
   -f value
         Read additional files for input.
   -i value
@@ -36,6 +39,8 @@ Options:
         Transformation to apply to input.
   -tf value
         Read additional files for transformations if applicable.
+  -tp value
+        Read a template file for multiple transformations and operations.
   -u value
         Read additional URLs for input.
   -v    Show verbose output when possible.
@@ -44,7 +49,7 @@ Options:
   -vvv
         Show verbose statistics output when possible.
 
-The -f, -k, -r, -tf, and -u flags can be used multiple times and together.
+The -f, -k, -r, -tf, -tp, and -u flags can be used multiple times and together.
 
 Transformation Modes:
   -t append
@@ -65,10 +70,12 @@ Transformation Modes:
         Transforms input into insert rules starting at index.
   -t mask -rm [uldsb] -v
         Transforms input by masking characters with provided mask.
+  -t mask-match -tf [file]
+        Transforms input by keeping only strings with matching masks from a mask file.
+  -t mask-retain -rm [uldsb] -tf [file]
+        Transforms input by creating masks that still retain strings from file.
   -t mask-swap -tf [file]
         Transforms input by swapping tokens from a partial mask file and a input file.
-  -t match -tf [file]
-        Transforms input by keeping only strings with matching masks from a mask file.
   -t overwrite -i [index]
         Transforms input into overwrite rules starting at index.
   -t pop -rm [uldsb]
@@ -81,8 +88,6 @@ Transformation Modes:
         Transforms input into prepend-shift rules.
   -t remove -rm [uldsb]
         Transforms input by removing characters with provided mask characters.
-  -t retain -rm [uldsb] -tf [file]
-        Transforms input by creating masks that still retain strings from file.
   -t swap -tf [file]
         Transforms input by swapping tokens with exact matches from a ':' separated file.
   -t toggle -i [index]

docs/USAGE.md

@@ -1,5 +1,5 @@
 # Password Transformation Tool (PTT) Usage Guide
-## Version 0.1.0
+## Version 0.2.0
 
 ### Table of Contents
 1. [Introduction](#introduction)
@@ -65,11 +65,26 @@ There are some additional notes when importing data:
 - When reading from standard input, the tool can detect chaining `ptt` commands
   when the `-v` flag is used. This can be used to pipe multiple commands together.
 - When reading from files, the tool can detect when `ptt` JSON output is used as input and will parse the JSON data.
+- The `-b` flag can be used to bypass map creation and use stdout as primary output. This can be useful for working with large amounts of data.
+    - If the `-b` flag is used, the final output will be empty and all
+      filtering and duplication removal will be disabled.
+- The `-d [0-2]` flag can be used to enable debug output. This will show the data
+  object after all transformations have been applied. There are two (2) levels
+  of debug output that can be used.
+    - Level 1 will not print each iteration transformation but overall input and output.
+    - Level 2 will print each iteration transformation and overall input and output.
+- The `-tp` flag can not be used with other transformations at the same time (`-t`). The
+  template file should contain a list of transformations and operations to apply
+  to the input data. The template file should be in JSON format.
+    - See `docs/template.json` ([link](https://github.com/JakeWnuk/ptt/blob/main/docs/template.json)) for an example.
+    - See `docs/templates/` ([link](https://github.com/JakeWnuk/ptt/blob/main/docs/templates/)) for more examples.
 
-The `-f`, `-k`, `-r`, `-tf`, and `-u` flags can be used multiple times and have
+The `-f`, `-k`, `-r`, `-tf`, `-tp`, and `-u` flags can be used multiple times and have
 their collective values combined. The rest of the flags can only be used once.
 
 #### Options:
+- `-b`: Bypass map creation and use stdout as primary output.
+- `-d`: Enable debug mode with verbosity levels [0-2].
 - `-f`: Read additional files for input.
 - `-i`: Starting index for transformations if applicable. Accepts ranges separated by '-'. (default 0)
 - `-k`: Only keep items in a file.
@@ -81,6 +96,7 @@ their collective values combined. The rest of the flags can only be used once.
 - `-rm`: Replacement mask for transformations if applicable. (default "uldsb")
 - `-t`: Transformation to apply to input.
 - `-tf`: Read additional files for transformations if applicable.
+- `-tp`: Read a template file for multiple transformations and operations.
 - `-u`: Read additional URLs for input.
 - `-v`: Show verbose output when possible.
 - `-vv`: Show statistics output when possible.
@@ -103,8 +119,8 @@ The following transformations can be used with the `-t` flag:
 - `dehex`: Transforms input by decoding $HEX[...] formatted
 - `mask`: Transforms input by masking characters with provided mask.
 - `remove`: Transforms input by removing characters with provided mask characters.
-- `retain`: Transforms input by creating masks that still retain strings from file.
-- `match`: Transforms input by keeping only strings with matching masks from a mask file
+- `mask-retain`: Transforms input by creating masks that still retain strings from file.
+- `mask-match`: Transforms input by keeping only strings with matching masks from a mask file
 - `swap`: Transforms input by swapping tokens with exact matches from a ':' separated file.
 - `pop`: Transforms input by generating tokens from popping strings at character boundaries.
 - `mask-swap`: Transforms input by swapping tokens from a partial mask file and a input file.
@@ -149,6 +165,7 @@ keywords above:
 - `ptt -t [transformation] -rm ulds`: Apply a transformation with a custom mask. Default is all characters.
 - `ptt -t [transformation] -i 5`: Apply a transformation starting at a specific index.
 - `ptt -i 1-5 -t [transformation]`: Apply a transformation starting at a specific index.
+- `ptt -tp template.json`: Apply multiple transformations and operations from a template file.
 
 #### Filter Formats:
 - `ptt -k keep.txt`: Keep only items in a file.

docs/template.json

@@ -0,0 +1,18 @@
+[
+    {
+        "StartIndex": 0,
+        "EndIndex": 0,
+        "Verbose": false,
+        "ReplacementMask": "uldsb",
+        "Bypass": false,
+        "TransformationMode": "mask"
+    },
+    {
+        "StartIndex": 0,
+        "EndIndex": 0,
+        "Verbose": false,
+        "ReplacementMask": "dsb",
+        "Bypass": false,
+        "TransformationMode": "mask"
+    }
+]

main.go

@@ -15,14 +15,15 @@ import (
 	"github.com/jakewnuk/ptt/pkg/utils"
 )
 
-var version = "0.1.3"
+var version = "0.2.0"
 var wg sync.WaitGroup
 var mutex = &sync.Mutex{}
 var retain models.FileArgumentFlag
 var remove models.FileArgumentFlag
 var readFiles models.FileArgumentFlag
 var readURLs models.FileArgumentFlag
 var transformationFiles models.FileArgumentFlag
+var templateFiles models.FileArgumentFlag
 var intRange models.IntRange
 var lenRange models.IntRange
 var primaryMap map[string]int
@@ -35,29 +36,29 @@ func main() {
 		fmt.Fprintf(os.Stderr, "ptt [options] [...]\nAccepts standard input and/or additonal arguments.\n\n")
 		fmt.Fprintf(os.Stderr, "Options:\n")
 		flag.PrintDefaults()
-		fmt.Fprintf(os.Stderr, "\nThe -f, -k, -r, -tf, and -u flags can be used multiple times and together.\n")
+		fmt.Fprintf(os.Stderr, "\nThe -f, -k, -r, -tf, -tp, and -u flags can be used multiple times and together.\n")
 		fmt.Fprintln(os.Stderr, "\nTransformation Modes:")
 		modes := map[string]string{
-			"append":                        "Transforms input into append rules.",
-			"append-remove":                 "Transforms input into append-remove rules.",
-			"append-shift":                  "Transforms input into append-shift rules.",
-			"prepend":                       "Transforms input into prepend rules.",
-			"prepend-remove":                "Transforms input into prepend-remove rules.",
-			"prepend-shift":                 "Transforms input into prepend-shift rules.",
-			"insert -i [index]":             "Transforms input into insert rules starting at index.",
-			"overwrite -i [index]":          "Transforms input into overwrite rules starting at index.",
-			"toggle -i [index]":             "Transforms input into toggle rules starting at index.",
-			"encode":                        "Transforms input by URL, HTML, and Unicode escape encoding.",
-			"decode":                        "Transforms input by URL, HTML, and Unicode escape decoding.",
-			"hex":                           "Transforms input by encoding strings into $HEX[...] format.",
-			"dehex":                         "Transforms input by decoding $HEX[...] formatted strings.",
-			"mask -rm [uldsb] -v":           "Transforms input by masking characters with provided mask.",
-			"remove -rm [uldsb]":            "Transforms input by removing characters with provided mask characters.",
-			"retain -rm [uldsb] -tf [file]": "Transforms input by creating masks that still retain strings from file.",
-			"pop -rm [uldsb]":               "Transforms input by generating tokens from popping strings at character boundaries.",
-			"match -tf [file]":              "Transforms input by keeping only strings with matching masks from a mask file.",
-			"swap -tf [file]":               "Transforms input by swapping tokens with exact matches from a ':' separated file.",
-			"mask-swap -tf [file]":          "Transforms input by swapping tokens from a partial mask file and a input file.",
+			"append":                             "Transforms input into append rules.",
+			"append-remove":                      "Transforms input into append-remove rules.",
+			"append-shift":                       "Transforms input into append-shift rules.",
+			"prepend":                            "Transforms input into prepend rules.",
+			"prepend-remove":                     "Transforms input into prepend-remove rules.",
+			"prepend-shift":                      "Transforms input into prepend-shift rules.",
+			"insert -i [index]":                  "Transforms input into insert rules starting at index.",
+			"overwrite -i [index]":               "Transforms input into overwrite rules starting at index.",
+			"toggle -i [index]":                  "Transforms input into toggle rules starting at index.",
+			"encode":                             "Transforms input by URL, HTML, and Unicode escape encoding.",
+			"decode":                             "Transforms input by URL, HTML, and Unicode escape decoding.",
+			"hex":                                "Transforms input by encoding strings into $HEX[...] format.",
+			"dehex":                              "Transforms input by decoding $HEX[...] formatted strings.",
+			"mask -rm [uldsb] -v":                "Transforms input by masking characters with provided mask.",
+			"remove -rm [uldsb]":                 "Transforms input by removing characters with provided mask characters.",
+			"mask-retain -rm [uldsb] -tf [file]": "Transforms input by creating masks that still retain strings from file.",
+			"pop -rm [uldsb]":                    "Transforms input by generating tokens from popping strings at character boundaries.",
+			"mask-match -tf [file]":              "Transforms input by keeping only strings with matching masks from a mask file.",
+			"swap -tf [file]":                    "Transforms input by swapping tokens with exact matches from a ':' separated file.",
+			"mask-swap -tf [file]":               "Transforms input by swapping tokens from a partial mask file and a input file.",
 		}
 
 		// Sort and print transformation modes
@@ -82,10 +83,13 @@ func main() {
 	transformation := flag.String("t", "", "Transformation to apply to input.")
 	replacementMask := flag.String("rm", "uldsb", "Replacement mask for transformations if applicable.")
 	jsonOutput := flag.String("o", "", "Output to JSON file in addition to stdout.")
+	bypassMap := flag.Bool("b", false, "Bypass map creation and use stdout as primary output.")
+	debugMode := flag.Int("d", 0, "Enable debug mode with verbosity levels [0-2].")
 	flag.Var(&retain, "k", "Only keep items in a file.")
 	flag.Var(&remove, "r", "Only keep items not in a file.")
 	flag.Var(&readFiles, "f", "Read additional files for input.")
 	flag.Var(&transformationFiles, "tf", "Read additional files for transformations if applicable.")
+	flag.Var(&templateFiles, "tp", "Read a template file for multiple transformations and operations.")
 	flag.Var(&intRange, "i", "Starting index for transformations if applicable. Accepts ranges separated by '-'.")
 	flag.Var(&lenRange, "l", "Keeps output equal to or within a range of lengths. Accepts ranges separated by '-'.")
 	flag.Var(&readURLs, "u", "Read additional URLs for input.")
@@ -97,9 +101,10 @@ func main() {
 	removeMap := utils.ReadFilesToMap(fs, remove)
 	readFilesMap := utils.ReadFilesToMap(fs, readFiles)
 	transformationFilesMap := utils.ReadFilesToMap(fs, transformationFiles)
+	transformationTemplateArray := utils.ReadJSONToArray(fs, templateFiles)
 	readURLsMap, err := utils.ReadURLsToMap(readURLs)
 	if err != nil {
-		fmt.Println("Error reading URLs:", err)
+		fmt.Fprintf(os.Stderr, "[!] Error reading URLs: %s\n", err)
 		return
 	}
 
@@ -108,7 +113,7 @@ func main() {
 	if (stat.Mode() & os.ModeCharDevice) == 0 {
 		primaryMap, err = utils.LoadStdinToMap(bufio.NewScanner(os.Stdin))
 		if err != nil {
-			fmt.Println("Error reading from stdin:", err)
+			fmt.Fprintf(os.Stderr, "[!] Error reading from stdin: %s\n", err)
 			return
 		}
 	}
@@ -123,16 +128,43 @@ func main() {
 		primaryMap = utils.CombineMaps(primaryMap, readFilesMap, readURLsMap)
 	}
 
+	// Bypass map creation if requested
+	if *bypassMap {
+		fmt.Fprintf(os.Stderr, "[*] Bypassing map creation and using stdout as primary output. Some features are disabled.\n")
+	}
+
 	// Apply transformation if provided
-	if *transformation != "" {
-		primaryMap = transform.TransformationController(primaryMap, *transformation, intRange.Start, intRange.End, *verbose, *replacementMask, transformationFilesMap)
+	if *transformation != "" && templateFiles == nil {
+		primaryMap = transform.TransformationController(primaryMap, *transformation, intRange.Start, intRange.End, *verbose, *replacementMask, transformationFilesMap, *bypassMap, *debugMode)
+	} else if templateFiles != nil && *transformation == "" {
+		fmt.Fprintf(os.Stderr, "[*] Using template files for multiple transformations.\n")
+
+		// Make a copy of the primary map to avoid modifying the original
+		temporaryMap := make(map[string]int)
+		for k, v := range primaryMap {
+			temporaryMap[k] = v
+		}
+
+		// Apply transformations from template files
+		for i, template := range transformationTemplateArray {
+			if i == 0 {
+				temporaryMap = transform.TransformationController(primaryMap, template.TransformationMode, template.StartIndex, template.EndIndex, template.Verbose, template.ReplacementMask, transformationFilesMap, template.Bypass, *debugMode)
+			} else {
+				temporaryMap = utils.CombineMaps(temporaryMap, transform.TransformationController(primaryMap, template.TransformationMode, template.StartIndex, template.EndIndex, template.Verbose, template.ReplacementMask, transformationFilesMap, template.Bypass, *debugMode))
+			}
+		}
+		primaryMap = temporaryMap
+
+	} else if *transformation != "" && templateFiles != nil {
+		fmt.Fprintf(os.Stderr, "[!] Transformation and template flags cannot be used together.\n")
+		return
 	}
 
 	// Process retain and remove maps if provided
 	if len(retainMap) > 0 || len(removeMap) > 0 {
 		primaryMap, err = format.RetainRemove(primaryMap, retainMap, removeMap)
 		if err != nil {
-			fmt.Println("Error processing retain and remove flags:", err)
+			fmt.Fprintf(os.Stderr, "[!] Error processing retain and remove flags: %s\n", err)
 			return
 		}
 	}
@@ -160,7 +192,7 @@ func main() {
 	if *jsonOutput != "" {
 		err = format.SaveArrayToJSON(*jsonOutput, primaryMap)
 		if err != nil {
-			fmt.Println("Error saving output to JSON:", err)
+			fmt.Fprintf(os.Stderr, "[!] Error saving output to JSON: %s\n", err)
 			return
 		}
 	}