Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Node API #291

Merged
merged 156 commits into from Jul 5, 2019

Conversation

Projects
None yet
3 participants
@d98762625
Copy link
Member

commented Apr 20, 2018

Core wrapping of operations into a callable format has initial implementation.

  • Core API functionality exists
  • Added help function which returns info on operation. Currently accepts string, returns one operation's metadata
  • Test capability added

To do:

  • Add Recipe or chef.bake implementation that takes recipe from web app JSON output
  • Help - try and use OperationConfig.json. It could be loaded in statically to avoid import issues we are having
  • return result as Dish - overwrite toString, toJSON and anything elser that console.log, builtin data types work.
  • Remove branching, etc. functions that will not be relevant
  • Export individual operations alongside chef (how to do this without manually declaring each op in the top level export?) e.g. import { fromBase32 } from 'cyberChef'
  • Test type conversion that uses Dish (test chef.translateTo)
  • Ensure errors come out of operations correctly (not just strings)
  • Consider result object, which has type conversion - chef.translateTo added
  • Dont decapitalise all caps functions e.g. ROT13
  • Use Objects for optional args rather than arrays
  • Add node API to docs
  • Add ability to use with callbacks as well as promises

Bugs

  • toBase64 shows no output?
  • crypto-api error when running build from node
  • ES5: help and bake are not part of the chef object - only in .default.

To check before release:

  • Node support (try in later versions than current)
  • One off test of each operation
  • Add link to wiki to docs and update wiki
  • What do we want in our release? src and build? es6 and es5 builds?
  • Diff , Regex and any other visual ops - how to summarise result?
  • Consider changing toggleString arg properties? string could be value
  • Generate REPL index - index could be reduced?

Feedback:

  • help could use search in the same way as the UI. Check for logic. e.g. chef.help("base64") and get a sensible answer.
  • ensure that bake is forgiving: accept "to base 64", "tobase64", etc. Use default args if none specified.

Future issues:

  • Add operation as input to help, so you can do chef.help(chef.toBase32)
  • Add search functionality
  • Add run to help output

@d98762625 d98762625 requested a review from n1474335 Apr 20, 2018

@n1474335 n1474335 added this to the v8.0.0 milestone May 14, 2018

@n1474335 n1474335 referenced this pull request May 26, 2018

Closed

Ciphers not loading #312

@d98762625 d98762625 force-pushed the d98762625:node-lib branch from a2d70d4 to 6084759 May 3, 2019

@d98762625

This comment has been minimized.

Copy link
Member Author

commented May 30, 2019

@n1474335 @j433866 This branch is now ready for final review. I welcome any comments!

  • ES6 style import does work, but we need to use deep imports (import {toHex} from cyberchef/src/node/index). This is documented on the node-lib branch on the wiki.
  • see node-lib branch on wiki to see example uses.

@n1474335 n1474335 changed the base branch from master to v9 Jun 19, 2019

@n1474335
Copy link
Member

left a comment

This looks great. Loads of work has gone into this. I'm really looking forward to using it!

There are really just a couple of things to sort out before this is ready to merge. I imagine that once it starts being used we will want to tweak it a little more, but I think it's largely ready for release.

Show resolved Hide resolved Gruntfile.js Outdated
Show resolved Hide resolved Gruntfile.js Outdated
Show resolved Hide resolved .travis.yml Outdated
Show resolved Hide resolved package.json Outdated
Show resolved Hide resolved src/core/Dish.mjs Outdated
Show resolved Hide resolved src/node/config/scripts/generateNodeIndex.mjs Outdated
Show resolved Hide resolved src/node/config/scripts/generateNodeIndex.mjs Outdated
Show resolved Hide resolved src/core/Utils.mjs Outdated
Show resolved Hide resolved tests/lib/utils.mjs Outdated
Show resolved Hide resolved tests/node/index.mjs Outdated
Show resolved Hide resolved src/node/index.mjs Outdated
@d98762625

This comment has been minimized.

Copy link
Member Author

commented Jul 1, 2019

@n1474335 Thanks for the comments - I'll address them later this week.

Show resolved Hide resolved src/core/Utils.mjs Outdated

@d98762625 d98762625 requested a review from n1474335 Jul 5, 2019

@d98762625

This comment has been minimized.

Copy link
Member Author

commented Jul 5, 2019

Review comments have been address and I sorted out all the conflicts with v9.

@n1474335

This comment has been minimized.

Copy link
Member

commented Jul 5, 2019

@d98762625 Getting some broken node tests:

🔥 Node API: Dish translation: ArrayBuffer and String
	Expected values to be strictly deep-equal:
	+ actual - expected
	
	  ArrayBuffer {
	+   [Uint8Contents]: <73 6f 6d 65 20 73 74 72 69 6e 67>,
	-   [Uint8Contents]: <00 00 00 00 00 00 00 00 00 00 00>,
	    byteLength: 11
	  }
🔥 Node API: Dish translation: ArrayBuffer and number
	Expected values to be strictly deep-equal:
	+ actual - expected
	
	  ArrayBuffer {
	+   [Uint8Contents]: <31 30 30>,
	+   byteLength: 3
	-   [Uint8Contents]: <00 00 00 00 00 00 00 00 00 00>,
	-   byteLength: 10
	  }
✔️️ Node API: Dish translation: ArrayBuffer and byte array
✔️️ Node API: Dish translation: ArrayBuffer and HTML
🔥 Node API: Dish translation: ArrayBuffer and BigNumber
	Expected values to be strictly deep-equal:
	+ actual - expected
	
	  ArrayBuffer {
	+   [Uint8Contents]: <34 30 30 31>,
	+   byteLength: 4
	-   [Uint8Contents]: <00 00 00 00 00 00 00 00 00 00>,
	-   byteLength: 10
	  }
✔️️ Node API: Dish translation: ArrayBuffer and JSON
🔥 Node API: Dish translation: ArrayBuffer and File
	Expected values to be strictly deep-equal:
	+ actual - expected
	
	  ArrayBuffer {
	+   [Uint8Contents]: <61 62 63 64>,
	+   byteLength: 4
	-   [Uint8Contents]: <00 00 00 00 00 00 00 00 00 00>,
	-   byteLength: 10
	  }
🔥 Node API: Dish translation: ArrayBuffer and ListFile
	Expected values to be strictly deep-equal:
	+ actual - expected
	
	  ArrayBuffer {
	+   [Uint8Contents]: <61 62 63 64 65 66 67 68 69 6a 6b>,
	+   byteLength: 11
	-   [Uint8Contents]: <00 00 00 00 00 00 00 00 00 00>,
	-   byteLength: 10
	  }
@d98762625

This comment has been minimized.

Copy link
Member Author

commented Jul 5, 2019

@n1474335 ew OK. What version of node are you running? I (and travis) are running 10.8.0

@n1474335

This comment has been minimized.

Copy link
Member

commented Jul 5, 2019

v11.13.0

I've fixed them 👍

I'm not sure why they were ever passing actually, they certainly shouldn't have been. They were comparing a Dish with data in to a blank ArrayBuffer.

@n1474335 n1474335 merged commit 0de89f3 into gchq:v9 Jul 5, 2019

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
license/cla Contributor License Agreement is signed.
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.