Support hyphen `-` and other special characters in variable names #2601

develohpanda · 2020-09-11T10:35:15Z

Updating variable access logic to work with {{ regular }} (which is current behavior) as well as {{ _['base-url'] }}, which caters for dashes and special characters in the key.

Nunjucks templates support JS square bracket notation out of the box, but not as the first node. For example, {{ ['base-url'] }} is invalid (as it is in JS) while {{ variable['base-url'] }} is valid. For this reason, we have exposed the template context object (which is provided to Nunjucks), on the variable _, allowing you to access root keys that contain dashes or special characters, like {{ _['base-url'] }}.

The context object is exposed under _ in addition to the current implementation, so the old syntax (without _) will still work, when importing workspaces, or upgrading versions of Insomnia, for example.

I selected _ somewhat arbitrarily. If we would like to change that to something like _ctx, _env, $, or a different character, now is the best time to chime in. _ feels least invasive.

The autocomplete drop-down has been updated to always prefix an environment variable with _, to keep things consistent. It will also only use square bracket notation where necessary. If there are no special characters, regular dot notation will be used, now prefixed with _.

Old left, new right, both work

Old returns NaN, new returns the correct variable

Hiding the _. prefix or the bracket syntax from the user in the nice rendered tag view is doable, and shouldn't be too tricky, but it would be somewhat misleading. Open to thoughts on that, but I will not change in this PR, it will be changed in a future PR.

Additionally, NeDB has rules to prevent a key from starting with $ or contain a ., so I have updated the regex key check for the environment editor. In the current implementation, errors are presented in console, and the changes do not persist (semi-silent failure), if you try to save something without following these rules which the existing regex did not check for.

Screenshots

Environment

Autocomplete dropdown

Tags

Raw syntax

Not found errors

Variable editor modal

Regex Checks

Most importantly, this doesn't impact any existing tags

TODO:

Extract reusable regex and hard-coded strings into constants
Ensure folder environment editor handles hyphen the expected way
Double check all regex related changes
Add unit tests where possible

To test:

Ensure export/import works
Check importers (postman)
Check openapi import
Ensure environment editor presents appropriate errors for invalid keys
Ensure folder environment editor presents appropriate errors for invalid keys

Fixes #1666, Fixes #456, Fixes #1960, Related to #2051, Fixes INS-118

packages/insomnia-app/app/templating/index.js

packages/insomnia-app/app/ui/components/templating/tag-editor.js

rhukster · 2020-09-11T11:46:09Z

👍 as a developer who works with Twig on a daily basis, I was going to suggest a custom filter to handle the dashes and it seems exactly the approach you have already gone with. I think your solution is solid. Retains template capabilities, UI mostly hides the filter so it does not impact usability. Really looking forward to this.

packages/insomnia-app/app/ui/components/codemirror/extensions/nunjucks-tags.js

gschier · 2020-09-11T16:06:52Z

Another option I've thought of in the past is to expose the root environment under $ (or something similar) just before rendering, then use square bracket syntax like {{ $['foo']['bar-haha+blah woo!']['baz'] }} instead of dot notation. You wouldn't need an extra filter and it works out of the box with nested properties.

As @rhukster mentioned, though, the UI hides it all so it doesn't really matter what it looks like as long as the implementation is solid and it's not a breaking change 👍

rhukster · 2020-09-11T18:30:01Z

Another option I've thought of in the past is to expose the root environment under $ (or something similar) just before rendering, then use square bracket syntax like {{ $['foo']['bar-haha+blah woo!']['baz'] }} instead of dot notation. You wouldn't need an extra filter and it works out of the box with nested properties.

Yah in Twig the option is there to use _context['foo']['bar-haha+blah woo!']['baz'] in the same way, I just didn't realize how similar Nunjucks was. The whole _context and square brackets is slightly uglier though, but if it's hidden, it really it comes down to what fits best for Insomnia.

develohpanda · 2020-09-11T23:20:58Z

I will explore this option as well 👍

Either way, the template and the UI will need to be different, so if the feature set is identical between the two approaches I would favour the custom filter still, as it feels a bit clearer/easier to explain. Although context provided by a $ also feels good. Will ponder 🤔

linear · 2020-09-15T05:16:36Z

INS-118 Support hyphens in Insomnia environment variable keys

Fixes #1666, Fixes #456, Fixes #1960, Fixes #2051

…rt-dash-in-variable-name

netlify · 2020-10-01T06:00:39Z

Deploy preview for insomnia-storybook ready!

Built with commit a9c827e

https://deploy-preview-2601--insomnia-storybook.netlify.app

develohpanda

Some comments for reviewers.

develohpanda · 2020-10-01T06:00:36Z

packages/insomnia-app/app/common/render.js

@@ -276,28 +277,30 @@ export async function getRenderContext(
  function getKeySource(subObject, inKey, inSource) {
    // Add key to map if it's not root
    if (inKey) {
-      keySource[inKey] = inSource;
+      keySource[templatingUtils.normalizeToDotAndBracketNotation(inKey)] = inSource;


The changes in this file relate to building the key source lookup, so we can know if data is coming from the base environment, a sub environment, or a folder. The notation needs to be the same as what the template tag itself contains, hence uses the forceBracket and normalize functions used by the key indexing logic.

develohpanda · 2020-10-01T06:04:13Z

packages/insomnia-app/app/ui/components/codemirror/extensions/nunjucks-tags.js

+      const contextForKey = con.keyContext[cleanedStr];
+
+      // Only prefix the title with context, if context is found
+      title = contextForKey ? `{${contextForKey}}: ${title}` : title;


contextForKey is root, or the sub environment, or the folder name, etc

develohpanda · 2020-10-01T06:05:00Z

packages/insomnia-app/app/ui/components/editors/__tests__/environment-editor.test.js

+  it.each(['a', 'ab', 'a$', 'a$b', 'a-b', `a${name}b`, `${name}ab`])('%s should be valid', key => {
+    expect(ensureKeyIsValid(key)).toBe(null);
+  });
+});


Tests to ensure a key cannot be _, or contain ., or start with $.

develohpanda · 2020-10-01T06:07:28Z

packages/insomnia-app/app/ui/containers/app.js

@@ -430,7 +431,7 @@ class App extends PureComponent {

  async _handleGetRenderContext() {
    const context = await this._fetchRenderContext();
-    const keys = getKeys(context);
+    const keys = getKeys(context, NUNJUCKS_TEMPLATE_GLOBAL_PROPERTY_NAME);


getKeys is recursive, and is defined in templating/utils.js. The second argument reflects the key prefix, as the algorithm traverses all nodes. We now start the traversal with _, so all keys are prefixed by _.

develohpanda · 2020-10-01T06:09:09Z

packages/insomnia-app/app/ui/components/editors/environment-editor.js

    // Check for invalid key names
+    // TODO: these only check root properties, not nested properties
    if (value && value.object) {
      for (const key of Object.keys(value.object)) {
-        if (!key.match(/^[a-zA-Z_$][0-9a-zA-Z_$]*$/)) {
-          warning = `"${key}" must only contain letters, numbers, and underscores`;
+        error = ensureKeyIsValid(key);
+        if (error) {
          break;
        }
      }
    }


There is a bug here in that both previously and now, only the root keys are checked. A recursive algorithm needs to be created to validate nested keys appropriately.

sonicyeti

Ultra comprehensive, great work!

koldoon · 2021-07-01T13:02:16Z

IMO there is nothing wrong to simply remove _. prefix. In most cases it looks dirty and doesn't really help, so it should be removed everywhere when possible.

develohpanda added 2 commits September 11, 2020 21:56

Support - in variable names

283930c

Cater for nunjucks tags

32c1411

develohpanda changed the title ~~Support '-' in variable names~~ Support hyphen - in variable names Sep 11, 2020

develohpanda mentioned this pull request Sep 11, 2020

[Usability] Dash in environment config key resolves value to NaN #456

Closed

develohpanda commented Sep 11, 2020

View reviewed changes

packages/insomnia-app/app/templating/index.js Outdated Show resolved Hide resolved

packages/insomnia-app/app/ui/components/templating/tag-editor.js Outdated Show resolved Hide resolved

develohpanda self-assigned this Sep 11, 2020

develohpanda mentioned this pull request Sep 11, 2020

Environmental Variable imports from postman evaluate to "NaN" #1666

Closed

gschier reviewed Sep 11, 2020

View reviewed changes

packages/insomnia-app/app/ui/components/codemirror/extensions/nunjucks-tags.js Outdated Show resolved Hide resolved

develohpanda and others added 3 commits September 22, 2020 18:19

WIP

86d51d1

Merge branch 'develop' of github.com:Kong/insomnia into feature/suppo…

6eeb542

…rt-dash-in-variable-name

Use JSON path

83203db

This comment has been minimized.

Sign in to view

Merge branch 'develop' into feature/support-dash-in-variable-name

2242670

This comment has been minimized.

Sign in to view

develohpanda added 6 commits September 29, 2020 19:16

Support dash

5e34ef8

Merge branch 'develop' of github.com:Kong/insomnia into feature/suppo…

8cf346e

…rt-dash-in-variable-name

Resolve nunjucks variables via a global variable

1a21e97

Undo more filter logic

4f47562

update comment

5df4412

Update function names

0cffa15

This comment has been minimized.

Sign in to view

develohpanda added 4 commits September 30, 2020 15:13

Show error if reserved key is selected

0b1f814

Remove warning logic from environment editor

5fd5443

Warn if invalid nedb key

cce5c1e

Warn if invalid nedb key

073f56e

develohpanda added 5 commits September 30, 2020 17:03

add unit tests for invalid key

6db3ab0

Update to _ as the key root

b3c135c

update tests

2f56f1c

Update warnings

af9c0f8

Merge branch 'develop' of github.com:Kong/insomnia into feature/suppo…

a9c827e

…rt-dash-in-variable-name

develohpanda commented Oct 1, 2020

View reviewed changes

develohpanda marked this pull request as ready for review October 1, 2020 06:10

develohpanda requested review from DMarby and sonicyeti October 1, 2020 06:10

develohpanda changed the title ~~Support hyphen - in variable names~~ Support hyphen - and other special characters in variable names Oct 1, 2020

DMarby approved these changes Oct 1, 2020

View reviewed changes

sonicyeti approved these changes Oct 1, 2020

View reviewed changes

develohpanda merged commit 1bbbf38 into develop Oct 1, 2020
1 check passed

develohpanda deleted the feature/support-dash-in-variable-name branch October 1, 2020 18:49

This was referenced Oct 1, 2020

Validate environment variables against nested keys #2684

Closed

Kebab-case environment variable is NaN #2399

Closed

develohpanda mentioned this pull request Nov 26, 2020

Breaking change in env variable auto-completion in 2020.5.0-beta.2 #2872

Closed

develohpanda mentioned this pull request Dec 15, 2020

unexpected "underscore and dot" at beginning of preset values #2942

Closed

nijikokun mentioned this pull request Mar 3, 2021

All environment variables prefixed with _. and existing ones changed to custom #3139

Closed

develohpanda mentioned this pull request Mar 17, 2021

[Feature Request] Remove "_." prefix when displaying or accessing variables #3064

Open

nijikokun mentioned this pull request Mar 23, 2021

Environment variable with hyphen/dash symbol not working #3213

Closed

Support hyphen `-` and other special characters in variable names #2601

Support hyphen `-` and other special characters in variable names #2601

develohpanda commented Sep 11, 2020 •

edited

rhukster commented Sep 11, 2020

gschier commented Sep 11, 2020 •

edited

rhukster commented Sep 11, 2020

develohpanda commented Sep 11, 2020 •

edited

linear bot commented Sep 15, 2020

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

netlify bot commented Oct 1, 2020

develohpanda left a comment

develohpanda Oct 1, 2020

develohpanda Oct 1, 2020

develohpanda Oct 1, 2020

develohpanda Oct 1, 2020 •

edited

develohpanda Oct 1, 2020

sonicyeti left a comment

koldoon commented Jul 1, 2021

Support hyphen - and other special characters in variable names #2601

Support hyphen - and other special characters in variable names #2601

Conversation

develohpanda commented Sep 11, 2020 • edited

Screenshots

rhukster commented Sep 11, 2020

gschier commented Sep 11, 2020 • edited

rhukster commented Sep 11, 2020

develohpanda commented Sep 11, 2020 • edited

linear bot commented Sep 15, 2020

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

netlify bot commented Oct 1, 2020

develohpanda left a comment

Choose a reason for hiding this comment

develohpanda Oct 1, 2020

Choose a reason for hiding this comment

develohpanda Oct 1, 2020

Choose a reason for hiding this comment

develohpanda Oct 1, 2020

Choose a reason for hiding this comment

develohpanda Oct 1, 2020 • edited

Choose a reason for hiding this comment

develohpanda Oct 1, 2020

Choose a reason for hiding this comment

sonicyeti left a comment

Choose a reason for hiding this comment

koldoon commented Jul 1, 2021

Support hyphen `-` and other special characters in variable names #2601

Support hyphen `-` and other special characters in variable names #2601

develohpanda commented Sep 11, 2020 •

edited

gschier commented Sep 11, 2020 •

edited

develohpanda commented Sep 11, 2020 •

edited

develohpanda Oct 1, 2020 •

edited