#521 Script Code Evaluation for variables

README.md

@@ -413,12 +413,12 @@ Environments give you the ability to customize requests using variables, and you
 Environments and including variables are defined directly in `Visual Studio Code` setting file, so you can create/update/delete environments and variables at any time you wish. If you __DO NOT__ want to use any environment, you can choose `No Environment` in the environment list. Notice that if you select `No Environment`, variables defined in shared environment are still available. See [Environment Variables](#environment-variables) for more details about environment variables.
 
 ## Variables
-We support two types of variables, one is __Custom Variables__ which is defined by user and can be further divided into __Environment Variables__, __File Variables__ and __Request Variables__, the other is __System Variables__ which is a predefined set of variables out-of-box.
+We support two types of variables, one is __Custom Variables__ which is defined by user and can be further divided into __Environment Variables__, __File Variables__, __Request Variables__  and __Script Variables__, the other is __System Variables__ which is a predefined set of variables out-of-box.
 
 The reference syntax of system and custom variables types has a subtle difference, for the former the syntax is `{{$SystemVariableName}}`, while for the latter the syntax is `{{CustomVariableName}}`, without preceding `$` before variable name. The definition syntax and location for different types of custom variables are different. Notice that when the same name used for custom variables, request variables takes higher resolving precedence over file variables, file variables takes higher precedence over environment variables.
 
 ### Custom Variables
-Custom variables can cover different user scenarios with the benefit of environment variables, file variables, and request variables. Environment variables are mainly used for storing values that may vary in different environments. Since environment variables are directly defined in Visual Studio Code setting file, they can be referenced across different `http` files. File variables are mainly used for representing values that are constant throughout the `http` file. Request variables are used for the chaining requests scenarios which means a request needs to reference some part(header or body) of another request/response in the _same_ `http` file, imagine we need to retrieve the auth token dynamically from the login response, request variable fits the case well. Both file and request variables are defined in the `http` file and only have __File Scope__.
+Custom variables can cover different user scenarios with the benefit of environment variables, file variables, and request variables. Environment variables are mainly used for storing values that may vary in different environments. Since environment variables are directly defined in Visual Studio Code setting file, they can be referenced across different `http` files. File variables are mainly used for representing values that are constant throughout the `http` file. Request variables are used for the chaining requests scenarios which means a request needs to reference some part(header or body) of another request/response in the _same_ `http` file, imagine we need to retrieve the auth token dynamically from the login response, request variable fits the case well. File, script and request variables are defined in the `http` file and only have __File Scope__.
 
 #### Environment Variables
 For environment variables, each environment comprises a set of key value pairs defined in setting file, key and value are variable name and value respectively. Only variables defined in selected environment and shared environment are available to you. You can also reference the variables in shared environment with `{{$shared variableName}}` syntax in your active environment. Below is a sample piece of setting file for custom environments and environment level variables:
@@ -532,6 +532,40 @@ GET {{baseUrl}}/comments/{{commentId}}/replies/{{getReplies.response.body.//repl
 
 ```
 
+### Script Variables
+With script variables it is possible to include JS code snippets. JS Code Snippets compiles to a CommonJS Module. JS Code Snippets support injection of all already defined variables. Request variables must be included by only name. A [require](https://nodejs.org/api/modules.html#modules_require_id) function is automatically provided.
+
+```http
+
+@currentDate = {{() => new Date()}}
+
+@authToken = {{(send) => require('logon.js')(send)}}
+
+# @name createComment
+POST {{baseUrl}}/comments HTTP/1.1
+Authorization: {{authToken}}
+Content-Type: application/json
+
+{
+    "content": "fake content"
+}
+
+###
+
+@commentId = {{(createComment, authToken) => createComment.response.body.$.id}}
+
+```
+
+All Script Variables in requests are evaluated each time. Script Variables assigend to file variables are cached.
+
+```http
+@cached = {{() => new Date()}}
+
+###
+GET {{baseUrl}}/comments?notcached={{() => new Date()}} HTTP/1.1
+
+```
+
 ### System Variables
 System variables provide a pre-defined set of variables that can be used in any part of the request(Url/Headers/Body) in the format `{{$variableName}}`. Currently, we provide a few dynamic variables which you can use in your requests. The variable names are _case-sensitive_.
 * `{{$aadToken [new] [public|cn|de|us|ppe] [<domain|tenantId>] [aud:<domain|tenantId>]}}`: Add an Azure Active Directory token based on the following options (must be specified in order):
@@ -550,7 +584,7 @@ System variables provide a pre-defined set of variables that can be used in any
   `appOnly`: Optional. Specify `appOnly` to use make to use a client credentials flow to obtain a token. `aadV2ClientSecret` and `aadV2AppUri`must be provided as REST Client environment variables. `aadV2ClientId` and `aadV2TenantId` may also be optionally provided via the environment. `aadV2ClientId` in environment will only be used for `appOnly` calls.
 
   `scopes:<scope[,]>`: Optional. Comma delimited list of scopes that must have consent to allow the call to be successful. Not applicable for `appOnly` calls.
-  
+
   `tenantId:<domain|tenantId>`: Optional. Domain or tenant id for the tenant to sign in to. (`common` to determine tenant from sign in).
 
   `clientId:<clientid>`: Optional. Identifier of the application registration to use to obtain the token. Default uses an application registration created specifically for this plugin.

src/models/variableType.ts

@@ -3,4 +3,5 @@ export enum VariableType {
     File,
     Request,
     System,
+    Script
 }

src/utils/httpVariableProviders/fileVariableProvider.ts

@@ -6,6 +6,7 @@ import { VariableType } from '../../models/variableType';
 import { EnvironmentVariableProvider } from './environmentVariableProvider';
 import { HttpVariable, HttpVariableProvider } from './httpVariableProvider';
 import { RequestVariableProvider } from './requestVariableProvider';
+import { ScriptVariableProvider } from './scriptVariableProvider';
 import { SystemVariableProvider } from './systemVariableProvider';
 
 type FileVariableValue = Record<'name' | 'value', string>;
@@ -29,6 +30,7 @@ export class FileVariableProvider implements HttpVariableProvider {
 
     private readonly innerVariableProviders: HttpVariableProvider[] = [
         SystemVariableProvider.Instance,
+        ScriptVariableProvider.Instance,
         RequestVariableProvider.Instance,
         EnvironmentVariableProvider.Instance,
     ];

src/utils/httpVariableProviders/scriptVariableProvider.ts

@@ -0,0 +1,216 @@
+
+
+import {OutgoingHttpHeaders} from 'http';
+import Module from 'module';
+import { dirname } from 'path';
+import vm from 'vm';
+import { TextDocument } from 'vscode';
+import { DocumentCache } from '../../models/documentCache';
+import { HttpRequest } from '../../models/httpRequest';
+import { VariableType } from '../../models/variableType';
+import { HttpClient } from '../httpClient';
+import { RequestVariableCache } from '../requestVariableCache';
+import { EnvironmentVariableProvider } from './environmentVariableProvider';
+import { FileVariableProvider } from './fileVariableProvider';
+import { HttpVariable, HttpVariableContext, HttpVariableProvider } from './httpVariableProvider';
+import { SystemVariableProvider } from './systemVariableProvider';
+
+interface ScriptVariableFunction {
+  name: string;
+  function: string;
+  args: string[];
+  values: any[];
+  result?: HttpVariable;
+}
+
+const FunctionReqex = /function\s?\(([0-9a-zA-z ,]*)\)\s?{(.*)}/;
+const ArrowFunctionRegex = /\(?\s?([0-9a-zA-z ,]*)\)?\s?=>\s?(.*)?/;
+
+export class ScriptVariableProvider implements HttpVariableProvider {
+
+  private providerRequestCache = {};
+
+  public readonly type: VariableType = VariableType.Script;
+
+  private readonly scriptVariableCache = new DocumentCache<Map<string, ScriptVariableFunction>>();
+
+  private static _instance: ScriptVariableProvider;
+
+  public static get Instance(): ScriptVariableProvider {
+    if (!this._instance) {
+      this._instance = new ScriptVariableProvider();
+    }
+    return this._instance;
+  }
+
+  public async has(name: string, document?: TextDocument, context?: HttpVariableContext | undefined): Promise<boolean> {
+    return !!this.parseScript(name);
+  }
+  public async get(name: string, document: TextDocument, context: HttpVariableContext): Promise<HttpVariable> {
+      const script = await this.parseCachedScript(name, document, context);
+      if (script && document) {
+        if (script.result) {
+          return script.result;
+        }
+        script.result = await this.executeScript(script, script.values, document.fileName);
+        return script.result;
+      } else {
+        return {
+          name,
+          error: 'script variable not valid',
+        };
+      }
+    }
+  public async getAll(document?: TextDocument, context?: HttpVariableContext | undefined): Promise<HttpVariable[]> {
+    return [];
+  }
+
+  private parseScript(name: string): ScriptVariableFunction | null {
+  const matches = ArrowFunctionRegex.exec(name) || FunctionReqex.exec(name);
+  if (matches && matches.length === 3) {
+    return {
+      name,
+      function: matches[2],
+      args: matches[1]
+        .split(',')
+        .map(obj => obj.trim())
+        .filter(obj => obj !== ''),
+      values: []
+    };
+  }
+    return null;
+  }
+
+  private async parseCachedScript(name: string, document: TextDocument, context: HttpVariableContext): Promise<ScriptVariableFunction | null> {
+    const script = this.parseScript(name);
+
+    if (script) {
+      for (const arg of script.args) {
+        script.values.push(await this.getScriptArgValue(arg, document, context));
+      }
+      // only cache on filevariables
+      if (context.parsedRequest === '') {
+        const cacheMap = this.getDocumentCache(document);
+        const cachedScript = cacheMap.get(name);
+
+        if (cachedScript
+          && this.arrayEquals(script.args, cachedScript.args)
+          && this.arrayEquals(script.values, cachedScript.values)) {
+          return cachedScript;
+        }
+
+        cacheMap.set(name, script);
+      }
+    }
+    return script;
+  }
+
+
+  private async executeScript(script: ScriptVariableFunction, scriptArgValues: any[], fileName: string): Promise<HttpVariable> {
+    try {
+      const wrappedArgs = ['exports', 'require', 'module', '__filename', '__dirname', ...script.args];
+
+      let moduleExports = `module.exports = ${script.function};`;
+      if (script.function.indexOf("return ") >= 0) {
+        moduleExports = script.function.replace("return ", "module.exports =");
+      }
+      const wrappedFunction = `(function (${wrappedArgs.join(',')}){
+          ${moduleExports}
+        })`;
+      const dir = dirname(fileName);
+      const scriptModule = new Module(fileName, module.parent as Module);
+      scriptModule.filename = fileName;
+      scriptModule.exports = {};
+      // see https://github.com/nodejs/node/blob/master/lib/internal/modules/cjs/loader.js#L565-L640
+      scriptModule.paths = (Module as any)._nodeModulePaths(dir);
+      const compiledWrapper = vm.runInThisContext(wrappedFunction, {
+        filename: fileName,
+        lineOffset: 0,
+        displayErrors: true
+      });
+      const scripteRequire: any = (id) => scriptModule.require(id);
+      // see https://github.com/nodejs/node/blob/master/lib/internal/modules/cjs/loader.js#L823-L911
+      scripteRequire.resolve = req => (Module as any)._resolveFilename(req, scriptModule);
+      compiledWrapper.apply(scriptModule.exports, [scriptModule.exports, scripteRequire, scriptModule, fileName, dir, ...scriptArgValues]);
+
+      let value = scriptModule.exports;
+      if (this.isPromise(scriptModule.exports)) {
+        value = await scriptModule.exports;
+      }
+      module.parent?.children.splice(module.parent.children.indexOf(scriptModule), 1);
+
+      return {
+        name: script.name,
+        value
+      };
+    } catch (error) {
+      return {
+        name: script.name,
+        error,
+      };
+    }
+  }
+
+  private getDocumentCache(document: TextDocument) {
+    let map = this.scriptVariableCache.get(document);
+    if (!map) {
+      map = new Map<string, ScriptVariableFunction>();
+      this.scriptVariableCache.set(document, map);
+    }
+    return map;
+  }
+
+  private async getScriptArgValue(arg: string, document: TextDocument, context: HttpVariableContext) {
+    if (arg === "send") {
+      return this.send;
+    }
+
+    const result = RequestVariableCache.get(document, arg);
+    if (result) {
+      return {
+        request: result.request,
+        response: result,
+      };
+    }
+    const providers = [FileVariableProvider.Instance,
+    SystemVariableProvider.Instance,
+    EnvironmentVariableProvider.Instance
+    ];
+    for (const provider of providers) {
+      if (await provider.has(arg, document)) {
+        if (!this.providerRequestCache[arg]) {
+          const promise = provider.get(arg, document, context);
+          this.providerRequestCache[arg] = promise;
+          const result = await promise;
+          delete this.providerRequestCache[arg];
+          return result.value;
+        }
+      }
+    }
+    return result;
+  }
+
+  private isPromise(obj: any): obj is Promise < any > {
+    return obj && obj.then;
+  }
+
+  private arrayEquals(a: any[], b: any[]) {
+    return Array.isArray(a) &&
+      Array.isArray(b) &&
+      a.length === b.length &&
+      a.every((val, index) => val === b[index]);
+  }
+
+  private send(request: {
+    method: string,
+    url: string,
+    headers: OutgoingHttpHeaders,
+    body: string | undefined,
+    rawBody?: string | undefined,
+    name?: string | undefined
+  }) {
+    const client = new HttpClient();
+    return client.send(new HttpRequest(request.method || 'GET', request.url, request.headers, request.body, request.rawBody, request.name));
+  }
+
+}

src/utils/variableProcessor.ts

@@ -4,13 +4,15 @@ import { EnvironmentVariableProvider } from './httpVariableProviders/environment
 import { FileVariableProvider } from './httpVariableProviders/fileVariableProvider';
 import { HttpVariableProvider } from './httpVariableProviders/httpVariableProvider';
 import { RequestVariableProvider } from './httpVariableProviders/requestVariableProvider';
+import { ScriptVariableProvider } from './httpVariableProviders/scriptVariableProvider';
 import { SystemVariableProvider } from './httpVariableProviders/systemVariableProvider';
 import { getCurrentTextDocument } from './workspaceUtility';
 
 export class VariableProcessor {
 
     private static readonly providers: [HttpVariableProvider, boolean][] = [
         [SystemVariableProvider.Instance, false],
+        [ScriptVariableProvider.Instance, false],
         [RequestVariableProvider.Instance, true],
         [FileVariableProvider.Instance, true],
         [EnvironmentVariableProvider.Instance, true],
@@ -55,40 +57,19 @@ export class VariableProcessor {
     }
 
     public static async getAllVariablesDefinitions(document: TextDocument): Promise<Map<string, VariableType[]>> {
-        const [, [requestProvider], [fileProvider], [environmentProvider]] = this.providers;
-        const requestVariables = await (requestProvider as RequestVariableProvider).getAll(document);
-        const fileVariables = await (fileProvider as FileVariableProvider).getAll(document);
-        const environmentVariables = await (environmentProvider as EnvironmentVariableProvider).getAll();
-
         const variableDefinitions = new Map<string, VariableType[]>();
-
-        // Request variables in file
-        requestVariables.forEach(({ name }) => {
-            if (variableDefinitions.has(name)) {
-                variableDefinitions.get(name)!.push(VariableType.Request);
-            } else {
-                variableDefinitions.set(name, [VariableType.Request]);
-            }
-        });
-
-        // Normal file variables
-        fileVariables.forEach(({ name }) => {
-            if (variableDefinitions.has(name)) {
-                variableDefinitions.get(name)!.push(VariableType.File);
-            } else {
-                variableDefinitions.set(name, [VariableType.File]);
-            }
-        });
-
-        // Environment variables
-        environmentVariables.forEach(({ name }) => {
-            if (variableDefinitions.has(name)) {
-                variableDefinitions.get(name)!.push(VariableType.Environment);
-            } else {
-                variableDefinitions.set(name, [VariableType.Environment]);
+        for (const [httpVariableProvider, caching] of this.providers) {
+            if (caching) {
+                const variables = await httpVariableProvider.getAll(document);
+                variables.forEach(({ name }) => {
+                    if (variableDefinitions.has(name)) {
+                        variableDefinitions.get(name)!.push(httpVariableProvider.type);
+                    } else {
+                        variableDefinitions.set(name, [httpVariableProvider.type]);
+                    }
+                });
             }
-        });
-
+        }
         return variableDefinitions;
     }
 }