root folder determination

[gerukin]

In the extension it seems that the httpyac root folder is always assumed to be vscode's root folder. So the extension looks for an env folder, or a package.json with httpyac key, or a .httpyac.json file there. I think this works fine in a lot of cases but it may break down in a few (ex: monorepo or strict hierarchical structure).

Example

I recently tried something like this:

|__ tests
   |__ http
      |__ env
      |  |__ .env
      |__ requests
      |  |__ something.http
      |__ package.json

In this structure, it seems the CLI will have no issues if run from /tests/http as it will assume this is its root folder (I think), but vscode will only look in its own project root so that the options have to be configured there:

|__ tests
|  |__ ...
|__ .httpyac.json

Personally I would prefer to keep everything in the /tests/http folder in this case.

Suggestion

This works but if at all possible, it would be nice to do something like this for vscode:

1. look into the current folder for config settings and defaults
2. if not found, look one folder up
3. until reaching vscode's project root (and then give up)

I realize this may be confusing to the extension if we have something like this:

|__ monorepo
   |__ tests1
      |__ env
      |  |__ .env
      |__ requests
      |__ .httpyac.json
   |__ tests2
      |__ env
      |  |__ .env
      |__ requests
      |__ .httpyac.json

When running something from tests1 no problem.. tests1's settings will apply, but then when running something from tests2 perhaps vscode-httpyac will not be sure what to do (which environments to show and/or apply). If it is possible to namespace and fully isolate when 2 or more configs are found this would be my preference, so that anything from tests1 is invisible to tests2... and when looking at a file in tests2 nothing from tests1 one shows or applies.

[linkdesu]

Wow, I have the same requirement! It seems a good chance for me to learn some vscode plugin development. 😂

[AnWeber]

I did not take care to separate the environment settings correctly in my implementation of vscode-httpyac. I have found this error before in workspace projects, but it also applies to this requirement.
I also want to avoid searching for the env files every time I execute a request. VS Code recently supports remote development and the File Magic would possibly delay the execution unnecessarily.
Your initial example could be solved using the httpyac.dotenvDirname setting, but this is not possible for the further example.
I will think again about the environment settings and the .httpyac.json.

@linkdesu I think it's great if you want to actively support me. I'm just afraid that it's not a good example to start with. I have unfortunately the access to the environementStore scattered and also not cleanly separated between program setting and execution settings.

[gerukin]

Thank you @AnWeber.

Yes, regarding the first example I had been using the dotenvDirName property, and it's been working fine. But it also means I have to keep .httpyac.json at the root of the project as a result, which is what I am trying to avoid ideally.

I also want to avoid searching for the env files every time I execute a request.

That makes sense. I don't know how much overhead this really represents.

One thing you may be able to do is just scan once when the http file is opened.

1. file opened -> do you already have a registered root part of this path? if yes you're done, if not..
2. scan the path to register a new root folder

Maybe 🤔. Anyway, you know what vscode extensions can / should do better than I so I'll let you do the thinking. 😇

[AnWeber]

Support for disconnected environments causes problems in some places:

• cookieStore is a singleton and needs to be environment related (possibly already a bug, this is a good first issue to contribute)
• userSessionStore (used for OAuth2 tokens) is also singleton, but notes server url and user name/ clientId and therefore maybe not faulty
• env and intellij environment files need to be searched, but performance impact should be minimal (test needed). I do not want to cache the files again, because the necessary reset of the cache was not found several times already.
• My biggest concern currently is the use of .httpyac.json. In this file application settings (log level) and execution settings (request, header, proxy, ...) are mixed up. However, since probably only the execution settings are needed, this should not be a problem. I will just ignore other settings and let vscode internal settings win.

I think the direction is good, but it will take a little while.

[AnWeber]

Just a quick update. I already have a working stand locally that searches the folders correctly via tree traversal. There also doesn't seem to be any negative performance impact when parsing or executing.
Since I have a few breaking changes in the configuration (remove dotenv and intellij settings), I don't want to ship it until version 3.0. For this one I would still like to include a hook api.

[gerukin]

Excellent! Sounds like a 3.0 makes the most sense indeed. Monorepo support and hooks are nice major features to have. 👍

Is the disabling of env / IntelliJ env settings temporary or will you no longer support them at all from 3.0 on?

---

To fully turn POSTman into PASTman in some of my more complicated projects, I plan on testing query dependency (query A and B referencing query X which has to run first and export / set some variables) next week.

Since I want to share code between vscode and IntelliJ users, I will also test the limits of IntelliJ... but it looks like it's just too primitive with no ability to run pre-request code, no dependent queries, and maybe (I need to check) no ability to require. That last one would be a deal breaker for sure for me. If I can't require with IntelliJ I'll just have to ask people to use the CLI only.

[AnWeber]

I no longer see the benefit in the settings (only httpyac configuration setting, not support for dotenv or intellij). I would have now dotenv and Intellij configurations enabled by default and based on the presence of such a file, detected the root.

I don't think require is available in Intellij. It is not mentioned in the docs. Dependent queries are only supported, if you manually ececute all queries in the right order.

[gerukin]

I no longer see the benefit in the settings (only httpyac configuration setting, not support for dotenv or intellij). I would have now dotenv and Intellij configurations enabled by default and based on the presence of such a file, detected the root.

Makes sense. Even now I actually don't use those settings anyway.

Dependent queries are only supported, if you manually ececute all queries in the right order.

Manually executing in the right order from the editor is essentially what I used to do in rest-client and what people would have to do in IntelliJ. Not ideal but it works indeed.

I don't think require is available in Intellij. It is not mentioned in the docs.

Yes, I was worried when I saw no mention of it and only very simple examples in their docs... but it doesn't necessarily mean it doesn't work. I'll try next week. That honestly would be my biggest problem. IntelliJ's support of > filename.js is a redeeming feature, but it's not flexible enough... and not being able to use libraries like jsonschema would be too limiting. I will probably just set things up so IntelliJ users can at best run the queries to easily see what endpoints return.. but any testing (or writing / testing the tests) will require the httpyac CLI or vscode.

[AnWeber]

Since according to Intellij documentation ECMAScript 5.1 is supported and the Idea is developed with Java, I expect Java Nashorn to be used. And in the youtrack of Idea also Nashorn is mentioned (IDEA-241576). And Nashorn does not support require and anything that goes in that direction is not nice and unfortunately I speak from experience.

and not being able to use libraries like jsonschema would be too limiting

Do you use json schema for simple validation in the test cases? I was wondering myself if this is useful and easy to do, but I don't have much experience with JSON schema. However, my concerns are more about how easily necessary JSON schemas can be generated.

[gerukin]

IMO a well defined JSONschema can be really useful. It can both help generate the open API style documentation for using the endpoints, and also feed the tests. The spec is so flexible that I find that the JSONschema replaces most (often all) tests I would manually write for the endpoint.

For JSONschema generally I do something like this:

{{
  // centralizes required stuff and makes all tests available here
  module.exports = require('../../tests/')
}}

### Endpoint to test
{{API_URL}}/path/to/stuff

{{
  // I generally test the status with an `is`, `any` or `between` test
  tests.status.is({ test, response, val: 200 })

  // and then test the schema based on a list of schemas in an open API spec
  tests.schema.is({ test, response, name: 'list' })
}}

The schemas are either downloaded or taken from the file system within the open API doc. So there are a bunch of functions for that..

const validate = require('jsonschema').validate

// functions to resolve paths and in-schema references to other files

module.exports.is = async ({ test, response, name }) => {
  // ex: this is where my open API spec lives when present in the repo
  const schema = require(`${baseRespSchemaDir}/${name}.json`)

  test('Schema is valid', function () {
    const validation = validate(
      response.parsedBody,
      convertRefs(schema, baseRespSchemaDir),  // the schema object goes here (custom function because I often deeply divide and nest my schemas)
      { required: true }
    )
    if (validation.valid) return true

    const error = new Error('Response schema is incorrect: ' + validation.errors[0].message)
    error.name = 'SCHEMA_ERR'
    throw error
  })
}

I usually like to keep schema files small and hierarchical like this:

// simple health check endpoint
// schemas/responses/status-ok.json
{
  "type": "object",
  "properties": {
    "status": { "type": "string", "const": "ok" }
  },
  "required": ["status"]
}

// simple list object with dynamic keys
// schemas/objects/list/index.json
{
  "type": "object",
  "patternProperties": {
    "^[A-Z]{3}$": {
      "$ref": "../something/index.json"
    }
  },
  "minProperties": 100
}

// the `something` referenced in the previous schema
// schemas/objects/something/index.json
{
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "last_update": {
      "type": "string",
      "pattern": "^20[1-9][0-9]-(([1][0-2])|([0][0-9]))-[0-3][0-9]$"
    }
  },
  "required": ["name"]
}

[AnWeber]

I will use this week to expand my documentation. The extension has already passed my first test. But I will use the version for one more week at work. You can install them using this guide.

vscode-httpyac-3.0.0-next.0.zip

[AnWeber]

I have now created a separate page for the documentation: https://httpyac.github.io/.
The last few days I have also already used the new version and have not noticed any more serious problems. For the reason I have published this as 3.0.
Through this version, starting from the opened http file, the project root is always searched. Here is described how this is determined.