form-data Requests with files is not saving src when exported to collection

[aksbenz]

App Details:

Postman for Windows
Version 6.0.9
win32 6.1.7601 / x64

Issue Report:

1. Observed since v6
2. Expected behavior:
   On exporting a collection with form-data requests having files, src attribute should be populated with the file name.
3. Actual behavior:
   src attribute is not present in exported collection.
   Example:
   "body": {
   "mode": "formdata",
   "formdata": [
   {
   "key": "document",
   "description": "",
   "type": "file"
   }
   ]
   }

Steps to reproduce the problem:

1. Add a post request.
2. Select body type as form-data
3. Add FILE type item to body and select a file from filesystem
4. Save request
5. Export collection

Temp.postman_collection.json.zip

[aksbenz]

@sivcan @shamasis @prashantagarwal
Please have a look. Thanks.

[glenpike]

Duplicate of #3740 ?

[aksbenz]

@glenpike , the issues seems to be the same, but this was working perfectly fine before with the last version before 6. The src had the file name, like this:

"formdata": [{
		"key": "document",
		"value": null,
		"type": "file",
		"src": "SampleImage.jpg"
	}
]

and this would run perfectly through newman.

But since version 6 there is no src attribute at all.

[glenpike]

Hi, Yes, it's been around for a couple of months - if you import a file used for Newman, then re-export it, the 'src' attribute is dropped. It's annoying because we have to diff our file then re-insert all the missing ones. It's also annoying because importing it never affected the file chooser in Postman, so getting rid of it is either a bug or a dumb choice and it's still not fixed...

…

On 12 March 2018 22:01:39 GMT+00:00, aksbenz ***@***.***> wrote: @glenpike , the issues seems to be the same, but this was working perfectly fine before with the last version before 6. The src had the file name, like this: ``` "formdata": [{ "key": "document", "value": null, "type": "file", "src": "SampleImage.jpg" } ] ``` and this would run perfectly through newman. But since version 6 there is no src attribute at all. -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #4320 (comment)

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.

[harryi3t]

It an expected behaviour. We don't save the file path for security reasons.

[aksbenz]

I Never mentioned the file path.....
Your own docs say that it is supposed to have the filename in src and the file with that name should be present at the location where newman is executed from.

That is what I am talking about. Even that is not happening currently. On exporting the collection evem the file namr is gone, and the whole api tests for file upload become useless.

Do you have any way to execute the file upload api requests through newman without manually going into a 5000 line json file and editing it ?

[a85]

@aksbenz I hear you.

@harryi3t @shamasis We can look into keeping the filename there and probably use a configuration in Postman/newman for the base path. What do you think?

[a85]

@harryi3t Please link other issues to this thread.

[TomONeill]

@harryi3t
I agree that it is a possible security concern, but looking at the usage of the feature, it's very, very minor to me. You could task yourselves with some extra work by adding a checkbox to the user interface that's labeled to something like "keep the (relative) path of this file in exports", to make the src property value exportable which is defaulted to off, if that helps with the security concern.

@a85
I'm not sure what you mean by base path. It is easier for me if it's a relative path, so I don't have to figure out the absolute path on the build machine and add that to the exported JSON by some PowerShell trickery (or something like that).
It would be awesome if you can figure out where the selected file is located and create a relative path based on where the export is going to be saved... But I'm not sure if the OS security allows you to do so, maybe by comparing some absolute paths, I'm not sure. I understand it's going to take some time to work that out.

[glenpike]

Ditto for @TomONeill's comment, but to add to the use-case we don't wish to use the src path in Postman, we just don't want it deleted when exporting a collection from Postman. We only use relative paths in src, but we don't use the files when we are actually using Postman - we've moved beyond these entries and are adding new tests, but every time we export, we have to then go through the diff, find all the 'nuked' entries and add them all back in. This is laborious and error prone.

[a85]

@TomONeill @glenpike I mean the relative path only. I have been thinking about the base path as a configuration option in Postman/newman so that the software knows where to determine what the path stored in the file is relative to. This will not be part of exported files and hence should not be a security issue.

@shamasis @harryi3t Am I correct in my understanding here and if yes, can we make this change inside the app? We can just start with relative paths though.

[glenpike]

I'd suggest the base path can be worked around by using a Postman variable if necessary, so is not something that is needed?

src: '{{base-path}}relative-path/to/my/file.txt'

[a85]

That's a good idea. But, it will then need to be added and might just fit better if we keep the behavior as configuration for the runtime itself. Will consider this while implementing this.

[larse514]

Would love to see a solution here. Based on other comments I will be looking into https://insomnia.rest/ for my next project

[mpatton7]

Just posting my workaround here in case it's useful for anyone else facing the same (frustrating) problem.

I configure all form-data requests with a duplicate key:

I then run a regex search/replace on the stringified Postman collection file, prior to running this through newman:

function replacePostmanCollectionFileUploadPaths(input) {
    let modifiedCollection = JSON.stringify(input);
    modifiedCollection = modifiedCollection.replace(/{\"key\":\"filekey\[(.*?)\]\",\"value\":\"(.*?)\",\"type\":\"text\"}/mg, '{"key":"\$1","src":"\$2","type": "file"}')
    return JSON.parse(modifiedCollection);
}

This allows me to quickly toggle the file key if I want to test quickly in the Postman app, and to keep a file src stored inside the request so that the context is not lost for other people maintaining the same Postman collection.

It's annoying to have to do this, given how simple it would presumably be to allow relative file paths to be stored as text values in Postman by default. But it solves our issue for now.

I am facing this issue and would like to try and implement this workaround from @jonlay but I don't quite understand how to run the function and provide the "input"? Should I be adding and executing the function as part of the collection itself or should I be processing the collection file from Javascript externally and if so how do I get the input and write it back to the same collection file?

[hanno-jonlay]

@mpatton7

or should I be processing the collection file from Javascript externally

This.

I have a little Express server configured, which I run newman on, and have some configuration set up to parse the Postman file.

Something like...

const express = require('express')
const app = express()
const port = 3000
const newman = require('newman');
const newmanCollection = require('./tests/test.postman_collection.json');

...

function replacePostmanCollectionFileUploadPaths(input) {
    let modifiedCollection = JSON.stringify(input);
    modifiedCollection = modifiedCollection.replace(/{\"key\":\"filekey\[(.*?)\]\",\"value\":\"(.*?)\",\"type\":\"text\"}/mg, '{"key":"\$1","src":"\$2","type": "file"}')
    return JSON.parse(modifiedCollection);
}

newman.run({
  collection: replacePostmanCollectionFileUploadPaths(newmanCollection),
  environment: newmanEnv,
  reporters: 'cli',
  timeoutRequest: 180000
});

Won't run out of the box, but you should be able to get the gist! It's quite a custom approach to an annoying problem, but works for me in this situation because I already have to do a lot of parsing of fixtures and other data before running my tests.

[mpatton7]

@jonlay thanks so much for getting back to me. Being somewhat a novice, I didn't realize it was as easy as using require! Thanks and lets hope we get a proper fix soon!

[saswatds]

Hello, @aksbenz and everyone who have been waiting for this feature. We are pleased to announce that you can test this feature out in our latest Canary release version 7.1.1-canary-1, which will also be shipped with the next stable production release very soon.

1. Now when the request is saved, the file path references will be persisted for subsequent use by you
2. You will also be able to use the collection runner for these files you have selected (Did not work before)
3. If you want to access the file in some other computer, simply keep another copy of the file with the same name in the same directory path in the other computer, preferably in the working directory.
4. A new Working Directory must already be created for you in your home directory called Postman/files. You should be able to change this directory on one of your likings in the settings.
5. Files kept in the working directory have their paths persisted relative to the working directory, so you and your fellow developers can work with file seamlessly without a worry.
6. In case of files selected outside working directory, you will need the setting Allow reading files outside working directory to be turned on in your other computers (This is a security feature to ensure that running untrusted collection do not get access to files on your system)

We would request all people who have been waiting for this feature to test it out and give your valuable feedback.
https://www.getpostman.com/downloads/canary

PS: Work on newman to support working-directory is ongoing and will be out before we move this feature to the stable version.

[Exodus]

@saswatds that's great news.

I'd like to add that this issue occurs not only on form-data but also on the binary type. Example collection JSON:

"body": {
    "mode": "file",
    "file": {}
}

File src attribute is removed on import/export.

[saswatds]

Hi @Exodus, we have fixed if for both form data and binary body type 😃. Do test it out!

[saswatds]

Hello everyone, we have released v7.1.0 of our production app which has this fixed.
I will be closing this issue now. If you are facing any other problem with form-data or binary body, feel free to open up a new issue.

Thanks.

[brendon-stephens]

@saswatds does this new change support the ability to have a dynamic src path through variables set in the pre-request script?

[saswatds]

@bstep93 This change does not support the ability to have a dynamic src yet.
We took up supporting the persistence and interoperability of file reference as a priority in this release. We are actively working and iterating on this and hopefully be able to provide support of dynamic src in future. There is another feature request for this #4521 and we will be using that as a master issue for this feature request

Thanks

[nanocent]

@a85 Feature Request - Add the working directory in postman settings.

Working Directory can be where the collections can be exported to easily and from where files used for file upload can be kept and referenced.
Also, the runner should support relative path and should look up this working directory for any file uploads where complete path is not given.

This will help in manually changing the file name in the scripts exported for newman tests or when importing a collection updated by someone else.

I am planning to commit the newman tests and test files to a repo and keep on updating them. The replacing part becomes an additional step everytime. Similarly while import, I have to set the file paths everytime.

[saswatds]

Hey @nanocent in the latest release of Postman App v7.1.1

• Going to settings will give you the ability to change your working directory.

• As part of this release, we also updated runner to support file resolution. If it finds a relative path, it tries to resolve it from the working directory.

• Newman v4.5.0 also supports this feature and you can set the working directory by passing the --working-dir cli option.

Do test it out and your feedbacks are always valuable.
Thanks

[sup-engineer]

does this fix resolve #4287 ? I dont see this referenced.

I am currently using the postman app Version 7.5.0 .

The data variables (from the data file) seem to work only on first run of the collection (in the collection runner). Subsequent runs using the retry button do not seem to be able to access the data variables from the data file (I am using csv)

[davepile]

I came here from #4824 which is the same issue as #4287 that @sup-engineer mentions just above.

#4824 and #4287 have been closed saying their issue will be fixed when #4320 is, but that is not the case. #4320 was fixed a four months ago and now there is no open issue regarding the Data File from the Runner screen issue. Should I open a new one?

[saswatds]

Hi @davepile, Issue #4287 has been incorrectly closed and it has nothing to do with persistence of file references.
I am reopening issue #4287 and will track that until resolution. Thanks for reaching out with this problem.

[GaneshUma]

i am facing this issue when i run through jenkins how to handle this

[BrambleRamble]

i am facing this issue when i run through jenkins how to handle this

@GaneshUma I use a different CI toolset but run Postman collections with Newman. In the exported collection, we have set the src like this:

"body": {
    "mode": "formdata",
    "formdata": [
        {
            "key": "file",
            "type": "file",
            "src": "relative_path_to_file_to_use"
        }
    ]
},

We run the collection from its location, so the relative_path_to_file_to_use is relative to both the collection and the execution location.

I hope that helps. It's been a while since I looked closely at this aspect of Postman's behaviour.

[GaneshUma]

Thank you so much this issue is got resolved when I place the collection file and data file in node machine.