New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
QUESTION: Working with the IFS #27
Comments
Have you tried piping output from the local system (via normal node.js |
That will create the file on the IFS at '/foo/bar2.txt' how to I write data to that file? |
I have tried the following and it creates the file, but it doesn't put any data into the file.
|
Following the above code if I replace 'ws.write(data)' with 'ws.pipe(data)' which is what I assume you mean to do by stream.pipe as ws is the created writeStream (ifs.createWriteStream), I get the following error. events.js:167 Error [ERR_STREAM_CANNOT_PIPE]: Cannot pipe, not readable
|
Hi @milesje Sorry to hear about the documentation. When writing a file to the IFS you want to create a readstream, that reads your file from nodejs, upload, etc and then pipes it to the writeStream. Imagine us taking bricks from a box A and moving it to box B. I am gonna stand at box A (readStream) and pass it to you (writeStream) and you will put it in box B. That's piping. Move one brick at a time from box A to box B. So when you do I put together some examples. I haven't tested them but hopefully this will help you in the right direction. Let me know if you need any further assistance :) To create a file on the IFS you can do: const writeStream = jt400.ifs().createWriteStream(fullFileName) This is a basic writeStream so you can pipe whatever you want into it. Basically empty box B ready to have something put into it.
const fs = require('fs') //nodejs filesystem
const path = require('path')
const filename = path·join(__dirname, 'test.txt')
const readStream = fs.createReadStream(filename) // Reading file from the nodejs app as a stream
readStream.pipe(writeStream)
busboy.on('file', function(fieldname, file, filename, encoding, mimetype) {
const saveTo = path.join(os.tmpDir(), path.basename(fieldname));
file.pipe(writeStream); // writesStream is our IFS writestream defined above
}); If you have a buffer you could use something like streamifer or other libraries that take your buffer and change it to stream. const createReadStream = require('streamifer').createReadStream
const path = require('path')
const filename = path·join(__dirname, 'test.txt')
const file = fs.readFileSync(filename) // Reading file from the nodejs app sync buffer
const readStream = createReadStream(file)
readStream.pipe(writeStream) Note that this is not really pure streaming since you are synchronously reading the file into a buffer (holding the whole process) and then changing it to a stream, which finally gets piped. So in our example moving from box A to box B. We're gonna first dump everything from box A to the floor and then you're gonna move it one-by-one to box B. Not as efficient. Let me know if that helps :) |
Here is my test code. doing this creates the file on the IFS but does not write any content into the created file. Am I missing something?
|
This would be so much simpler if it also had writeFile function like 'fs' does. Just declare the file you want and the data to be written to the file and your done.
|
Doing the readStream & writeStream purly with 'fs' works.
But replacing the fs.createWriteStream with ifs.createWriteStream doesn't actually write the data into the file on the IFS. |
Could it have anything to do with text encoding? 'utf8' vs whatever is used on the IBM i (AS/400) |
This is an open source project, you should add it! |
If there's no data in the file at all, I doubt it... |
And reading a file from the IFS and saving it locally works.
|
If I knew how I would be happy to create a Pull Request to add this functionality. But as I can't get the writeStream to work.... |
If I read the file off of the IFS and then pipe it into another file on the IFS it works, my issue appear to only be when trying to pipe a local file (Windows FileSystem) to a file on the IFS.
|
Yes that is a nice method but not very efficient since you are writing the whole buffer instead of streaming it. This module was developed for Tryggingamidstodin (an Icelandic insurance company) that has AS400 legacy systems. We kinda figured that other companies would be dealing with the same problem so we made it open source. This module is basically a node version of the IBM Java Toolbox and choosing what methods would be available in this module was marely based on the needs of Tryggingamidstodin. We needed to save files to the IFS and we like streams so we implemented that feature. The writeStream is mapping IFSFileOutputStream and has been working in production here. It is possible to map other functions from the com.ibm.as400.access package
I'm not sure what I can tell you about that. I can stream with 'fs' and the 'ifs'. The original example the docs was bascially reading a file from IFS and saving it as another file
Ok its good good to hear that reading works. Just note that your second example where you pipe directly is a tad better/faster since you're not waiting for the whole file and then writing.
Do you get any errors when you write to the ifs? What happens if you do |
If you clone the project and build it you should be able to mess with some tests. Checkout: /lib/test/ifs-spec.ts - line 55 or just ind "should write file"
There's a another test for binary data, same file, line 83 "should pipe image" where we pipe image. Maybe it helps if you mess around with these tests. |
OK, I'm wondering if it has something to do with text encoding ('utf8', ...)
Results from running...
|
@milesje Ehm I can't really help with the 5250 terminal since I don't have it installed and I never work with it. It might be that the file is there with all the data and the terminal is just not showing it. That would explain alot, but I don't have any ways of really testing it. Can you map the IFS drive in windows and check if you see the file there? If you open it in notepad or some other editor. |
If I map the IFS folder to my windows PC I'm able to view the file, and it does have data in it. It is a little weird that when viewing the file in the 5250 terminal it doesn't show any data. I have a few IBM i experts here where I work I will have them look over the file that is being created and see if they can figure out why its not view-able in the 5250 terminal. |
Awesome, good to hear that it's working. I'm closing this issue since the module is writing content to the file and the issue seems more related to the terminal. |
If I try to open the file in a 5250 terminal using QSH (QShell) with the cat command I get an error.
This leads me to believe that it is a text encoding issue. I am able to use the cat command on files that I created on my Windows PC and then transferred to the IFS by mapping the IFS folder as a network folder. |
@bergur I disagree that the issue is with the terminal. |
There are several ways of checking encoding of a file For example: https://stackoverflow.com/questions/3710374/get-encoding-of-a-file-in-windows I am open to the thought that the module is doing something wrong and I've reopened the issue but I'm afraid that you have to give use a little more that relates to module itself and how it is incorrectly writing files. One solution would be to try using the IBM Java Toolbox to see if you are getting the same behavour. That is: What happens if you do the exact same thing through the java code this module is based on. If its the same behaviour then there's not much we can do. If it behaves differently then we know that the mapping from node-to-java is doing something wrong and we can try to isolate that. Other food for thoughts: Is there a standard encoding on IFS? What encoding does the terminal show? |
If I open the file in the 5250 terminal and tell it to Display as HEX I get the this.
and the text I sent was
|
I will build a small Java app that use the jt400.jar and see what I get. Java is something I do know and it shouldn't take me long to put together a test for this. |
Here is a sample Java program using the jt400.jar that writes the file and it is readable via the 5250 terminal.
|
I think the key in the above program is the last parameter of the IFSFileOutputStream which is the text encoding. |
So if this library would allow for the CCSID (text encoding) to be set, that would fix all of my problems... so far anyway! |
For my Java test code was I am using jt400 version 9.4 in case that matters. |
This is the line that appears to be your issue line in the code. This is a PR I made really quickly that is a similar type modification to the node-jt400 code to allow the library path to be supplied to the PGM call, maybe you can use that to guide you if you're up to modifying this library. |
I have already forked the project and I made the changes, but when I run the test "npm run test" it errors out. I'm not sure if the test worked before my changes or not. I will submit a pull request as soon as I get it working. |
I couldn't get all the tests to work either, so I only ran my new test when I submitted my PR. |
Hi guys, sorry for dropping the ball on this. I will look at your code @milesje and run everything here. If everything passess then I see no reason not to accept your code. |
Closing this issue, this is resolved in #28 - thanks everybody. |
Really wish for some better documentation. Especially around the IFS and Program calls.
How would you go about writing a file from the local system (where the Node app is running) to the IFS?
Or just how to write data to the a file on the IFS?
The text was updated successfully, but these errors were encountered: