globSource missing!

[ekkis]

in this README: https://github.com/ipfs/interface-js-ipfs-core/blob/master/SPEC/FILES.md it is stated I can:

const IPFS = require('ipfs')
const { globSource } = IPFS

but when I do globSource is undefined. I initially installed ipfs but then installed ipfs-http-client, neither of which this works for. can we fix the documentation, and what is the correct way to import the contents of a directory now?

p.s. the above I'm attempting in NodeJs with v41.0.1 of ipfs-http-client and v0.40.0 of ipfs

p.p.s. the docs are also wrong in that they point the reader to do:

const IPFS = require('ipfs')
const { globSource } = IPFS
const ipfs = await IPFS.create()

for await (const file of ipfs.add(urlSource('https://ipfs.io/images/ipfs-logo.svg'))) {

but, of course, urlSource is not defined (also not in IPFS). And the link offered in the document: "A great source of [examples] can be found in the tests for this API." https://github.com/ipfs/interface-ipfs-core/blob/master/src/files-regular generates a 404. grr...

finally, when looking through the test suite I find https://github.com/ipfs/js-ipfs-http-client/blob/master/test/files-mfs.spec.js where the following code is displayed:

 it('.add directory with progress option', async () => {
    [...]
    // TODO: needs to be using a directory
    const res = await all(ipfs.add(testfile, { progress: progressHandler }))

so that is of no help either. so frustrating. all I want to do is import a directory

[alanshaw]

@ekkis there's been a delay and the latest versions of js-ipfs-http-client and js-ipfs that these docs refer to haven't yet been published (although there is a prerelease of js-ipfs-http-client 42.0.0-pre.0).

Until ipfs-http-client@42 and ipfs@0.41 are released, please checkout the docs at this revision: https://github.com/ipfs/interface-js-ipfs-core/tree/e51d27cb04faf87b15c03bf7a4055b216c45895f

If you want to add a directory recursively with the current version, you can use addFromFs - https://github.com/ipfs/interface-js-ipfs-core/blob/e51d27cb04faf87b15c03bf7a4055b216c45895f/SPEC/FILES.md#addfromfs

[ekkis]

thank you @alanshaw. on closer inspection I realised that the latest on NPM is missing the last line of what I find on github, namely:

Object.assign(ipfsClient, { Buffer, CID, multiaddr, multibase, multicodec, multihash, globSource, urlSource })

which contains what I need. I will pursue addFromFs as suggested. thanks again

[ekkis]

in trying your suggestion, I have:

var args = process.argv.slice(2);
const IPFS = require('ipfs')
IPFS.create().then(async ipfs => {
    for await (const f of ipfs.addFromFs(args[0], {recursive: false})) {
        console.log(f);
    }   
})

but I'm getting:

(node:81525) UnhandledPromiseRejectionWarning: TypeError: ipfs.addFromFs(...) is not a function or its return value is not async iterable
at IPFS.create.then (/Users/ekkis/dev/put:4:29)
(node:81525) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 2)

I've verified that ipfs does contain that method name and according to the docs I should receive an array of promises (which would be iterable). is there working code I can look at or can you see the problem with the code?

[ekkis]

if I look at addFromFs, it's curious:

function () {
    var args = [].slice.call(arguments)
    var ctx = this
    if (args.length >= 1 &&
        typeof args[args.length - 1] === 'function') {
      // callback mode
      var cb = args.pop()
      fn.apply(this, args)
        .then(function (val) { cb.call(ctx, null, val) },
          function (err) { cb.call(ctx, err) })
        return
    }
    // promise mode
    return fn.apply(ctx, arguments)
  }

so it serves as a passthrough for some other function stored in fn which I gather is in the enclosing scope (and I have no idea what it does or how to figure it out). since addFromFs is invoked on ipfs I also looked at that object for clues but there's no fn in it. help?

[ekkis]

when I look in ipfs/src/core/index.js I don't see an addFromFs exported. I don't understand how this works at all. I do see this.files = components.filesMFS(this) but when I look at ipfs.files I only see: cp, mkdir, stat, rm, read, readReadableStream, readPullStream, write, mv, flush, ls, lsReadableStream, lsPullStream... no addFromFs

[ekkis]

ah... so the method is included in Object.assign(this, components.filesRegular(this)) which is require('./add-from-fs')(self) which itself is (self) => require('../../runtime/add-from-fs-nodejs')(self), which contains:

'use strict'

const callbackify = require('callbackify')
const globSource = require('ipfs-utils/src/files/glob-source')
const all = require('async-iterator-all')

module.exports = self => {
  return callbackify.variadic(async (...args) => { // eslint-disable-line require-await
    const options = typeof args[args.length - 1] === 'string' ? {} : args.pop()

    return all(self._addAsyncIterator(globSource(...args, options), options))
  })  
}

and I gather variadic is responsible for generating the method I'm seeing that references fn... but given the context here I don't see fn defined. my god, how complicated. ugh

[ekkis]

oh my gosh. I'm such an idiot. I'm still attempting the loop as it was with the globSource. what I needed was simply:

var args = process.argv.slice(2);
const IPFS = require('ipfs')
IPFS.create().then(async ipfs => {
    const res = await ipfs.addFromFs(args[0], {recursive: true});
    console.log(res)
})