New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
⚡️ v0.41.0 RELEASE 🚀
#2656
Comments
|
What steps are required to define the issue and include a resolution for #2161 in this release? Transmitting large files between browsers seems like a primary use case. |
|
With any luck this will be magically be fixed in this version with the new libp2p release which includes libp2p/js-libp2p-mplex#100 I agree this is an important bug to get squashed. If you’re willing it would be good to get your testing when the prerelease comes out. |
|
|
|
|
|
|


alanshaw commentedDec 9, 2019
•
edited
repo stat --humanbitswap stat --humanlibp2pOptionspassed to libp2p bundle functionwebui@2.7.2ipfs add --hiddenflagFeb 14
We've completed a HUGE refactor to js-ipfs internals🥳 , switching to using Promises and
async/awaitover Callbacks and using async iterables instead of Node.js and Pull Streams. Ok, tell a lie, it's actually not just js-ipfs internals, it's the whole stack, including libp2p, IPLD and multiformats, you might call it a ground up re-write, but, you know, we don't like to brag.It's been a long and emotional voyage but this refactor brings a plethora of incredible immediate and future benefits to consumers, contributors and core developers that make all the work and effort worthwhile. To summarise, we've:
async/awaitso we’ll get better error stack traces and improved readability and maintainabilityUsing
async/awaitin JavaScript is gaining a lot of traction in the ecosystem and is rapidly becoming the de facto way of writing idiomatic JS. We want js-ipfs to move with the times and continue to be attractive to contributors by using modern JS features, techniques and practices. The big idea with these changes is for the code to be easier to contribute to, easier to understand, easier to maintain, and be faster and smaller than ever.This change is so big, and so significant, we wrote a whole blog post about it to explain the motivations behind the changes...in🤣 !
excruciatingdelightful depthWe've also compiled some stats on this refactor for your viewing pleasure:
package.jsonipfs-http-clientbrowser bundle (unpkg.com/ipfs-http-client@41.0.1 vs unpkg.com/ipfs-http-client@42.0.0) - that's 60% smaller!ipfs-http-client(net)...and a lot of those stats are just for js-ipfs and js-ipfs-http-client - the tip of the iceberg! We saw changes similar to this for between 60-70 dependencies across IPFS, libp2p, IPLD and multiformats.
Turns out, it's really important for package managers to retain file metadata, particularly last modified time (
mtime). Filemtimeallows them to selectively sync only data that has changed. Up until now if you wanted to host a large data set on IPFS, like a package manager's repository, it would be difficult to update."What about the permenant web?" I hear you cry. Well, this absolutely doesn't prevent a particular snapshot of a package manager's repository from being permenantly available. Metadata just enables diffs to be imported, instead of the whole thing. So, when I say "difficult to update", like I did up there, I mean slow and/or impossible. When you have Terabytes (or more) of package data and someone publishes a new package, it's kinda inconvenient to import everything again, when only a little part changed. File🥳
mtimeis a really good indicator of which things have changed, so you can use it in IPFS now!For example, there's two new options to
jsipfs addthat allowmodeandmtimeto be preserved as the file is added to IPFS:Ok, no big changes there aside from the new options, buuut, now when you list directory contents you get
ModeandMtimeinfo:The coolest thing is that it's completely backwards compatible. The CID for a given file/directory only changes if you opt in to metadata, otherwise the CIDs remain the same. Hooray!
There's a bunch of changes that add metadata capability to the CLI, HTTP and core API both for inputs and outputs. There's also a couple of new MFS commands🧙♂️ .
touchandchmodwhich allow you to change the metadata whenever you like! MagicSee the API Changes sections below for details of all the new UnixFS v1.5 stuffs.
Core API
There are significant and breaking core API changes in this release. Please see the migration guide.
IPFS is not a class that can be instantiated - use
IPFS.create. An IPFS node instance is not an event emitter.The
initoption passed toIPFS.createwill now not take any initialization steps if it is set tofalse. Previously, the repo would be initialized if it already existed. This is no longer the case. If you wish to initialize a node but only if the repo exists, passinit: { allowNew: false }to the constructor.Instance
.readyproperty has been removed. Please useIPFS.createinstead.IPFS.createNodehas been removed, please useIPFS.createinstead.Callbacks are no longer supported on any API methods. Please use a utility such as
callbackifyon API methods that return Promises to emulate previous behaviour. See the migration guide for more info.Delegated peer and content routing modules are no longer included as part of core (but are still available if starting a js-ipfs daemon from the command line). If you wish to use delegated routing and are creating your node programmatically in Node.js or the browser you must
npm install libp2p-delegated-content-routingand/ornpm install libp2p-delegated-peer-routingand provide configured instances of them inoptions.libp2p. See the module repos for further instructions:addnow returns an async iterable.addnow acceptsmodeandmtimeoptions on inputs to allow setting mode and mtime metadata for added files. See the core interface docs for more info.addresults now contain acidproperty (a CID instance) instead of a stringhashproperty.addresults now includemodeandmtimeproperties if they were set.addReadableStream,addPullStreamhave been removed. Please see the migration guide for more info.addFromStreamhas been removed. Useaddinstead.addFromFshas been removed. Please use the exportedglobSourceutility and pass the result toadd. See the glob source documentation for more details and an example.addFromURLhas been removed. Please use the exportedurlSourceutility and pass the result toadd. See the URL source documentation for more details and an example.bitswap.statresult has changed -wantlistand values are now an array of CID instances andpeersis now astring[]of peer IDs.bitswap.wantlistnow returns an array of CID instances.block.rmnow returns an async iterable.block.rmnow yields objects of{ cid: CID, error: Error }.block.statresult now contains acidproperty (whose value is a CID instance) instead of akeyproperty.dht.findProvs,dht.provide,dht.putanddht.querynow all return an async iterable.dht.findPeer,dht.findProvs,dht.provide,dht.putanddht.querynow yield/return an object{ id: string, addrs: Multiaddr[] }instead of aPeerInfoinstance(s).files.chmodhas been added. See the core interface docs for info.files.flushnow returns the root CID for the path that was flushed (/by default)files.lsPullStreamandfiles.lsReadableStreamhave been removed. Please see the migration guide for more info.files.lsnow returns an async iterable.files.lsresults now contain acidproperty (whose value is a CID instance) instead of ahashproperty.files.lsresults now includemodeandmtimeproperties if they were set. See the core interface docs for more info.files.lsno longer takes alongoption (in core) - you will receive all data by default.files.mkdirnow acceptsmodeandmtimeoptions to allow setting mode and mtime metadata. See the core interface docs for more info.files.readPullStreamandfiles.readReadableStreamhave been removed. Please see the migration guide for more info.files.readnow returns an async iterable.files.statresult now contains acidproperty (whose value is a CID instance) instead of ahashproperty.files.statresult now includesmodeandmtimeproperties if they were set. See the core interface docs for more info.files.touchhas been added. See the core interface docs for info.files.writenow acceptsmodeandmtimeoptions to allow setting mode and mtime metadata. See the core interface docs for more info.getnow returns an async iterable. Thecontentproperty value for objects yielded from the iterator is now an async iterable that yieldsBufferListobjects.idresult has changed, theaddressesproperty is now aMultiaddr[]name.resolvenow returns an async iterable. It yields increasingly more accurate resolved values as they are discovered until the best value is selected from the quorum of 16. The "best" resolved value is the last item yielded from the iterator. If you are interested only in this best value you could useit-lastto extract it like so:object.getnow accepts atimeoutoption. It will cause the method to throw with aTimeoutErrorif no data is received within the timeout window. It can be passed as anumberor astring. If anumberis passed it is interpreted as milliseconds, if a string is passed it is interpreted as a human readable duration.lsnow returns an async iterable.lsresults now contain acidproperty (whose value is a CID instance) instead of ahashproperty.lsresults now includemodeandmtimeproperties if they were set. See the core interface docs for more info.pin.addresults now contain acidproperty (a CID instance) instead of a stringhashproperty.pin.addnow accepts atimeoutoption. It will cause the method to throw with aTimeoutErrorif no data is received within the timeout window. It can be passed as anumberor astring. If anumberis passed it is interpreted as milliseconds, if a string is passed it is interpreted as a human readable duration.pin.lsnow returns an async iterable.pin.lsresults now contain acidproperty (a CID instance) instead of a stringhashproperty.pin.rmresults now contain acidproperty (a CID instance) instead of a stringhashproperty.pingnow returns an async iterable.refsandrefs.localnow return an async iterable.refsnow accepts atimeoutoption. It will cause the method to throw with aTimeoutErrorif no data is received within the timeout window. It can be passed as anumberor astring. If anumberis passed it is interpreted as milliseconds, if a string is passed it is interpreted as a human readable duration.repo.gcnow returns an async iterable.stats.bwnow returns an async iterable.swarm.peersnow returns an array of objects with apeerproperty that is astring, instead of aPeerIdinstance.swarm.addrsnow returns an array of objects{ id: string, addrs: Multiaddr[] }instead ofPeerInfoinstances.HTTP API
/api/v0/addnow supports the following additional multipart headers to allowmodeandmtimemetadata to be set on individual files:modefile mode to apply to created UnixFS entries[string]mtimemodification time in seconds before or since the Unix Epoch to apply to created UnixFS entries[number]mtime-nsecsmodification time fraction in nanoseconds[number]/api/v0/addnow returns fileMode,MtimeandMtimeNsecsif set./api/v0/file/lshas been removed, please use/api/v0/lsinstead./api/v0/files/chmodhas been added and supports the following query string args:argpath of file to apply mode to[string]modefile mode to apply[string]/api/v0/files/lsnow returns fileMode,MtimeandMtimeNsecsif set./api/v0/files/mkdirnow supports supports the following additional query string args:modefile mode to apply[string]mtimemodification time in seconds before or since the Unix Epoch to apply[number]/api/v0/files/statnow returns fileMode,MtimeandMtimeNsecsif set./api/v0/files/touchhas been added and supports the following query string args:argpath of file to apply mode to[string]mtimemodification time in seconds before or since the Unix Epoch to apply[number]/api/v0/files/writenow supports the following additional multipart headers:modefile mode to apply to created UnixFS entries[string]mtimemodification time in seconds before or since the Unix Epoch to apply to created UnixFS entries[number]/api/v0/lsnow returns fileMode,MtimeandMtimeNsecsif set.CLI
jsipfs add [file...]now supports the following flags to respect and applymodeandmtimemetadata of files added from the file system or explicitly set them:--preserve-modeautomatically apply permissions to created UnixFS entries from the file system[boolean] [default: false]--preserve-mtimeautomatically apply modification time to created UnixFS entries from the file system[boolean] [default: false]--modefile mode to apply to created UnixFS entries[string]--mtimemodification time in seconds before or since the Unix Epoch to apply to created UnixFS entries[number]--mtime-nsecsmodification time fraction in nanoseconds[number]jsipfs file lshas been removed, please usejsipfs lsinstead.jsipfs files chmod [mode] [path]has been added.jsipfs files lsnow prints filemodeandmtime.jsipfs files mkdirnow supports the following flags:--modefile mode to apply to created UnixFS entries[string]--mtimemodification time in seconds before or since the Unix Epoch to apply to created UnixFS entries[number]jsipfs files statnow prints filemodeandmtime.jsipfs files touch [path]has been added and supports the following flags:--mtimemodification time in seconds before or since the Unix Epoch to apply to created UnixFS entries[number]jsipfs files writenow supports the following flags:--modefile mode to apply to created UnixFS entries[string]--mtimemodification time in seconds before or since the Unix Epoch to apply to created UnixFS entries[number]jsipfs lsnow prints filemodeandmtime.Other changes
sharness(Does not runjs-ipfs)webui(Does not depend onjs-ipfsorjs-ipfs-http-client)ipfs-desktop(Does not depend onjs-ipfsorjs-ipfs-http-client)ipfs-pubsub-roomdoes not depend on IPFS anymore chore: remove ipfs-pubsub-room from CI #2765🙌🏽 Want to contribute?
Would you like to contribute to the IPFS project and don't know how? Well, there are a few places you can get started:
help wantedlabel in the js-ipfs repoThe best place to ask your questions about IPFS, how it works and what you can do with it is at discuss.ipfs.io. We are also available at the
#ipfschannel on Freenode.The text was updated successfully, but these errors were encountered: