New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Windows Support #1017

Closed
daviddias opened this Issue Sep 13, 2017 · 38 comments

Comments

Projects
None yet
10 participants
@daviddias
Member

daviddias commented Sep 13, 2017

More and more users have been trying to use js-ipfs on Windows and seeing their experiments fail. This happens because, in fact, js-ipfs doesn't have Windows support at the moment.

Reported issues:

We welcome a captain to develop Windows support for js-ipfs. It should be really close and mostly be an issue with pathing and native deps.

@Beanow

This comment has been minimized.

Beanow commented Sep 20, 2017

@victorb

This comment has been minimized.

Member

victorb commented Sep 20, 2017

Did a look into this as I have a windows machine close by.

First discover was that fs-ext does not like windows, and gives a "binding.fcntl is not a function" error (as in #1016)

The only module that uses fs-ext is lock-me (https://github.com/dignifiedquire/lock-me) which is supposed to work on Windows but does not for some reason, it crashes instead of using javascript version of fs-ext.

However, fs-ext is a optional dependency, and removing it from lock-me makes both portable and unix tests on lock-me to pass.

Also, removing js-ext from node_modules in js-ipfs, makes me able to run the daemon successfully at least. Have not yet tried to add/get files though.

Code I tested with:

const ipfs = require('./src/code/index.js')
const node = new ipfs()
node.once('ready', () => console.log('ready'))
node.on('error', (err) => console.log(err))

As I usually don't develop on windows, I didn't have a dev env setup. But I found that windows-build-tools is a very useful module for setting up everything required for it to work. So after installing nodejs, running npm install --global --production windows-build-tools installs python and any .NET Framework stuff needed to develop.

@JonasDev99

This comment has been minimized.

JonasDev99 commented Oct 11, 2017

C:\Program Files\nodejs\node.exe" "C:\Program Files\nodejs\node_modules\npm\bin\npm-cli.js" install ipfs

Standard error:
npm WARN deprecated ipld-eth-star@1.4.1: deprecated in favor of js-ipld-ethereum https://github.com/ipld/js-ipld-ethereum
npm WARN deprecated babel-preset-es2015@6.24.1: 🙌 Thanks for using Babel: we recommend using babel-preset-env now: please read babeljs.io/env to update!
npm WARN lifecycle npm is using C:\Program Files\nodejs\node.exe but there is no node binary in the current PATH. Use the --scripts-prepend-node-path option to include the path for the node binary npm was executed with.
'node' is not recognized as an internal or external command,
operable program or batch file.
'node' is not recognized as an internal or external command,
operable program or batch file.
'node' is not recognized as an internal or external command,
operable program or batch file.
'node' is not recognized as an internal or external command,
operable program or batch file.
'node' is not recognized as an internal or external command,
operable program or batch file.
'node' is not recognized as an internal or external command,
operable program or batch file.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fs-ext@1.0.0 (node_modules\fs-ext):
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fs-ext@1.0.0 install: node-gyp configure build
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: Exit status 1
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: gc-stats@1.0.2 (node_modules\gc-stats):
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: gc-stats@1.0.2 install: node-pre-gyp install --fallback-to-build
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: Exit status 1

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! leveldown@1.9.0 install: prebuild-install || node-gyp rebuild
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the leveldown@1.9.0 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR! C:\Users...\AppData\Roaming\npm-cache_logs\2017-10-11T15_58_12_138Z-debug.log

Standard output:

fs-ext@1.0.0 install C:\Users\Jonas\WebstormProjects\untitled\node_modules\fs-ext
node-gyp configure build

C:\Users...\WebstormProjects\untitled\node_modules\fs-ext>if not defined npm_config_node_gyp (node "C:\Program Files\nodejs\node_modules\npm\bin\node-gyp-bin\....\node_modules\node-gyp\bin\node-gyp.js" configure build ) else (node "" configure build )

gc-stats@1.0.2 install C:\Users...\WebstormProjects\untitled\node_modules\gc-stats
node-pre-gyp install --fallback-to-build

keccak@1.3.0 install C:\Users\Jonas\WebstormProjects\untitled\node_modules\keccak
prebuild-install || node-gyp rebuild || echo "Keccak bindings compilation fail. Pure JS implementation will be used."

C:\Users...\WebstormProjects\untitled\node_modules\keccak>if not defined npm_config_node_gyp (node "C:\Program Files\nodejs\node_modules\npm\bin\node-gyp-bin\....\node_modules\node-gyp\bin\node-gyp.js" rebuild ) else (node "" rebuild )
"Keccak bindings compilation fail. Pure JS implementation will be used."

leveldown@1.9.0 install C:\Users...\WebstormProjects\untitled\node_modules\leveldown
prebuild-install || node-gyp rebuild

C:\Users...\WebstormProjects\untitled\node_modules\leveldown>if not defined npm_config_node_gyp (node "C:\Program Files\nodejs\node_modules\npm\bin\node-gyp-bin\....\node_modules\node-gyp\bin\node-gyp.js" rebuild ) else (node "" rebuild )

Process finished with exit code 1

@victorb

This comment has been minimized.

Member

victorb commented Oct 11, 2017

 npm WARN lifecycle npm is using C:\Program Files\nodejs\node.exe but there is no node binary in the current PATH. Use the --scripts-prepend-node-path option to include the path for the node binary npm was executed with.

and

'node' is not recognized as an internal or external command,
operable program or batch file.

makes me believe that you have not installed node correctly. You should make sure the node binary is in your $PATH variable. You can find how to do that here: https://stackoverflow.com/a/27864253/360186

@daviddias daviddias added the ready label Oct 13, 2017

@mkg20001

This comment has been minimized.

Contributor

mkg20001 commented Oct 18, 2017

I finally got a windows VM setup and did some tests what works:
The gateway works after doing "jsipfs init" which crashes with Error: Invalid key: /CIQGKXXGEEETAXRWRSFJYDYW5VSDR5YLCWWGVI6N4KJ2Y6MYMKNDG6I (but "jsipfs daemon" still works fine).
After a peer connects and initializes the /ipfs/bitswap/ protocol the daemon crashes with the same error.

I suspect this error is occuring because \ or / is used in paths or path splitting where it shouldn't be used on windows. (Most likely it tries to split some string that has / with \)

I would really like help getting jsipfs to run on windows, but I don't think that I could work on that so much.

@richardschneider

This comment has been minimized.

Contributor

richardschneider commented Oct 31, 2017

@dryajov I'm willing to spend some time on getting this to work on Windows. What would you like me to look at.

@richardschneider

This comment has been minimized.

Contributor

richardschneider commented Oct 31, 2017

I tried installing, npm install js-ipfs, and got errors on fs-ext. It turns out that my node version (5.7.0) is the culprit. Updating it to version 8.8.1 resolved the issues and js-ipfs now installs without an error.

You can download the latest node for windows at https://nodejs.org/en/download/current/

Unfortunately, as @mkg20001 reported, I now get an Invalid Key error.

@dryajov

This comment has been minimized.

Member

dryajov commented Oct 31, 2017

Hi @richardschneider! Thanks so much for looking into this, I believe the next steps would be figuring out the error that both @mkg20001 and you are seeing as well as any other that pop up. We also need CI setup with this, which I'll look into next.

Keep up the awesome work @mkg20001 and @richardschneider! ❤️

@richardschneider

This comment has been minimized.

Contributor

richardschneider commented Oct 31, 2017

@dryajov I recommend using AppVeyor for a CI. I've used it for both JS and C# projects to test on windows and the people are really responsive.

@dryajov

This comment has been minimized.

Member

dryajov commented Oct 31, 2017

👍 I believe that's what we're using already for some of our projects - check github.com/ipfs/aegir for example (that's our built/release system).

@pvh

This comment has been minimized.

pvh commented Nov 2, 2017

Hi guys, I've been doing some hacking on IPFS/libp2p on Windows. The ipfs-pubsub-room-demo doesn't send direct messages due to something going wrong with the direct dial. Same code works fine on my linux box. @dryajov and I did some diagnostics and isolated it to just node on Windows (example code works in the browser and on my linux machine.)

@dryajov

This comment has been minimized.

Member

dryajov commented Nov 2, 2017

@pvh yeah, windows support is precarious at best at this moment, we're just starting this work and so far it's quite possible that most things are broken. @richardschneider has been doing great work in getting the datastores to behave in windows, but there are quite possible more things broken.

It would help if you could provide a full trace of your runs in windows, we use the debug module so enabling the DEBUG=* env variable should yield some some pretty verbose output, if you could create a gist with that output, that would go a long way helping us troubleshoot this.

Thanks for helping us getting IPFS working on windows!

@richardschneider richardschneider referenced this issue Nov 3, 2017

Closed

Windows #18

@daviddias

This comment has been minimized.

Member

daviddias commented Nov 3, 2017

Fixing Datastore Paths

A number of PRs have been opened to fix the lack of OS agnostic'ity of datastore. These are:

I'm going to start reviewing these. To make sure we don't break anything, I would really appreciate if someone could test it all out to js-ipfs (npm link all the things) and confirm it works on windows and that the repo is still interoperable with Go)

@richardschneider

This comment has been minimized.

Contributor

richardschneider commented Nov 3, 2017

My current understanding is that repo/blocks is interoperable with Go; even with sharding!

However, repo/datastore with levelUp is not.

@the-townsend

This comment has been minimized.

the-townsend commented Nov 3, 2017

I'd be happy to help here as it would be good to get js-ipfs on windows. Happy to take on direction on where to put my efforts.

@richardschneider

This comment has been minimized.

Contributor

richardschneider commented Nov 3, 2017

@the-townsend Thanks for the offer. I'm in the middle of getting the datastores to work on windows. Hopefully will be done today.

Then we can take a look at any outstanding issues.

@daviddias

This comment has been minimized.

Member

daviddias commented Nov 4, 2017

We will need some help on adding all the badges to all the repos with appveyor enabled. I'll be going repo by repo and enabling and listing here where there is a interop bug

@the-townsend

This comment has been minimized.

the-townsend commented Nov 4, 2017

I have now managed to install. Details attached/linked below (*). I was using Node 9, which may have been the issue. The README for js-ipfs has 'node@6 or above'. However it could also be the change that @diasdavid made that made it work. [Note I uninstalled/deleted ''.window-build-tools'' which changed the set up issues previously, so this isn't a dependency as far as I can see.]

Unfortunately, as you can see from the attached I am getting the same issue as above coming from 'datastore' (specifically key.js); am going to try and take a look at those now.

(*)
IPFS_Window_Install.txt

@richardschneider

This comment has been minimized.

Contributor

richardschneider commented Nov 4, 2017

@diasdavid We need ipfs/js-datastore-level#4 and then update js-ipfs-repo before checking the other modules.

@richardschneider

This comment has been minimized.

Contributor

richardschneider commented Nov 4, 2017

@diasdavid @dignifiedquire The other modules should be run on appveyor. I'm happy to create the appveyor.yaml and do the testing but maybe one of you should create the appveyor account.

@richardschneider

This comment has been minimized.

Contributor

richardschneider commented Nov 4, 2017

NPM reports the following dependents of js-ipfs-repo.

  • ipld-resolver
  • ethereumjs-ipld-dump not part of IPFS
  • ipld-cli
  • js-libp2p-pstn-nodenot part of IPFS
  • ipfs

and these for interface-datastore

  • ~~datastore-ipfs-ro-hook~ not part of IPFS
  • libp2p-kad-dht
  • ipfs-repo
  • datastore-fs
  • datastore-core
  • datastore-level
  • ipld-resolver
  • datastore-ipfs-http-api not part of IPFS
@the-townsend

This comment has been minimized.

the-townsend commented Nov 7, 2017

Not sure what the protocol here is.
https://github.com/kumavis/ethereumjs-ipld-dump/blob/master/src/index.js
has '/.jsipfs' lines 23. Do we edit that code and give a pullrequest to 'kumavis'?

@richardschneider

This comment has been minimized.

Contributor

richardschneider commented Nov 7, 2017

If its not part of IPFS we can ignore it.

@the-townsend

This comment has been minimized.

the-townsend commented Nov 8, 2017

Can't see that anything needs doing on

ipld-cli

@richardschneider

This comment has been minimized.

Contributor

richardschneider commented Nov 8, 2017

@the-townsend Thanks I'll tick it off the list

@the-townsend

This comment has been minimized.

the-townsend commented Nov 8, 2017

js-libp2p-pstn-node is not part of ipfs

@richardschneider

This comment has been minimized.

Contributor

richardschneider commented Nov 8, 2017

@the-townsend js-ipfsd-ctl is not working on windows. Its blocking js-ipfs-api from being tested.

Would you mind looking at it.

@the-townsend

This comment has been minimized.

the-townsend commented Nov 9, 2017

@richardschneider

This comment has been minimized.

Contributor

richardschneider commented Nov 9, 2017

It's mainly a package to spinup an IPFS server. Could you get the tests running on a windows machine

@the-townsend

This comment has been minimized.

the-townsend commented Nov 9, 2017

Results attached.

27 passing (55s)
3 pending
1 failing

Can't immediately see whether this is a simple problem or not.
js-ipfsd-ctl-test-results.txt

@the-townsend

This comment has been minimized.

@richardschneider

This comment has been minimized.

Contributor

richardschneider commented Nov 10, 2017

It looks like js-ipfsd-ctl is receiving some TLC this comment

@daviddias

This comment has been minimized.

Member

daviddias commented Nov 10, 2017

The next big piece should be getting https://github.com/ipfs/js-ipfs-api to work on Windows too.

@daviddias

This comment has been minimized.

Member

daviddias commented Nov 10, 2017

Windows support is here!!! 🌟🚀🎆

image

With #1065

BIG THANK YOU to @richardschneider who almost single-handedly defeated all the Windows dragons 🐉 and brought Windows support to js-ipfs and its ipfs dependencies.

Let's celebrate in the next all hands! @richardschneider will you be able to join and give us a demo?

@daviddias

This comment has been minimized.

Member

daviddias commented Nov 10, 2017

Closing this issue. Let's track the last js dragon on js-ipfs-api at ipfs/js-ipfs-http-client#408 (not required for js-ipfs to work on Windows)

@daviddias daviddias closed this Nov 10, 2017

@wafflebot wafflebot bot removed the ready label Nov 10, 2017

@the-townsend

This comment has been minimized.

the-townsend commented Nov 11, 2017

@diasdavid @richardschneider
FYI getting below when trying to replicate locally:
...
22 passing (2m)
1 pending
7 failing
...

[details attached]

I can't access the test results above easily to try and see what's going on.

js-ipfs-test-results.txt

@daviddias daviddias referenced this issue Nov 20, 2017

Closed

⚡️ v0.27.0 RELEASE 🚀 #1095

24 of 24 tasks complete
@barnendu

This comment has been minimized.

barnendu commented Jan 27, 2018

I had same issue, jsipfs was giving me error while installing. Its has been solved after using
npm install --global --production windows-build-tools in windows 10 run power shell has admin. Then
run npm install ipfs --global

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment