Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

linux version. #104

Open
rharunzade opened this issue May 18, 2017 · 114 comments
Open

linux version. #104

rharunzade opened this issue May 18, 2017 · 114 comments
Assignees
Projects

Comments

@rharunzade
Copy link

It will be great to have a linux version of this app. please, do it.

@aaronsace
Copy link
Member

aaronsace commented May 19, 2017

If you want to test: https://github.com/PowerShell/PowerShell

I've never used Linux...

@aaronsace
Copy link
Member

I assume it all runs the same but would need Linux binaries of all the miners?

@rharunzade
Copy link
Author

it doesn't work in linux powershell.
Here'is log https://pastebin.com/KArTtyvy

@aaronsace
Copy link
Member

aaronsace commented May 20, 2017

I wonder if it just doesn't support short names; 'Sort' is short for 'Sort-Object'

Have you got a newbie Linux install that I could install in Microsoft Hyper-V?

@rharunzade
Copy link
Author

Oh, I've didn't used Hyper-V, but you can try VirtualBox for linux play.
Or You can try Wubi - install linux as a program in Windows

@fonyo
Copy link
Member

fonyo commented Jun 13, 2017

I've got two new rigs coming in the next couple of weeks (clearly only waiting on cards and mobos). My plan is to install Linux on them. A Linux-MPM would be awesome! I'll do some testing tomorrow.

@rharunzade
Copy link
Author

@fonyo it will be great.

@fonyo
Copy link
Member

fonyo commented Jun 14, 2017

@aaronsace will I have conflicts if using the latest version of PS v6.0.0 beta2? I can see on the release page that the web request functions and dir/path handling have changed in v6. Is PS v0.5.0 the one or equivalent we use here? I can already see the nightmares incoming if not having the right PS version.

@aaronsace
Copy link
Member

Scripts in different PowerShell versions are not necessarily forward/backward compatible.
If you can get it working in v6 beta and still works fine on v5 then that will be great as it will be ready for Linux and v6.

@rharunzade
Copy link
Author

I think it's not a good idea to use powershell for linux version. There is a native "bash" lang in Linux.

@aaronsace
Copy link
Member

PowerShell and .net is crossplatform though. I'm not going to rewrite in bash.

@fonyo
Copy link
Member

fonyo commented Jun 14, 2017

@aaronsace Right, I can't do miracles but I'm trying. I managed to get it downloading stuff but as soon as the download finishes it throws "Error: Incorrect command line". Any ideas?

I shall carry on regardless, I really need to dig into your code and familiarise myself with it which I quite enjoy doing while having my 3rd double espresso. :)

@fonyo
Copy link
Member

fonyo commented Jun 14, 2017

Oh it's the 7zip.exe and its implementation :D

@fonyo
Copy link
Member

fonyo commented Jun 14, 2017

Created bin folder manually. If I manually include the extracted folders, the script actually starts running. I see now that this needs a significant rewrite for linux which I am unable to do myself but I can create a trimmed down version that does claymore-dual only and "switches" the dcri values automatically which is the only reason I started doing this.

@aaronsace
Copy link
Member

I will try this soon and see where the issues lie. Least I know there are people to test it.

@rharunzade
Copy link
Author

wow, I see you work is going fine. thanx guys and a good luck! you're welcome if you need help or testing!

@fonyo
Copy link
Member

fonyo commented Jun 14, 2017

So it downloads the files then it looks like they get deleted or MPM deletes them assuming a successful 7zip extraction? Not quite sure as MPM does it over and over and after so many tries the downloaded archives just stay in root folder for some reason.

I can only test CPU miners as I am running 16.04 in VirtualBox. One other big chunk of work comes from getting all the linux miners. I must be blind but cannot seem to find where you are storing the links to the miners. Please enlighten me. @aaronsace

I haven't mentioned before but I overcome OP's problem by replacing the syntaxes that threw errors (eg. Sort-Object, Compare-Object etc). Not sure why it doesn't like the shorter ones but using longer syntax solves the problem.

Testers will be definitely needed, matey. :) Unfortunately, I cannot commit my linux server to this but will be able test it on larger rigs once my orders finally arrive. @rharunzade

@lee-aaron
Copy link
Collaborator

@fonyo I've started rewriting the ps1 commands to be readable for Linux but I'm coming across issues like the Unblock-File command isn't found. Does it work for you? Perhaps it might just be that PowerShell isn't compatible with 17.04 yet although I managed to get through some errors through downloading the missing (old) libraries.

@aaronsace
Copy link
Member

aaronsace commented Jun 14, 2017

This may be useful to skip 'Unblock-File' which is most likely windows specific:

if (Get-Command $cmdName -errorAction SilentlyContinue) {
    "$cmdName exists"
}

@lee-aaron
Copy link
Collaborator

lee-aaron commented Jun 15, 2017

I did some more testing. Found out some more syntax errors that I can fix. I ended up creating a bash script to run powershell. Here's a link to what I ran in Ubuntu 17.04 GNOME. As fonyo pointed out, the files get downloaded and then deleted. I believe this is due to the fact that the Bin folder doesn't get created as a result of syntax issues. I'll look into fixing a bunch of this.

@fonyo
Copy link
Member

fonyo commented Jun 15, 2017

@lee-aaron Great stuff, mate! Care to share the changed files here so I can compare them with my test branch? Please also include the bash script, maybe I am missing something and you do better.

To answer your previous question, I've also had issues with the Unblock-File command but the error did not stop the script from running.

@fonyo
Copy link
Member

fonyo commented Jun 15, 2017

@aaronsace I cannot seem to find where you are storing the links to the miner archives MPM references to. Can you please enlighten me?

@aaronsace
Copy link
Member

It's with the files in the miners folder. The Linux version will need different links and different exe files. It will need a variable to state if the miner command line is for Linux or Windows.

@lee-aaron
Copy link
Collaborator

@fonyo I put the files on my forked version of MPM. It's under a folder for Linux. Everything is there with my bash script and log files if you are interested. I came across Permission Denied errors which indicates that the user doesn't have RWX permissions. So practically we need to change the permissions on each child item which the line with Unblock-File does. There's a linux implementation of it. I think it has to do with sudo chmod.

@lee-aaron
Copy link
Collaborator

@aaronsace Does the 7z extraction method occur in the Include.ps1 file?
Start-Process "7z.exe" "x $FileName -o$(Split-Path $Path)/$FolderName_Old -y -spe" -Wait
It seems like the errors I'm getting is primarily due to the fact that the Bin folder is missing since zip files don't get extracted. Perhaps I can do further testing with a tar.gz file such as Claymore and see what happens.

@aaronsace
Copy link
Member

Yes, it's the "Expand-WebRequest" function.

"Expand-Archive" only supports zip so needed 7z.

@lee-aaron
Copy link
Collaborator

I did some testing recently. I started basic with unzipping zip files with the unzip command in linux but realized there were other files in different formats. I had to resort to using 7z (Linux version). So far, everything seems to be working as the zip files are being downloaded and then unzipped into the right folders. It's just the Windows miners for experimentation. I'll search up the Linux links and use 7z for them as well. However, I currently am a bit stuck on resolving these errors as I'm practically a newbie to Powershell. While I am a programmer, I'd still like some clarification. Here is a link to my logs. From my understanding, it looks like Powershell has trouble comparing two objects perhaps of different types? One is null or different type and is being compared to some value? I think resolving the DifferenceObject error will set me on the right path.

@aaronsace
Copy link
Member

aaronsace commented Jun 17, 2017

It's not loading the miners correctly. I think it's just saying that 'type' is null. So will need to look into why 'Get-ChildItemContent' is not loading it correctly.

Have you got this:
https://code.visualstudio.com/#alt-downloads
https://marketplace.visualstudio.com/items?itemName=ms-vscode.PowerShell

@aaronsace
Copy link
Member

All the aliases should be changed to the full names then debug from there.

@lee-aaron
Copy link
Collaborator

I just installed VS and Powershell for it. I'll look into changing the the aliases as they came up as warnings in the editor (ie ForEach should be ForEach-Object etc.). Anything else I should look out for?

@papampi papampi reopened this Jan 5, 2018
@papampi
Copy link
Collaborator

papampi commented Jan 5, 2018

Sorry, wrong click

@aaronsace
Copy link
Member

aaronsace commented Jan 5, 2018

@papampi hbh3 is now working on a Linux version. This this the branch to follow should you want to keep up on the progress (none as of yet):
https://github.com/MultiPoolMiner/MultiPoolMiner/tree/Linux

@lee-aaron
Copy link
Collaborator

lee-aaron commented Jan 5, 2018

I actually made some progress with conversion of batch scripts to shell scripts. Should I make a request to merge into Linux branch? Regarding to get a miner to run on Linux, I can't get ccminerklaust to compile on mine. It keeps saying I'm missing a cuda_runtime_api headerfile. But step wise, I think it would be like

  1. wget [link]
  2. tar -zxvf [tar.gz] -C [directory]
  3. ./autogen.sh ... [other compile/building scripts]
  4. run main miner script in new terminal window (with gnome-terminal?)

Edit: I forgot to install CUDA but yeah for ccminerklaust: autogen -> configure -> build -> ccminer

Edit 2: Running ccminer with --no-cpu-verify but getting rejected for low difficulty share on lyra2v2

Edit 3: Fixed it to run neoscrypt and works

@aaronsace
Copy link
Member

@lee-aaron Nice, yes, best to do a pull on that branch then @hbh3 will review/merge it.

@bviktor
Copy link

bviktor commented Jan 5, 2018

TBH I don't quite get it. Why convert from one platform-specific format to another platform-specific format? And then maintain all logic in 2 formats? Why not convert the remaining batches to PowerShell? Or convert it all to Python and be done with it, even :)

@aaronsace
Copy link
Member

@bviktor PowerShell is open source and cross platform.

@bviktor
Copy link

bviktor commented Jan 5, 2018

Who said it isn't? I even suggested going full PowerShell. I was responding to this:

I actually made some progress with conversion of batch scripts to shell scripts.

@aaronsace
Copy link
Member

@bviktor Most likely the start.bat files.

@bviktor
Copy link

bviktor commented Jan 5, 2018

You mean those like 5 files in the tree root? That seems like a no brainer. Still, I don't quite see converting them to bash as a useful effort. Why not convert them to PowerShell. MPM runs in PS anyway :)

@vhatzist
Copy link

vhatzist commented Jan 5, 2018

@lee-aaron

About below:

run main miner script in new terminal window (with gnome-terminal?)

How about using screen to run the miner scripts on new screens? This way the Linux implementation of MPM would be desktop-environment agnostic and could even run on Linux machines with no GUI installed.

Viewing the miner output would be as easy as connecting to the screen that runs it via the CLI.

@lee-aaron
Copy link
Collaborator

@vhatzist Well I just was providing some sample steps. It is completely flexible to make it usable among all distros.

@hbh3
Copy link
Member

hbh3 commented Jan 6, 2018

I'll be investigating setting it up for use with and without window managers.

@vhatzist
Copy link

vhatzist commented Jan 6, 2018

@lee-aaron

Well I just was providing some sample steps. It is completely flexible to make it usable among all distros.

Do you have any preliminary ideas as to how to support different distros or different desktop environments or even CLI-only machines without a desktop environment? This is why using 'screen' came to mind, or converting MPM to a daemon that will spawn the miners in the background...

@papampi
Copy link
Collaborator

papampi commented Jan 6, 2018

screen is very good, we use it in nvOC almost for all important scripts (watchdog, miner, temp control ,...)

@papampi
Copy link
Collaborator

papampi commented Jan 6, 2018

@aaronsace
Thanks for new linux tree,
I just downloaded it, made some changes for start.sh, miner and some more. but still cant get mpm to launch the miner.
Is this the correct line to start it on linux :

pwsh -noexit -executionpolicy bypass -file ./MultiPoolMiner.ps1 "&-wallet 1Q24z7gHPDbedkaWDTFqhMF8g7iHMehsCb -username papampi -workername multipoolminer -region europe -currency btc,usd,eur -type nvidia -poolname miningpoolhub -algorithm equihash "

@apiontek
Copy link

I am interested to help with this. I've been hacking together my own script in python to fetch pool api data and make decisions on what miners to run, so I have some familiarity with the process. I haven't used powershell much, let alone on linux, but I have 2 linux & 2 win machines doing mining, so I could test things out and contribute as I learn.

@hbh3
Copy link
Member

hbh3 commented Jan 18, 2018

@apiontek I have already started on the Linux conversion. Work is progressing and I should have something to show soon.

@CharlTruter
Copy link

Hi @hbh3,

Any progress here? Need any help with the conversion to speed things up?

@hbh3
Copy link
Member

hbh3 commented Feb 6, 2018

Yes, I've progressed past several hurdles, but there are always new ones to jump over. Thanks for the offer.

@ajpsp5
Copy link

ajpsp5 commented Apr 8, 2018

I'm struggling to get this to work on Ubuntu 14.04. I keep getting errors:

+                                                          ~~~~~~~~~~~~~~~~                                                    
+ CategoryInfo          : NotSpecified: (:) [ConvertFrom-Json], ArgumentException                                              
+ FullyQualifiedErrorId : System.ArgumentException,Microsoft.PowerShell.Commands.ConvertFromJsonCommand                        
ConvertFrom-Json : Conversion from JSON failed with error: Additional text encountered after finished reading JSON content: {. 
Path '', line 18, position 0.                                                                                                  
At /home/ethos/multipoolminer/linux-repo/Include.psm1:198 char:113                                                             
+ ... 1" | Where-Object BaseName -EQ $Name | Get-Content | ConvertFrom-Json                                                    
+                                                          ~~~~~~~~~~~~~~~~                                                    
+ CategoryInfo          : NotSpecified: (:) [ConvertFrom-Json], ArgumentException                                              
+ FullyQualifiedErrorId : System.ArgumentException,Microsoft.PowerShell.Commands.ConvertFromJsonCommand                        
                                                                                                                               
INFO: Selecting best pool for each algorithm.                                                                                  
INFO: Getting miner information.                                                                                               
INFO: Calculating profit for each miner.                                                                                       
WARNING: No miners available. 

Powershell Version:

PowerShell v6.1.0-preview.1 

MultiPoolMiner Branch:

Linux

Command I run:

pwsh -noexit -executionpolicy bypass -file ./MultiPoolMiner.ps1 "&-wallet 16kcdsWD2h7YAdv9YGMyP9KCpcNEHE9zFM -username ajpsp5 -
workername AMD_rig_1 -region us -currency USD -type AMD -poolname miningpoolhub -algorithm CryptoNight,Decred,Decrednicehash,Et
hash,Equihash,Groestl,Lbry,Lyra2re2,Lyra2z,Neoscrypt,Pascal,Sib,Skunk -donate 10 -watchdog -minerstatusurl https://multipoolmin
er.io/monitor/miner.php -switchingprevention 2"

@hbh3
Copy link
Member

hbh3 commented Apr 8, 2018

Adam, I would not try the linux branch right now as it is not ready. Still more work to be done.

@ajpsp5
Copy link

ajpsp5 commented Apr 8, 2018

@hbh3

Do you know of any alternatives for linux based systems?

@ajpsp5
Copy link

ajpsp5 commented Apr 16, 2018

I end up creating a package for automatically switching to the most profitable algorithm. I hope it helps anyone else that needs an Ethos/Linux alternative.

https://github.com/ajpsp5/ethos-auto-miner

@sephir69
Copy link

1 hour a day donation seems a bit much to me

@MaynardMiner
Copy link

I have written a powershell based switching application that was originally a fork of sniffdog miner, but has been completely (about 85%) re-written at this point, which works 100% natively in linux. I didn't even know MPM existed when I started it. I started it, as sniffdog disappeared.

There are a whole bunch of complications, besides just getting powershell to work:

1.) zombie applications. (no process handles to deal with it).
2.) ports fail to close.
3.) api differences for same miner between windows/linux.
4.) process tracking with powershell is horrible in linux.
5.) half the cmdlets in linux exist in powershell (though its getting better).
6.) the moment you make it- everyone wants it to work in every linux distribution imaginable, and expect it to work in their particular flavor right out of the box.
7.) linux mining os developers get very mad at you, very, very, quick.
8.) certain cmdlets may or may not work in linux (straight glitches).

I have written an open sourced miner in powershell that works for linux- SWARM. You are welcome to reference it, but I have re-written most of every function every part of sniffdog miner in order to get it to work, and I would imagine much of mpm would have to be re-written as well.

If you have questions on using powershell linux, let me know. I've become a wizzard at it now. I would assist here, but my project is full-time work at this point.

@aaronsace
Copy link
Member

@MaynardMiner Well, the last guy who I paid was useless but if you want to join the organization then let me know.

@MaynardMiner
Copy link

MaynardMiner commented Jan 16, 2019

I don't particularly want to join, as my project direction currently deviates/going a different direction than yours, and I would like to stay on top of it (It's a full time job). It's no offense to you- You guys are geniuses in your own rights.

But I would like to keep a dialog with you guys, and offer solutions/contributions as you require them. I know by solving problems you may have, only helps me solve mine. Feel free to tag or ask me questions if you see I made something that you haven't implemented yet, and I can explain what I did.

My codebase is open source- You are welcome to reference it as needed.

@MaynardMiner
Copy link

I'll tell you what, though. Let me finish my next release, because zergpool is back on scene and I have to get them in- And I will try plugging MPM into linux, and see what can be done. If I see ways to make it work, and can formulate an outline to do so, and what needs to be done- I'll message you.

I don't like being the only guy on the linux side. It's a lot of work, as I'm sure you know. It would be nice to come up with a way to bridge projects in some form, even though we are going in different directions.

But I have to plug it in, and calculate time vs. energy to do. I will get back to you.

@starrbuck68
Copy link

How's that Linux version coming along?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Version 3
Core MPM
Development

No branches or pull requests