Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Could not load plugin: c:\ado\plus\g\gtools_windows.plugin #35

Closed
dgcbritto opened this issue Mar 6, 2018 · 67 comments
Closed

Could not load plugin: c:\ado\plus\g\gtools_windows.plugin #35

dgcbritto opened this issue Mar 6, 2018 · 67 comments
Assignees
@mcaceresb
Labels
bug

Comments

@dgcbritto
Copy link

Hi guys,

I'm trying to run gtools on a Windows Server 2016 with Stata 15, but I'm getting the following error:

Stata MP 15 6 users output:
gcollapse y, by(x)
Could not load plugin: c:\ado\plus\g\gtools_windows.plugin
(error occurred while loading _gtools_internal.ado)

Any tips on how to solve it?

Thanks a lot for this amazing command!
@mcaceresb
Copy link
Owner

Can you run

!dir c:\ado\plus\g\*gtools*
!dir c:\ado\plus\s\*spooky*

And post the output?

@dgcbritto
Copy link
Author

dgcbritto commented Mar 6, 2018 via email

@mcaceresb
Copy link
Owner

mcaceresb commented Mar 6, 2018 •
edited

Perhaps; I suspected the issue is that the files are missing. Can you physically navigate to those directories using file explorer? If they are indeed blank, then gtools was not installed correctly. Run

local github "https://raw.githubusercontent.com"
net install gtools, from(`github'/mcaceresb/stata-gtools/master/build/)

And check again using file explorer. If this does not fix the problem then I assume it's a problem with permissions, since it's a server, and you'll have to install gtools in a different folder.

Run the above first. If it's a permissions issue let me know and I can guide you through that install.

@dgcbritto
Copy link
Author

dgcbritto commented Mar 6, 2018 via email

@mcaceresb
Copy link
Owner

You mean that the folders are blank, right? Perhaps there is a discrepancy in the ado paths. How about:

adopath
disp "`c(sysdir_plus)'"
disp "`c(sysdir_personal)'"

You can try and download everything manually. I have placed just the relevant files in the release page (https://github.com/mcaceresb/stata-gtools/releases). You can download gtools-0.12.5.zip, extract the file, and add

adopath + /path/to/gtools

to your scripts.

@dgcbritto
Copy link
Author

These commands show:
. disp "`c(sysdir_plus)'"
c:\ado\plus/

. disp "`c(sysdir_personal)'"
c:\ado\personal/

I mannually removed all gtools installation files from c:/ado/plus and manually installed it from the zip file, as you said. However, I get the same error message while running gcollapse:
"Could not load plugin: c:\ado\plus\g\gtools_windows.plugin
(error occurred while loading _gtools_internal.ado)"
I've also tried to use gegen to check it was some specific problem with gcollapse, but the command gives exactly the same error.

Could it have to do with my Windows version? Windows Server 2016

@mcaceresb
Copy link
Owner

You said "mannually removed all gtools installation files from c:/ado/plus"; I thought the folders were empty? Anyway, if you did it manually then yes, it might be your windows version.

To check, extract the zip in some a folder but don't move the files. For example, if you extracted them to "C:\testing\gtools" then run, from Stata,

cd C:\testing\gtools
clear
set obs 10
gen x = 1
gcollapse x
capture program drop gtools_plugin
program gtools_plugin, plugin using("gtools_windows.plugin")

And let me know what happens.

@dgcbritto
Copy link
Author

No, it wasn't empty before. Installation with net install was successful at first

I've run your code but continue to get the same message:
". gcollapse x
Could not load plugin: .\gtools_windows.plugin
(error occurred while loading _gtools_internal.ado)"

"gcollapse x
Could not load plugin: .\gtools_windows.plugin
(error occurred while loading _gtools_internal.ado)"

@mcaceresb mcaceresb added the bug label Mar 7, 2018
@mcaceresb mcaceresb self-assigned this Mar 7, 2018
@mcaceresb
Copy link
Owner

I was hoping it was just a permissions thing or an install problem, but it certainly look like it's an issue with the plugin. Hail Mary pass: If you run "disp c(bit)" from Stata, what's the answer?

I don't have access to Windows Server 2016 and I don't know why it wouldn't work there, so this will probably go unfixed for some time.

@dgcbritto
Copy link
Author

"disp c(bit)
64"

I've given another shot running it in Compatibility mode with Windows 7&8 and executing as administrator but no luck.
No worries, thanks a lot in any case. That's a great tool, severely improve work with huge datasets!

@ppaspp
Copy link

ppaspp commented May 11, 2018

I ran into the same problem running gegen with Stata 15.1 on Windows Server 2016.

Gtools ado is installed in a network folder and runs fine from Linux and regular Windows Systems but fails on Windows Server. I also tried compatibility mode (win 7) without success.

Any chance to get this bug fixed. I would like to help, although I have no C++ programming skills.

@mcaceresb
Copy link
Owner

  1. Are you able to install programs on the server?
  2. If not, does the server happen to have Cygwin and MinGW?

Hopefully this is an issue with the gtools code that I can fix rather than spookyhash. If the answer to both of the above are true, I'll try to upload some test plugins for you to try out on the server and see if we can narrow down the problem anyway.

@ppaspp
Copy link

ppaspp commented May 16, 2018

no, to both questions.
Will check with the IT department if they can provide an suitable environment.

@mcaceresb
Copy link
Owner

Let's try something else before. I'll be uploading some test plugins later today and was hoping you could test them out. No extra programs required.

@ppaspp
Copy link

ppaspp commented May 16, 2018

ok

mcaceresb added a commit that referenced this issue May 16, 2018
@mcaceresb
Test files to debug issue #35
81c3c1d
@mcaceresb
Copy link
Owner

Please download test.do, test1.plugin, test2.plugin, and spookyhash.dll from here. Then run test.do.

I expect the first test to run and the second should fail. Let me know if that is the case.

@ppaspp
Copy link

ppaspp commented May 16, 2018

as expected:

. cd
L:\Cloud\gtest

. do "L:\Cloud\gtest\test.do"

. program test1, plugin using(test1.plugin) 

. plugin call test1
Hello World

. 
. program test2, plugin using(test2.plugin) 
Could not load plugin: .\test2.plugin
r(9999);

end of do-file

r(9999);

@mcaceresb
Copy link
Owner

mcaceresb commented May 16, 2018 •
edited

Ok, then the programs I referenced above are not actually the ones required. Do you have access to Visual Studio on the server?

@dgcbritto
Copy link
Author

dgcbritto commented May 16, 2018 via email

@mcaceresb
Copy link
Owner

mcaceresb commented May 16, 2018 •
edited

  • Can you open the Visual Studio Developer Command Prompt? The command you will need is msbuild
  • You will need a copy of the spookyhash repo.
  • Last, you need premake5

The steps are

cd spookyhash\build
premake5.exe vs2013

Replace vs2013 with vs2017 or similar depending on your version. Then open the VS Developer Command Prompt and run

cd spookyhash\build
msbuild SpookyHash.sln

This should generate a version of spookyhash.dll compatible with your system. Let me know how far you get.

PS: If you cannot find msbuild in your path or you can't open the VS command prompt, the spookyhash repo recommends trying to use the VS GUI. We should try as a backup just in case, but my experience was that the GUI-compiled version did not work with Stata, and I wasn't sure how to configure VS via the GUI to get it to work with Stata. But we can try if you can't get msbuild to work.

@ppaspp
Copy link

ppaspp commented May 16, 2018

sry, no visual studio

@dgcbritto
Copy link
Author

dgcbritto commented May 16, 2018 via email

@mcaceresb
Copy link
Owner

You have to download it and copy it to that location. See: https://premake.github.io/download.html (DL link). Unzip and copy the exe to the build folder.

@dgcbritto
Copy link
Author

dgcbritto commented Jun 5, 2018 via email

@mcaceresb
Copy link
Owner

No worries. I'm glad it's working on your end.

@ppoppitz was experiencing the same issue, so I reckon this might still pop up in the future. Did you ever manage to get premake5.exe to produce an .sln file and were you able to compile it on Visual Studio?

I gather the answer to both of those is no but I wanted to double check.

@mcaceresb mcaceresb reopened this Jun 5, 2018
houtanb added a commit to houtanb/stata-gtools that referenced this issue Jan 5, 2019
@houtanb
Compile spookyhash directly into plugin. mcaceresb#35
99cf2b6
houtanb added a commit to houtanb/stata-gtools that referenced this issue Jan 5, 2019
@houtanb
Compile spookyhash directly into plugin. mcaceresb#35
a98bbf8
houtanb added a commit to houtanb/stata-gtools that referenced this issue Jan 5, 2019
@houtanb
Compile spookyhash directly into plugin. mcaceresb#35
c34ee41
mcaceresb added a commit that referenced this issue Jan 6, 2019
@mcaceresb
Merge pull request #52 from houtanb/plugin_load_issue_on_windows_serv…
2f376a8 
…er_issue_30

Compile spookyhash directly into plugin. #35
@mcaceresb
Copy link
Owner

@houtanb If I did this correctly, I merged your pull request and fixed up the make script. Install from branch develop-spooky-builtin and test it out.

@houtanb
Copy link
Contributor

houtanb commented Jan 6, 2019

@mcaceresb thanks so much. We only have one computer where we can reproduce the error. I'll test that this week and get back to you.

@karldw
Copy link

karldw commented Jan 18, 2019

@mcaceresb, this worked for me. Thanks!

@mcaceresb
Copy link
Owner

Good to know this issue might be fixed.

@houtanb Did you have a chance to test it out? Compiling spookyhash directly with the plugin doesn't seem to affect gtools in any other way so it will probably make it to master, but if it didn't work for you perhaps there is something else we can try before that.

@houtanb
Copy link
Contributor

houtanb commented Jan 20, 2019

@mcaceresb my colleague has been busy with other things and has unfortunately not had the time to test it. I will be sure to let you know once he does (hopefully soon)

@houtanb
Copy link
Contributor

houtanb commented Jan 20, 2019

It would be create if @dgcbritto @ppoppitz @dtetley001 and @tanby91 could test it too, if they still have access to the machines where the bug was encountered...

@ppaspp
Copy link

ppaspp commented Jan 20, 2019 via email

@houtanb
Copy link
Contributor

houtanb commented Jan 21, 2019

@mcaceresb, just FYI, we tested it and the change works for us.

@fink42
Copy link

fink42 commented Jan 23, 2019

I have also been having this problem on both a Windows Server 2012 and a Windows 10 Pro.
What should I do to check whether the current fix solves my specific problem?
I tried uninstalling gtools and then:
net install gtools, from(https://raw.githubusercontent.com/mcaceresb/stata-gtools/develop/build/)

But I am still getting the same error:
Could not load plugin: c:\ado\plus\e\env_set_windows_v3.plugin

@mcaceresb
Copy link
Owner 

net uninstall gtools
net install gtools, from(https://raw.githubusercontent.com/mcaceresb/stata-gtools/develop/build/)

Start a new session; make sure which gtools gives version 1.2.7 and that findfile spookyhash.dll returns an error. If it still fails to load then the change isn't working for some reason.

@fink42
Copy link

fink42 commented Jan 24, 2019 •
edited

I tried the above. Uninstalling and installing from the developer branch goes smooth and the installed version is 1.2.7.
Running the command findfile spookyhash.dll returns file "spookyhash.dll" not found.
But I'm still getting Could not load plugin: c:\ado\plus\e\env_set_windows_v3.plugin

I'm happy to help in any way possible. I have access to Visual Studio on the Windows 10 Pro machine, if that could be helpful.

Update:
I tried installing gtools form the developer branch on one of my colleagues computers and it worked! The only different between the two machine that I can come up with is the Windows build version. I´m on 17134 (April 2018 Update) and the one working is on 16299 (Fall Creators Update).

@mcaceresb
Copy link
Owner

Interesting. I'll be merging this change to master in the future, since some people reported this fixed worked, but I won't close this issue.

@fink42 Do you have cygwin installed? It might be easier to try it that way. If you don't, then can you follow the steps in this comment?

@ppaspp
Copy link

ppaspp commented Jan 25, 2019

running fine here. thanks again.

@fink42
Copy link

fink42 commented Jan 25, 2019

Perfect!
When the fix is available by SSC I will test on the Windows Server 2012.

@mcaceresb I don't have cygwin. Will try generating a new spookyhash.dll following the guide from the comment. I vaguely remember trying it previously with a failing build as a result. Will give it a new shot and return with the results.

@fink42
Copy link

fink42 commented Jan 25, 2019

I managed to build a new spookyhash.dll.
I'm not quite sure what then to do with it?

@mcaceresb
Copy link
Owner

@fink42 Please uninstall gtools and go back to the SSC version. Run "findfile spookyhash.dll" and replace the file with the version you just built. In a new session, try running any gtools commands again.

@fink42
Copy link

fink42 commented Jan 25, 2019

@mcaceresb I tried installing via SSC and then replacing the spookyhash.dll, but the problem still persist. I might mention that the laptop is issued by a university. I have previously encountered certain applications and permissions being "blocked" by the IT administrators. They might have changed something with the latest version of Windows that causes gtools to fail, as it works fine on another university laptops running an older version of Windows 10.

I will contact the IT administrators and explain that it works on my colleague’s laptop but not on mine. They might know something which I don't

@BenjoSilver
Copy link

BenjoSilver commented Feb 4, 2019 •
edited

@mcaceresb Having the same problem. I can not load gtools in R Studio after installing. I have the latest R version.
the error says

load('gtools')
Error in readChar(con, 5L, useBytes = TRUE) : cannot open the connection
In addition: Warning message:
In readChar(con, 5L, useBytes = TRUE) :
cannot open compressed file 'gtools', probable reason 'No such file or directory'

My gtools version

gtools NA NA NA "yes" "3.5.2"

@mcaceresb
Copy link
Owner

@BenjoSilver I believe you are looking for the R package of the same name. This repo is about a Stata plugin called gtools. I was unaware of the identically named R package.

Good luck with the error (I should point out that I believe R's gtools is up to 3.8 these days. Maybe upgrading will help?)

@BenjoSilver
Copy link

BenjoSilver commented Feb 4, 2019 •
edited

@mcaceresb Thanks a bunch!

@mcaceresb
Copy link
Owner

@BenjoSilver By the way your error is trying to use load(gtools) instead of library(gtools), The latter loads the pacakge called gtools, the former tries to load an R object, which does not exist.

@mcaceresb
Copy link
Owner

@fink42 I'm closing this issue since it seems to be fixed for several people and following this thread is very confusing. If you still experience it please open anew bug report with the details.

mcaceresb added a commit that referenced this issue Feb 10, 2019
@mcaceresb
Merge 'develop' gtools-1.3.0 (2019-02-08); greshape, gstats, bug fixes
c22b3ac 
Features

- `greshape long` and `greshape wide` are a fast alternative to
  reshape. It additionally implements `greshape spread` and `greshape
  gather`, which are analogous to the `spread` and `gather` commands
  from R's `tidyr`.

- `gstats` is a general-purpose wrapper for misc functions.

- `gstats winsor` added for Winsorizing and trimming data.

- `gstats winsor` takes weights.

Enhancements

- Added docs, examples for `greshape` and `gstats`

- SpookyHash code compiled directly as part of the plugin. Closes #35

- Deleted all ancillary files and code related to `spookyhash.dll`

- Faster hash sort with integer bijection (two-pass radix sorts for
  smaller integers; undocumented option `_ctolerance()` allows the user
  to force the regular counting sort).

- Faster index copy when every observation is read (simply assign the
  index pointer to `st_info->index`)

- Faster hash collision checking (simply use `memcmp` to compare the
  first chunk of bytes in the group with all the other ones; previous
  versions compared each individual observation, which is nice but
  hardly better than comparing the two chunks of memory are the same).

Bug fixes

- Fixes #49; gtools looks for a free timer and skips the benchmark if
  it can't find one.

- Stata 14.0 no longer tries to load SPI version 3 (loads version 2).

- Fixes #51: `gstats winsor` no longer fails with percentiles not strictly
  between 0 and 100.

- Fixes #50: `gstats winsor` no longer fills in missing values as the
  high percentile.
@mcaceresb mcaceresb mentioned this issue May 3, 2019
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mcaceresb mcaceresb

Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
9 participants
@houtanb @mcaceresb @ppaspp @karldw @dtetley001 @fink42 @dgcbritto @tanby91 @BenjoSilver