Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Added windows batch files #94

Merged
merged 4 commits into from

2 participants

@ctasada

Allows to execute Voldemort directly in a Windows development system

@jayjwylie jayjwylie was assigned
@jayjwylie jayjwylie was assigned
@jayjwylie

Sorry I am slow on reviewing this request. I need to find a windows dev box to test the changes.

@jayjwylie

OK. I got a windows dev box and have made some progress verifying the .bat files. At this points, I have a couple concerns and am blocked on completing my testing of the .bat files.

Concern I: Paths with spaces in them do not seem to be tolerated. I initially had everything in "C:\Documents and Settings...". This lead to badness in the bin/voldemort-server.bat (and I assume the other .bat files too). Is this easy to fix? to document? Or, does everyone on windows just know to avoid using paths with spaces in them.

Concern II: I am happy to do this initial test of this commit. We will not re-check these .bat files when we do new releases though. Are you OK being the community's windows person / owner of these .bat files? If so, then maybe change the headers in them to indicate your role (and the voldemort team at LinkedIn's lack of a role) wrt windows deployments.

I am still blocked in the following manner. When I invoke bin/voldemort-server.bat home v_home v_config the server starts, but then hits a problem midway through initialization:

[2012-11-01 13:13:05,741 voldemort.server.storage.StorageService] INFO Opening store 'test-view' (view).
Exception in thread "main" java.lang.IllegalArgumentException: java.lang.ClassNotFoundException: voldemort.store.views.UpperCaseView
at voldemort.utils.ReflectUtils.loadClass(ReflectUtils.java:59)
at voldemort.store.views.ViewStorageConfiguration.loadTransformation(ViewStorageConfiguration.java:86)
at voldemort.store.views.ViewStorageConfiguration.getStore(ViewStorageConfiguration.java:59)
at voldemort.server.storage.StorageService.openStore(StorageService.java:342)
at voldemort.server.storage.StorageService.startInner(StorageService.java:298)
at voldemort.server.AbstractService.start(AbstractService.java:66)
at voldemort.server.VoldemortServer.startInner(VoldemortServer.java:255)
at voldemort.server.AbstractService.start(AbstractService.java:66)
at voldemort.server.VoldemortServer.main(VoldemortServer.java:307)
Caused by: java.lang.ClassNotFoundException: voldemort.store.views.UpperCaseView
at java.net.URLClassLoader$1.run(Unknown Source)
at java.net.URLClassLoader$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Unknown Source)
at voldemort.utils.ReflectUtils.loadClass(ReflectUtils.java:57)
... 8 more
I believe I may be missing some ant configuration or other setting. I cannot figure out why UpperCaseView is not being found.

The .bat file sets up the following classpath:
".";"C:\jwylie\voldemort\bin..\lib\avro-1.4.0.jar";"C:\jwylie\voldemort\bin..\lib\catalina-ant.jar";"C:\jwylie\voldemort\bin..\lib\colt-1.2.0.jar";"C:\jwylie\voldemort\bin..\lib\commons-codec-1.3.jar";"C:\jwylie\voldemort\bin..\lib\commons-collections-3.2.1.jar";"C:\jwylie\voldemort\bin..\lib\commons-dbcp-1.2.2.jar";"C:\jwylie\voldemort\bin..\lib\commons-io-1.4.jar";"C:\jwylie\voldemort\bin..\lib\commons-lang-2.4.jar";"C:\jwylie\voldemort\bin..\lib\commons-logging-1.1.1.jar";"C:\jwylie\voldemort\bin..\lib\commons-pool-1.5.2.jar";"C:\jwylie\voldemort\bin..\lib\compress-lzf-0.9.1.jar";"C:\jwylie\voldemort\bin..\lib\google-collect-1.0.jar";"C:\jwylie\voldemort\bin..\lib\httpclient-4.1.2.jar";"C:\jwylie\voldemort\bin..\lib\httpcore-4.1.2.jar";"C:\jwylie\voldemort\bin..\lib\jackson-core-asl-1.4.0.jar";"C:\jwylie\voldemort\bin..\lib\jackson-mapper-asl-1.4.0.jar";"C:\jwylie\voldemort\bin..\lib\jdom-1.1.jar";"C:\jwylie\voldemort\bin..\lib\je-4.1.17.jar";"C:\jwylie\voldemort\bin..\lib\jetty-6.1.18.jar";"C:\jwylie\voldemort\bin..\lib\jetty-util-6.1.18.jar";"C:\jwylie\voldemort\bin..\lib\jline-0.9.94.jar";"C:\jwylie\voldemort\bin..\lib\jna.jar";"C:\jwylie\voldemort\bin..\lib\jopt-simple-3.1.jar";"C:\jwylie\voldemort\bin..\lib\junit-4.6.jar";"C:\jwylie\voldemort\bin..\lib\libthrift-0.5.0.jar";"C:\jwylie\voldemort\bin..\lib\log4j-1.2.15.jar";"C:\jwylie\voldemort\bin..\lib\mockito-all-1.8.5.jar";"C:\jwylie\voldemort\bin..\lib\paranamer-2.1.jar";"C:\jwylie\voldemort\bin..\lib\protobuf-java-2.3.0.jar";"C:\jwylie\voldemort\bin..\lib\servlet-api-2.5.jar";"C:\jwylie\voldemort\bin..\lib\slf4j-api-1.5.6.jar";"C:\jwylie\voldemort\bin..\lib\slf4j-log4j12-1.5.6.jar";"C:\jwylie\voldemort\bin..\lib\snappy-0.2.jar";"C:\jwylie\voldemort\bin..\lib\velocity-1.6.2.jar";"C:\jwylie\voldemort\bin..\lib\xercesImpl-2.9.1.jar";"C:\jwylie\voldemort\bin..\dist\voldemort-0.96hb1-src.jar";"C:\jwylie\voldemort\bin..\dist\voldemort-0.96hb1.jar";"C:\jwylie\voldemort\bin..\dist\voldemort-contrib-0.96hb1.jar";"C:\jwylie\voldemort\bin..\dist\voldemort-test-0.96hb1.jar":C:\jwylie\voldemort\bin..\dist\resources

Any pointers on the steps necessary to successfully run bin\voldemort-server.bat are much appreciated.

Thanks,
Jay

@ctasada

Hi Jay,

I finally found the time to attack again this issue. I already fixed the problem with the spaces and I'm checking that everything is working as expected, but I'm lost with your comment about the

When I invoke bin/voldemort-server.bat home v_home v_config the server starts, but then hits a problem midway through initialization:

First I checked the voldemort-server.sh and I saw that only supports 1 argument. Then I checked the VoldemortServer.java and I saw that supports 2 arguments. But I don't see anywhere support for 3 arguments. Can you help me to understand what are you trying to do there?

@ctasada

Regarding your second concern I have no problem being the contact/maintainer of these files, how do you suggest to modify the headers?

@jayjwylie

Hi Carlos,

First I checked the voldemort-server.sh and I saw that only supports 1 argument. Then I checked the VoldemortServer.java and I saw that supports 2 arguments. But I don't see anywhere support for 3 arguments. Can you help me to understand what are you trying to do there?

This was a typo in my comment. Sorry! I only had the two arguments 'v_home and v_config'. In trying to make my comment clearer by adding the 'v_' prefix, I mangled the comment. Sorry.

I'll download your update and test the fix that handles spaces in paths.

I am not sure how to modify headers to better attribute ownership/responsibility for code. I'll ask around to get a sense of best practice. Suggestions are welcome.

Thanks,
Jay

@ctasada

Hi Jay,

Any progress?

Regards,
Carlos

@jayjwylie
@jayjwylie

Carlos,

This 90% looks good to me. At this point, I just have an "administrative" issue sI want addressed.

It is really hard for us to test windows stuff. We can get windows boxes (with some effort) to test with, but we always start from scratch for environment. Basically, we cannot test the windows stuff (even a simple BAT file) easily. So, I'd like the comment at the top to clearly note that the BAT files are not tested with each release.

Also, so long as the license stuff is stated as Apache, the copyright can be listed for the person/company that wrote the code.

How does the below comment block look to you. Please tweak the copyright holder as needed and the statement that the file is not tested with each release as needed. Then I can merge.

sorry this took so long.

Cheers,
Jay

REM
REM Copyright 2013 Carlos Tasada
REM
REM Licensed under the Apache License, Version 2.0 (the "License");
REM you may not use this file except in compliance with the License.
REM You may obtain a copy of the License at
REM
REM http://www.apache.org/licenses/LICENSE-2.0
REM
REM Unless required by applicable law or agreed to in writing, software
REM distributed under the License is distributed on an "AS IS" BASIS,
REM WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
REM See the License for the specific language governing permissions and
REM limitations under the License.
REM
REM ** This Windows BAT file is not tested with each Voldemort release. **

@ctasada ctasada Update Windows Batch files
Updated copyright info to make clear that these files are not maintained
by LinkedIn
0d9a47d
@ctasada

Hi Jay,

I've updated the files as requested. I think your proposal was good.

Regards,
Carlos.

@jayjwylie jayjwylie merged commit 8c6d25d into voldemort:master
@jayjwylie jayjwylie was unassigned by ctasada
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Sep 12, 2012
  1. @ctasada

    Added windows batch files

    ctasada authored
Commits on Sep 13, 2012
  1. @ctasada
Commits on Nov 26, 2012
  1. @ctasada
Commits on Feb 19, 2013
  1. @ctasada

    Update Windows Batch files

    ctasada authored
    Updated copyright info to make clear that these files are not maintained
    by LinkedIn
This page is out of date. Refresh to see the latest.
View
46 bin/run-class.bat
@@ -0,0 +1,46 @@
+@echo off
+
+REM
+REM Copyright 2013 Carlos Tasada
+REM
+REM Licensed under the Apache License, Version 2.0 (the "License");
+REM you may not use this file except in compliance with the License.
+REM You may obtain a copy of the License at
+REM
+REM http://www.apache.org/licenses/LICENSE-2.0
+REM
+REM Unless required by applicable law or agreed to in writing, software
+REM distributed under the License is distributed on an "AS IS" BASIS,
+REM WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+REM See the License for the specific language governing permissions and
+REM limitations under the License.
+REM
+REM ** This Windows BAT file is not tested with each Voldemort release. **
+
+set argC=0
+for %%a in (%*) do set /a argC+=1
+if %argC% geq 1 goto :continue
+echo %0 java-class-name [options]
+goto :eof
+:continue
+
+SET BASE_DIR=%~dp0..
+SET CLASSPATH=.
+
+set VOLDEMORT_CONFIG_DIR=%1%/config
+
+for %%j in ("%BASE_DIR%\dist\*.jar") do (call :append_classpath "%%j")
+for %%j in ("%BASE_DIR%\lib\*.jar") do (call :append_classpath "%%j")
+set CLASSPATH=%CLASSPATH%;"%BASE_DIR%\dist\resources"
+goto :run
+
+:append_classpath
+set CLASSPATH=%CLASSPATH%;%1
+goto :eof
+
+:run
+if "%VOLD_OPTS%" == "" set "VOLD_OPTS=-Xmx2G -server -Dcom.sun.management.jmxremote"
+java -Dlog4j.configuration=%VOLDEMORT_CONFIG_DIR%\log4j.properties %VOLD_OPTS% -cp %CLASSPATH% %*
+
+endlocal
+:eof
View
22 bin/voldemort-admin-tool.bat
@@ -0,0 +1,22 @@
+@echo off
+
+REM
+REM Copyright 2013 Carlos Tasada
+REM
+REM Licensed under the Apache License, Version 2.0 (the "License");
+REM you may not use this file except in compliance with the License.
+REM You may obtain a copy of the License at
+REM
+REM http://www.apache.org/licenses/LICENSE-2.0
+REM
+REM Unless required by applicable law or agreed to in writing, software
+REM distributed under the License is distributed on an "AS IS" BASIS,
+REM WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+REM See the License for the specific language governing permissions and
+REM limitations under the License.
+REM
+REM ** This Windows BAT file is not tested with each Voldemort release. **
+
+SET BASE_DIR=%~dp0..
+
+call "%BASE_DIR%/bin/run-class.bat" voldemort.VoldemortAdminTool %*
View
22 bin/voldemort-performance-tool.bat
@@ -0,0 +1,22 @@
+@echo off
+
+REM
+REM Copyright 2013 Carlos Tasada
+REM
+REM Licensed under the Apache License, Version 2.0 (the "License");
+REM you may not use this file except in compliance with the License.
+REM You may obtain a copy of the License at
+REM
+REM http://www.apache.org/licenses/LICENSE-2.0
+REM
+REM Unless required by applicable law or agreed to in writing, software
+REM distributed under the License is distributed on an "AS IS" BASIS,
+REM WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+REM See the License for the specific language governing permissions and
+REM limitations under the License.
+REM
+REM ** This Windows BAT file is not tested with each Voldemort release. **
+
+SET BASE_DIR=%~dp0..
+
+call "%BASE_DIR%/bin/run-class.bat" voldemort.performance.benchmark.Benchmark %*
View
22 bin/voldemort-rebalance.bat
@@ -0,0 +1,22 @@
+@echo off
+
+REM
+REM Copyright 2013 Carlos Tasada
+REM
+REM Licensed under the Apache License, Version 2.0 (the "License");
+REM you may not use this file except in compliance with the License.
+REM You may obtain a copy of the License at
+REM
+REM http://www.apache.org/licenses/LICENSE-2.0
+REM
+REM Unless required by applicable law or agreed to in writing, software
+REM distributed under the License is distributed on an "AS IS" BASIS,
+REM WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+REM See the License for the specific language governing permissions and
+REM limitations under the License.
+REM
+REM ** This Windows BAT file is not tested with each Voldemort release. **
+
+SET BASE_DIR=%~dp0..
+
+call "%BASE_DIR%/bin/run-class.bat" voldemort.client.rebalance.RebalanceCLI %*
View
49 bin/voldemort-server.bat
@@ -0,0 +1,49 @@
+@echo off
+
+REM
+REM Copyright 2013 Carlos Tasada
+REM
+REM Licensed under the Apache License, Version 2.0 (the "License");
+REM you may not use this file except in compliance with the License.
+REM You may obtain a copy of the License at
+REM
+REM http://www.apache.org/licenses/LICENSE-2.0
+REM
+REM Unless required by applicable law or agreed to in writing, software
+REM distributed under the License is distributed on an "AS IS" BASIS,
+REM WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+REM See the License for the specific language governing permissions and
+REM limitations under the License.
+REM
+REM ** This Windows BAT file is not tested with each Voldemort release. **
+
+set argC=0
+for %%a in (%*) do set /a argC+=1
+if %argC% leq 2 goto :continue
+echo USAGE: bin/voldemort-server.bat [voldemort_home_dir] [voldemort_config_dir]
+goto :eof
+:continue
+
+setlocal
+
+SET BASE_DIR=%~dp0..
+SET CLASSPATH=.
+
+set VOLDEMORT_CONFIG_DIR=%1%/config
+rem call %VOLDEMORT_CONFIG_DIR%/voldemort-env.bat
+
+for %%j in ("%BASE_DIR%\dist\*.jar") do (call :append_classpath "%%j")
+for %%j in ("%BASE_DIR%\lib\*.jar") do (call :append_classpath "%%j")
+set CLASSPATH=%CLASSPATH%:"%BASE_DIR%\dist\resources"
+goto :run
+
+:append_classpath
+set CLASSPATH=%CLASSPATH%;%1
+goto :eof
+
+:run
+if "%VOLD_OPTS%" == "" set "VOLD_OPTS=-Xmx2G -server -Dcom.sun.management.jmxremote"
+java %VOLD_OPTS% -cp %CLASSPATH% voldemort.server.VoldemortServer %*
+
+endlocal
+:eof
View
34 bin/voldemort-shell.bat
@@ -0,0 +1,34 @@
+@echo off
+
+REM
+REM Copyright 2013 Carlos Tasada
+REM
+REM Licensed under the Apache License, Version 2.0 (the "License");
+REM you may not use this file except in compliance with the License.
+REM You may obtain a copy of the License at
+REM
+REM http://www.apache.org/licenses/LICENSE-2.0
+REM
+REM Unless required by applicable law or agreed to in writing, software
+REM distributed under the License is distributed on an "AS IS" BASIS,
+REM WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+REM See the License for the specific language governing permissions and
+REM limitations under the License.
+REM
+REM ** This Windows BAT file is not tested with each Voldemort release. **
+
+set argC=0
+for %%a in (%*) do set /a argC+=1
+if %argC% geq 2 goto :continue
+echo "USAGE: bin/voldemort-shell.bat store_name bootstrap_url [command_file] [--client-zone-id <zone-id>]"
+goto :eof
+:continue
+
+setlocal
+SET BASE_DIR=%~dp0..
+
+call "%BASE_DIR%\bin\run-class.bat" jline.ConsoleRunner voldemort.VoldemortClientShell %*
+
+endlocal
+
+:eof
Something went wrong with that request. Please try again.