Error: "The input line is too long" on Windows

[fmeili]

I was testing Kafka on Windows and found a limitation / problem in how the classpath is constructed with the batch file "kafka-run-class.bat" (which is used when starting zookeeper/kafka). In this batch file the classpath is constructed jar by jar with the absolute path for each jar. Because of the huge number of jar files, the Windows cmd.exe environment limit of 8191 character is reached very fast under real life installations (see: https://support.microsoft.com/de-de/help/830473/command-prompt-cmd-exe-command-line-string-limitation).

If this batch-file would use the java classpath syntax with wildcards, the problem should be solvable.
e.g changing the following lines in the "kafka-run-class.bat" from

rem Classpath addition for release
for %%i in ("%BASE_DIR%\libs\*") do (
	call :concat "%%i"
)

to

rem Classpath addition for release
call :concat "%BASE_DIR%\libs\*;"

would solve the problem.

Thanks,
Frank

[lkgendev]

Created a kafka issues about the same problem
https://issues.apache.org/jira/browse/KAFKA-6988
And made a pull request with a similar solution but it also shorten path in Unix
apache/kafka#5960

Is there a way to speed up the addition of the fix to Kafka ?

[iamsivav]

@fmeili .. Thanks Frank!!! It works!!!
I got struck by this issue when i try to set up my development environment in my windows laptop.
It is necessary to incorporate this solution ASAP.

[Diarsid]

Kafka 2.6.0 problem still exist

[zombiehugs]

Confirmed @fmeili solution does indeed work on Windows 10 and Kafka 2.7.0

[cristianhoyos66-zz]

I'm having the same issue with kafka 2.13-2.8.0

[samuel14luis]

same

[tlf30]

Hit this same issue. It would be great to get this fixed.

[shivaleo74]

Thanks, This suggestion is still working even in 2024

[fmeili1]

It's funny... six years and still not fixed although it's such a small and simple change. Maybe not many are using Kafka under Windows? - but even under Linux it would be an improvement. Let the JVM itself create the classpath at runtime instead on relying on a shell with their own specific restrictions - additionally the behavior on Windows and Linux would be exact the same if the JVM would do it.
Btw. If using the same syntax on Linux, don't forget to change the semicolon to a colon at the end of the wildcard path (beside the other obvious changes if using Linux shell)!