Invalid requirement: '\x00'

[dan-danciu]

I ended up following this this to the letter on two different PCs:
https://serverless.com/blog/serverless-python-packaging/

But I still get the same error everywhere:

  Error --------------------------------------------------                                                                            
                                                                                                                                      
  Error: STDOUT:                                                                                                                      
                                                                                                                                      
  STDERR: ERROR: Invalid requirement: '\x00' (from line 1 of /var/task/requirements.txt)                                              
                                                                                                                                 
      at D:\git\numpy-test\node_modules\serverless-python-requirements\lib\pip.js:320:13                                              
      at Array.forEach (<anonymous>)                                                                                                  
      at installRequirements (D:\git\numpy-test\node_modules\serverless-python- 
 requirements\lib\pip.js:307:28)                        
      at installRequirementsIfNeeded (D:\git\numpy-test\node_modules\serverless-python- 
 requirements\lib\pip.js:551:3)                 
      at ServerlessPythonRequirements.installAllRequirements (D:\git\numpy- 
 test\node_modules\serverless-python-requirements\lib\pip.js
  :630:29)                                                                                                                              
      at ServerlessPythonRequirements.tryCatcher (D:\git\numpy- 
 test\node_modules\bluebird\js\release\util.js:16:23)                   
      at Promise._settlePromiseFromHandler (D:\git\numpy- 
 test\node_modules\bluebird\js\release\promise.js:547:31)                     
      at Promise._settlePromise (D:\git\numpy- 
 test\node_modules\bluebird\js\release\promise.js:604:18)                                
      at Promise._settlePromise0 (D:\git\numpy- 
 test\node_modules\bluebird\js\release\promise.js:649:10)                               
      at Promise._settlePromises (D:\git\numpy- 
 test\node_modules\bluebird\js\release\promise.js:729:18)                               
      at _drainQueueStep (D:\git\numpy-test\node_modules\bluebird\js\release\async.js:93:12)                                          
      at _drainQueue (D:\git\numpy-test\node_modules\bluebird\js\release\async.js:86:9)                                               
      at Async._drainQueues (D:\git\numpy-test\node_modules\bluebird\js\release\async.js:102:5)                                       
      at Immediate.Async.drainQueues [as _onImmediate] (D:\git\numpy- 
 test\node_modules\bluebird\js\release\async.js:15:14)            
      at processImmediate (internal/timers.js:439:21)                                                                                 
      at process.topLevelDomainCallback (domain.js:131:23)                                                                            
                                                                                                                                      
     For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.                                              

The requirements.txt is absolutely fine in my project root folder but when I look at it in the .serverless folder it looks completely messed up:

root:

.serverless:

I can not replace it as the plugin copies it every time serverless tries to package the thing so what am I doing wrong?

[devFranz]

I have the same issue. I ended up installing the modules in the root folder (not neat solution but it worked at the end).

[mpalmerjac]

I solved this problem by saving the contents of requirements.txt as ascii, not unicode. Worked fine after that.

[hariprasad0511]

I solved this problem by saving the contents of requirements.txt as ascii, not unicode. Worked fine after that.

Worked for me too.

Although I would like o understand why and if serverless itself can't handle this conversion.

[n-wagner]

The above helped fix the problem for me too, but I think I may have figured out what the underlying issue is here.

If you're running pip freeze > requirements.txt on PowerShell you have to be careful what version of PowerShell you're on. The > operator is a shortcut to Out-File and the default encoding parameter for it in Powershell 5.1 at least is Unicode which is UTF-16LE as per this specification.

Default value: Unicode

Moreover, it looks like even though this is corrected in the current latest release for PowerShell, the > operator may still output the UTF-16LE format as per this specification for PowerShell 7.1.

Default value: UTF8NoBOM

and

Out-File and the redirection operators > and >> create UTF-16LE, which notably differs from Set-Content and Add-Content.

respectively.

This doesn't appear to be an issue with this plugin, the way to fix it definitively is to execute:

pip freeze | Out-File -Encoding UTF8 requirements.txt

when generating requirements files on PowerShell or use the original redirection method on Command Prompt.

[f00f]

Converting the requirements.txt to UTF8 fixed the issue for me. Thanks for the updated PoSH command, @n-wagner.

However, I wonder where this goes wrong? python -m pip install -r .\requirements.txt works for me even with a UTF16LE file.
Would be nice for noobs (like me) if this were fixed.

[n-wagner]

@f00f i haven’t touched this in a while but if I recall correctly that isn’t the issue. It’s that the serverless plug-in rewrites the file to another directory and it assumes a UTF8 encoding when it opens it. The failure happens when it attempts to read the UTF16LE file like a UTF8 and then dump it to another requirements.txt file. So pip install may work fine but the rewrite portion of the plug-in is more brittle. This is to guard against that. Hope that helps!

[ebartan]

pip freeze | Out-File -Encoding UTF8 requirements.txt it's work it for me

The above helped fix the problem for me too, but I think I may have figured out what the underlying issue is here.

If you're running pip freeze > requirements.txt on PowerShell you have to be careful what version of PowerShell you're on. The > operator is a shortcut to Out-File and the default encoding parameter for it in Powershell 5.1 at least is Unicode which is UTF-16LE as per this specification.

Default value: Unicode

Moreover, it looks like even though this is corrected in the current latest release for PowerShell, the > operator may still output the UTF-16LE format as per this specification for PowerShell 7.1.

Default value: UTF8NoBOM

and

Out-File and the redirection operators > and >> create UTF-16LE, which notably differs from Set-Content and Add-Content.

respectively.

This doesn't appear to be an issue with this plugin, the way to fix it definitively is to execute:

pip freeze | Out-File -Encoding UTF8 requirements.txt

when generating requirements files on PowerShell or use the original redirection method on Command Prompt.

[f00f]

@pgrzesik: Would a "fix" be to check the file encoding before opening it as utf8, and if it's a different encoding, issue an error and exit?
(note: the offending file read is probably in function generateRequirementsFile in the file lib/pip.js)