Issue with plugin upload functionality #762

samwmarsh · 2024-02-28T18:35:19Z

Describe the bug
Currently, the offline pluginset uploaded by tenable to https://plugins.nessus.org/get.php?f=sc-plugins-diff.tar.gz (passing username/password strings) is too large for ssl.py to handle (larger than 32-bit limit). This causes issues when using the following code to upload plugins:

    with open('sc-plugins-diff.tar.gz', 'rb') as pluginfile:
        print(sc.feeds.process('active', pluginfile))

To Reproduce
Steps to reproduce the behavior:

Download sc-plugins-diff.tar.gz
Run the above python code, having authenticated to an sc instance
See error

    print(sc.feeds.process('active', pluginfile))
  File "/var/lib/jenkins/workspace/Nessus/Plugin-Uploader/venv/lib64/python3.6/site-packages/tenable/sc/feeds.py", line 100, in process
    filename = self._api.files.upload(fobj)
  File "/var/lib/jenkins/workspace/Nessus/Plugin-Uploader/venv/lib64/python3.6/site-packages/tenable/sc/files.py", line 33, in upload
    'Filedata': fobj}).json()['response']['filename']
  File "/var/lib/jenkins/workspace/Nessus/Plugin-Uploader/venv/lib64/python3.6/site-packages/restfly/session.py", line 631, in post
    return self._req('POST', path, **kwargs)
  File "/var/lib/jenkins/workspace/Nessus/Plugin-Uploader/venv/lib64/python3.6/site-packages/restfly/session.py", line 516, in _req
    timeout=self._timeout, **kwargs)
  File "/var/lib/jenkins/workspace/Nessus/Plugin-Uploader/venv/lib64/python3.6/site-packages/requests/sessions.py", line 529, in request
    resp = self.send(prep, **send_kwargs)
  File "/var/lib/jenkins/workspace/Nessus/Plugin-Uploader/venv/lib64/python3.6/site-packages/requests/sessions.py", line 645, in send
    r = adapter.send(request, **kwargs)
  File "/var/lib/jenkins/workspace/Nessus/Plugin-Uploader/venv/lib64/python3.6/site-packages/requests/adapters.py", line 450, in send
    timeout=timeout
  File "/var/lib/jenkins/workspace/Nessus/Plugin-Uploader/venv/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 722, in urlopen
    chunked=chunked,
  File "/var/lib/jenkins/workspace/Nessus/Plugin-Uploader/venv/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 416, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/var/lib/jenkins/workspace/Nessus/Plugin-Uploader/venv/lib64/python3.6/site-packages/urllib3/connection.py", line 244, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/usr/lib64/python3.6/http/client.py", line 1254, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib64/python3.6/http/client.py", line 1300, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib64/python3.6/http/client.py", line 1249, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib64/python3.6/http/client.py", line 1075, in _send_output
    self.send(chunk)
  File "/usr/lib64/python3.6/http/client.py", line 996, in send
    self.sock.sendall(data)
  File "/usr/lib64/python3.6/ssl.py", line 968, in sendall
    v = self.send(byte_view[count:])
  File "/usr/lib64/python3.6/ssl.py", line 937, in send
    return self._sslobj.write(data)
  File "/usr/lib64/python3.6/ssl.py", line 601, in write
    return self._sslobj.write(data)
OverflowError: string longer than 2147483647 bytes

Expected behavior
Plugin to upload and process without failing out.

Screenshots
edit: This seems to cause an error when uploading through the UI too, I've raised as a case with Tenable directly.

System Information (please complete the following information):

OS: tenable.sc running on CentOS 7.9, pytenable running within Jenkins
Architecture 64-bit
Version latest

The text was updated successfully, but these errors were encountered:

kintaroju · 2024-02-28T22:19:03Z

Hi,

I too am having the EXACT same issue, as I was trying to upload the plugin file manually provided by Tenable and unfortunately all the Tenable.SC plugins have exceed 2GB now. Last month it was just under the 2GB limit. I've sent a ticket to Tenable support and see if they have a hotfix or something to apply to Tenable.SC to fix this issue. I will try to update this github ticket accordingly.

For the methods to update the plugins I did exactly what you did which was via webgui and pyTenable script, both failed.

aseemsavio · 2024-02-29T05:48:13Z

The Security Center team is aware and is working on this issue, and they're planning to fix it in SC v6.4.0, which is expected to come out in the first half of this year.

kintaroju · 2024-02-29T05:50:57Z

That is great to hear so the bigger question is this affecting all versions of tenable.sc or just 6.3.0?

aseemsavio · 2024-02-29T06:04:28Z

@kintaroju this seems to affect all SC versions. In the mean time, the SC team has put out a workaround for this issue. You may have to reach out to Customer Support for the same.

kintaroju · 2024-02-29T06:19:27Z

Good to know thanks for the info. I will remind the tech about not requiring the legendary diagnostics file to deal with this issue lol

kintaroju · 2024-03-01T17:56:14Z

@aseemsavio I finally got a response from Tenable and they provided me a workaround (can't post the workaround as they indicated they will post a KB on this), except it only worked for the WebGUI and not via pyTenable. So the question is for a pyTenable perspective how can I upload the plugins, or do we have to wait till there is an official patch before pyTenable can be fixed as well?

samwmarsh · 2024-03-05T15:53:26Z

@kintaroju they gave me 2 KBs, neither worked, they seem to have managed to minimize the plugin-set size, though this doesn't work with pyTenable.

samwmarsh · 2024-04-10T12:31:07Z

Hey,
I'm now able to upload manually, however I'm getting the following error from pytenable:

Requests Library Error: ('Connection aborted.', timeout('The write operation timed out',))

samwmarsh · 2024-05-10T14:27:52Z

We're being hit by the 32-bit limit again for this.

OverflowError: string longer than 2147483647 bytes

SteveMcGrath · 2024-05-10T15:54:13Z

looks like the error is related to how requests is handling files differently than raw encoded data. Took aq little digging but this seems to be related:

https://stackoverflow.com/questions/22915295/python-requests-post-and-big-content/22915488#22915488

SteveMcGrath · 2024-05-10T20:18:11Z

After some testing the memory usage using the file upload went from 2G to to under 50M. I think this should long-term solve this issue on the library-front.

samwmarsh · 2024-05-21T15:48:44Z

Thanks for resolving this @SteveMcGrath - do we have an estimated release date? I notice the last release is >1mo ago so presumably its soon-ish?

SteveMcGrath · 2024-05-21T16:38:02Z

yeah, just trying to work through the backlog of PRs before releasing. If you're in a rush, you can always test with a pip install from git:

pip install git+https://github.com/tenable/pytenable.git

samwmarsh · 2024-05-21T16:58:57Z

All good, we're in an airgapped network so happy to wait. Soon-ish is good enough for me!

aseemsavio added Tenable.sc Tenable.sc Package upstream-api The issue is related to the calling API or API Docs labels Feb 29, 2024

SteveMcGrath added bug This is a bug with the pyTenable library upstream-lib and removed upstream-api The issue is related to the calling API or API Docs labels May 10, 2024

SteveMcGrath added a commit that referenced this issue May 10, 2024

Updated SC Files method as per #762

3a8038f

SteveMcGrath mentioned this issue May 10, 2024

Updated SC Files method as per #762 #771

Merged

9 tasks

SteveMcGrath added a commit that referenced this issue May 14, 2024

Updated SC Files method as per #762

1af134f

SteveMcGrath added a commit that referenced this issue May 14, 2024

Updated SC Files method as per #762

38301bd

SteveMcGrath added a commit that referenced this issue May 14, 2024

Updated SC Files method as per #762

33e4cb3

SteveMcGrath closed this as completed in #771 May 14, 2024

SteveMcGrath added a commit that referenced this issue May 14, 2024

Updated SC Files method as per #762

08cce90

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Issue with plugin upload functionality #762

Issue with plugin upload functionality #762

samwmarsh commented Feb 28, 2024 •

edited

kintaroju commented Feb 28, 2024

aseemsavio commented Feb 29, 2024

kintaroju commented Feb 29, 2024

aseemsavio commented Feb 29, 2024

kintaroju commented Feb 29, 2024

kintaroju commented Mar 1, 2024

samwmarsh commented Mar 5, 2024 •

edited

samwmarsh commented Apr 10, 2024

samwmarsh commented May 10, 2024

SteveMcGrath commented May 10, 2024

SteveMcGrath commented May 10, 2024

samwmarsh commented May 21, 2024

SteveMcGrath commented May 21, 2024

samwmarsh commented May 21, 2024

Issue with plugin upload functionality #762

Issue with plugin upload functionality #762

Comments

samwmarsh commented Feb 28, 2024 • edited

kintaroju commented Feb 28, 2024

aseemsavio commented Feb 29, 2024

kintaroju commented Feb 29, 2024

aseemsavio commented Feb 29, 2024

kintaroju commented Feb 29, 2024

kintaroju commented Mar 1, 2024

samwmarsh commented Mar 5, 2024 • edited

samwmarsh commented Apr 10, 2024

samwmarsh commented May 10, 2024

SteveMcGrath commented May 10, 2024

SteveMcGrath commented May 10, 2024

samwmarsh commented May 21, 2024

SteveMcGrath commented May 21, 2024

samwmarsh commented May 21, 2024

samwmarsh commented Feb 28, 2024 •

edited

samwmarsh commented Mar 5, 2024 •

edited