Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds support for "local" LLMs via ollama [WIP] #762

Merged
merged 17 commits into from
Jan 27, 2024

Conversation

emeryberger
Copy link
Member

  • Adds Ollama support (locally installed open source models).
  • Now persists all chosen parameters (in class "persistent") in local storage.
  • Spawns a web server to enable access to localhost when ollama is running locally. This won't work in settings where it's not possible to open a server, like Colab

TO DO:

  • Test on Jupyter and other platforms
  • Implement logic to support Amazon Bedrock usage.

scalene/scalene_utility.py Fixed Show fixed Hide fixed
scalene/scalene_profiler.py Fixed Show fixed Hide fixed
scalene/scalene_utility.py Fixed Show fixed Hide fixed
scalene/scalene_profiler.py Fixed Show fixed Hide fixed

# Open web browser to local server
webbrowser.open(f'http://localhost:{port}/')
except:

Check notice

Code scanning / CodeQL

Empty except Note

'except' clause does nothing but pass and there is no explanatory comment.

# Open web browser to local server
webbrowser.open(f'http://localhost:{port}/')
except:

Check notice

Code scanning / CodeQL

Except block handles 'BaseException' Note

Except block directly handles BaseException.
shutil.copy(file_path, os.path.join(temp_dir, 'index.html'))

# Open web browser in a new subprocess
url = f'http://localhost:{port}/'

Check notice

Code scanning / CodeQL

Unused local variable Note

Variable url is not used.
self.end_headers()
return
else:
return http.server.SimpleHTTPRequestHandler.do_GET(self)

Check warning

Code scanning / CodeQL

Use of the return value of a procedure Warning

The result of
SimpleHTTPRequestHandler.do_GET
is used even though it is always None.
server_running = True

class CustomHandler(http.server.SimpleHTTPRequestHandler):
def do_GET(self):

Check notice

Code scanning / CodeQL

Explicit returns mixed with implicit (fall through) returns Note

Mixing implicit and explicit returns may indicate an error as implicit returns always return None.
import shutil
import socket
import socketserver
import subprocess

Check notice

Code scanning / CodeQL

Unused import Note

Import of 'subprocess' is not used.
# url = f"file:///{output_fname}"
# webbrowser.open(url)
# show_browser(output_fname, SCALENE_PORT, Scalene.__orig_python)
if True:

Check warning

Code scanning / CodeQL

Constant in conditional expression or statement Warning

Testing a constant will always give the same result.
# Create a temporary directory
with tempfile.TemporaryDirectory() as temp_dir:
# Create a command with the required flags
chrome_cmd = f'{chrome_path} %s --disable-web-security --user-data-dir="{temp_dir}"'

Check failure

Code scanning / CodeQL

Potentially uninitialized local variable Error

Local variable 'chrome_path' may be used before it is initialized.
@emeryberger emeryberger merged commit 9b63cc1 into master Jan 27, 2024
13 of 14 checks passed
@emeryberger emeryberger deleted the other_vendor_support branch January 27, 2024 18:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant