Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor setup.py #48

Merged
merged 1 commit into from
Oct 12, 2020
Merged

Conversation

turbaszek
Copy link
Member

This PR introduces some structure to setup.py by encapsulating
some parts of the logic into functions and creating the main
function.

Comment on lines -41 to -42
from elasticsearch import VERSION as ES_VERSION
ES_MAJOR = ES_VERSION[0]
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was never used

Comment on lines -52 to -62
elif dopip:
print("Before we get started, we need to install some modules")
print("Hang on!")
try:
subprocess.check_call(('pip3','install','elasticsearch', 'certifi', 'bcrypt'))
from elasticsearch import Elasticsearch
except:
print("Oh dear, looks like this failed :(")
print("Please install elasticsearch and certifi before you try again:")
print("pip install elasticsearch certifi")
sys.exit(-1)
Copy link
Member Author

@turbaszek turbaszek Oct 10, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Installing dependencies for users is a rather bad pattern in my opinion. Users should fix it by running the command that was provided:

pip3 install elasticsearch certifi bcrypt

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Telling users to utilize requirements.txt would probably be the best bet here. That way you can also use pipenv easily.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Totally agree, I will remove it then - I left it just to preserve the logic

Comment on lines -93 to -138
while hostname == "":
hostname = input("What is the hostname of the ElasticSearch server? [localhost]: ")
if hostname == "":
print("Using default; localhost")
hostname = "localhost"
while port < 1:
try:
port = input("What port is ElasticSearch listening on? [9200]: ")
if port == "":
print("Using default; 9200")
port = 9200
port = int(port)
except ValueError:
pass

while dbname == "":
dbname = input("What would you like to call the DB index [kibble]: ")
if dbname == "":
print("Using default; kibble")
dbname = "kibble"

while mlserver == "":
mlserver = input("What is the hostname of the outgoing mailserver? [localhost:25]: ")
if mlserver == "":
print("Using default; localhost:25")
mlserver = "localhost:25"

while shards < 1:
try:
shards = input("How many shards for the ElasticSearch index? [5]:")
if shards == "":
print("Using default; 5")
shards = 5
shards = int(shards)
except ValueError:
pass

while replicas < 0:
try:
replicas = input("How many replicas for each shard? [1]: ")
if replicas == "":
print("Using default; 1")
replicas = 1
replicas = int(replicas)
except ValueError:
pass
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of this, we can use default values in argparser arguments

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not opposed to this :) In the longer run, people can always edit the .yaml file later on if they need to make changes. Having it as CLI args would help with automated setups.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having it as CLI args would help with automated setups.

That was the main idea, see #50

@turbaszek
Copy link
Member Author

➜ python setup/setup.py -e 0.0.0.0 -a
Welcome to the Apache Kibble setup script!
Configuring Apache Kibble server with the following arguments:
- hostname: 0.0.0.0
- port: 9200
- dbname: kibble
- shards: 5
- replicas: 1

Creating index kibble_api
Creating index kibble_ci_build
Creating index kibble_ci_queue
Creating index kibble_code_commit
Creating index kibble_code_commit_unique
Creating index kibble_code_modification
Creating index kibble_evolution
Creating index kibble_file_history
Creating index kibble_forum_post
Creating index kibble_forum_topic
Creating index kibble_ghstats
Creating index kibble_im_stats
Creating index kibble_im_ops
Creating index kibble_im_msg
Creating index kibble_issue
Creating index kibble_logstats
Creating index kibble_email
Creating index kibble_mailstats
Creating index kibble_mailtop
Creating index kibble_organisation
Creating index kibble_view
Creating index kibble_publish
Creating index kibble_source
Creating index kibble_stats
Creating index kibble_social_follow
Creating index kibble_social_followers
Creating index kibble_social_follower
Creating index kibble_social_person
Creating index kibble_uisession
Creating index kibble_useraccount
Creating index kibble_message
Creating index kibble_person
Indices created!

Creating administrator account
Account created!

/Users/tomaszurbaszek/kibble/setup/../api/yaml/kibble.yaml already exists! Writing to /Users/tomaszurbaszek/kibble/setup/../api/yaml/kibble.yaml.tmp instead
Writing Kibble config to /Users/tomaszurbaszek/kibble/setup/../api/yaml/kibble.yaml.tmp

All done, Kibble should...work now :)

@turbaszek
Copy link
Member Author

@sharanf @Humbedooh @michalslowikowski00 would love to get a review on this one. Once we are ok with the proposed changes I will adjust the documentation.

@turbaszek turbaszek marked this pull request as ready for review October 10, 2020 23:17
@Humbedooh
Copy link
Member

Generally for this change. I think we should perhaps also - and this will break the api/ dir a moment, move away from separating host, port, uriprefix and ssl parameters and just accept the standard modern URL parameter instead. That is, http://localhost:9200/ or https://mydb.example.com/ES/ instead of having three or four different variables that pieces it together.

@turbaszek
Copy link
Member Author

That is, http://localhost:9200/ or https://mydb.example.com/ES/ instead of having three or four different variables that pieces it together.

I agree and as a followup I wanted to figure out something around configuration. The rough idea was to abandon the setup script in favor of providing default yaml config and documentation pointing users to change it to suits their needs.

This PR was mostly about improving code and getting familiar with the codebase 👍

@Humbedooh
Copy link
Member

We'd still need something to set up the indices, but that could be a non-interactive smaller script.
How about we merge this, and figure out the bigger picture from there on? No harm done in merging and then forgetting this file later on in favor of something new :)

@turbaszek
Copy link
Member Author

We'd still need something to set up the indices, but that could be a non-interactive smaller script.

I think the indices are configured as expected.

How about we merge this, and figure out the bigger picture from there on?

Definitely agree 👍

@Humbedooh
Copy link
Member

I'll give it a final look-over later today and merge things then.

setup/setup.py Outdated Show resolved Hide resolved
setup/setup.py Outdated Show resolved Hide resolved
This PR introduces some structure to setup.py by encapsulating
some parts of the logic into functions and creating the main
function.
def get_user_input(msg: str, secure: bool = False):
value = None
while not value:
value = getpass(msg) if secure else input(msg)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice.

@turbaszek
Copy link
Member Author

@Humbedooh are we ok with moving on with this PR?

@Humbedooh Humbedooh merged commit a2d61ac into apache:master Oct 12, 2020
@turbaszek turbaszek mentioned this pull request Oct 17, 2020
turbaszek added a commit to PolideaInternal/kibble that referenced this pull request Oct 17, 2020
This PR introduces some structure to setup.py by encapsulating
some parts of the logic into functions and creating the main
function.
turbaszek added a commit to PolideaInternal/kibble that referenced this pull request Oct 17, 2020
This PR introduces some structure to setup.py by encapsulating
some parts of the logic into functions and creating the main
function.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants