Tmux script #771

noah-iversen · 2023-05-26T20:37:08Z

Tmux script

What user problem are we solving?

Not a user problem, it's an internal solution.

What solution does this PR provide?

This solution provides a tmux script that seamlessly spins up both the backend and frontend instances. The instances will run in the background.

Testing Methodology
How did you test your changes and verify that existing
functionality is not broken

Started with a fresh repo pull and fresh install of WSL and ran through instructions to deploy the script. No issues with on my machine. Currently working on both Mac and PC.

Any other considerations
When deploying on Windows, since there's more steps there's more room for package issues and problems with WSL. But the script itself isn't faulty.

* Fixed exempt paths, init swagger docs * open api documentation - initial start * putting open api docs into openapi directory within backend * remove debug print * rename some open api yamls * open api github action * 🎨 Format Python code with psf/black * github action fix * gh action fix * open api validator install cmd fix * spectral * rename file * typo * fix * debug * swagger cli * fix * typo * gh action debug * --no color args * ensure all files get validated by swagger-cli * created yaml file for default dataset * Added yaml files for all apis and updates dlapi.openapi.yaml * removed duplicates from dlpai and edited updateUserProgressData descriptions * removed duplicate keys from dlpapi * corrected types --------- Co-authored-by: Faris Durrani <fdurrani@mythics.com> Co-authored-by: karkir0003 <karkir0003@users.noreply.github.com> Co-authored-by: Austin Tsang <47958374+Dashrekker@users.noreply.github.com>

* attempt at ecr put image: * relaxed ecr rules to push every time * skipped tests * new commands * mini docker * complete example * complete * example * reverted example * fixed Dockerfile

github-actions

Remaining comments which cannot be posted as a review comment to avoid GitHub Rate Limit

pyright

backend/endpoints/s3_edpoints.py|4 col 8| Import "boto3" could not be resolved (reportMissingImports)
backend/endpoints/s3_edpoints.py|5 col 6| Import "flask" could not be resolved (reportMissingImports)
backend/endpoints/s3_edpoints.py|6 col 6| Import "flask" could not be resolved (reportMissingImports)
backend/endpoints/s3_edpoints.py|7 col 6| Import "werkzeug.utils" could not be resolved (reportMissingImports)
backend/endpoints/test_endpoints.py|1 col 6| Import "flask" could not be resolved (reportMissingImports)
backend/endpoints/train_endpoints.py|7 col 6| Import "flask" could not be resolved (reportMissingImports)
backend/endpoints/train_endpoints.py|8 col 6| Import "flask" could not be resolved (reportMissingImports)
backend/endpoints/train_endpoints.py|36 col 8| Import "boto3" could not be resolved (reportMissingImports)
backend/endpoints/trainspace_endpoints.py|3 col 6| Import "flask" could not be resolved (reportMissingImports)
backend/endpoints/trainspace_endpoints.py|4 col 6| Import "flask" could not be resolved (reportMissingImports)
backend/endpoints/trainspace_endpoints.py|42 col 25| Argument missing for parameter "trainspace_data" (reportGeneralTypeIssues)
backend/endpoints/utils.py|9 col 41| Function with declared type of "dict[Unknown, Unknown]" must return value on all code paths
Type "None" cannot be assigned to type "dict[Unknown, Unknown]" (reportGeneralTypeIssues)

github-actions · 2023-05-26T20:37:43Z

backend/driver.py

-    write_to_bucket,
-)
-from backend.middleware import middleware
+from flask import Blueprint, Flask, send_from_directory


🚫 [pyright] _{reported by reviewdog 🐶}
Import "flask" could not be resolved (reportMissingImports)

github-actions · 2023-05-26T20:37:43Z

backend/aws_helpers/dynamo_db_utils/file_upload_db.py

+    changevar,
+)
+from backend.common.constants import FILE_UPLOAD_TABLE_NAME, AWS_REGION
+from boto3.dynamodb.conditions import Key


🚫 [pyright] _{reported by reviewdog 🐶}
Import "boto3.dynamodb.conditions" could not be resolved (reportMissingImports)

github-actions · 2023-05-26T20:37:44Z

backend/aws_helpers/dynamo_db_utils/file_upload_db.py

+    uid: str
+    created: str
+    filename: str
+    ttl: int = None


🚫 [pyright] _{reported by reviewdog 🐶}
Expression of type "None" cannot be assigned to declared type "int"
Type "None" cannot be assigned to type "int" (reportGeneralTypeIssues)

github-actions · 2023-05-26T20:37:44Z

backend/aws_helpers/dynamo_db_utils/file_upload_db.py

+
+
+@enumclass(
+    DataClass=FileUploadData,


🚫 [pyright] _{reported by reviewdog 🐶}
Argument of type "Type[FileUploadData]" cannot be assigned to parameter "DataClass" of type "BaseData" in function "enumclass"
"Type[type]" is incompatible with "Type[BaseData]" (reportGeneralTypeIssues)

github-actions · 2023-05-26T20:37:44Z

backend/aws_helpers/dynamo_db_utils/file_upload_db.py

+
+
+@changevar(
+    DataClass=FileUploadData, EnumClass=FileUploadEnums, partition_key=PRIMARY_KEY


🚫 [pyright] _{reported by reviewdog 🐶}
Argument of type "Type[FileUploadData]" cannot be assigned to parameter "DataClass" of type "BaseData" in function "changevar"
"Type[type]" is incompatible with "Type[BaseData]" (reportGeneralTypeIssues)

github-actions · 2023-05-26T20:37:47Z

backend/endpoints/aws_endpoints.py

+        print(f"sqs outcome: {queue_send_outcome}")
+        status_code = queue_send_outcome["ResponseMetadata"]["HTTPStatusCode"]
+        if status_code != 200:
+            return send_error("Your training request couldn't be added to the queue")


🚫 [pyright] _{reported by reviewdog 🐶}
Expression of type "tuple[str, Literal[400]]" cannot be assigned to return type "str"
"tuple[str, Literal[400]]" is incompatible with "str" (reportGeneralTypeIssues)

github-actions · 2023-05-26T20:37:48Z

backend/endpoints/aws_endpoints.py

+            return send_success(
+                {"message": "Successfully added your training request to the queue"}
+            )


🚫 [pyright] _{reported by reviewdog 🐶}
Expression of type "tuple[str, Literal[200]]" cannot be assigned to return type "str"
"tuple[str, Literal[200]]" is incompatible with "str" (reportGeneralTypeIssues)

github-actions · 2023-05-26T20:37:48Z

backend/endpoints/aws_endpoints.py

+                {"message": "Successfully added your training request to the queue"}
+            )
+    except Exception:
+        return send_error("Failed to queue data")


🚫 [pyright] _{reported by reviewdog 🐶}
Expression of type "tuple[str, Literal[400]]" cannot be assigned to return type "str"
"tuple[str, Literal[400]]" is incompatible with "str" (reportGeneralTypeIssues)

github-actions · 2023-05-26T20:37:48Z

backend/endpoints/dataset_endpoints.py

@@ -0,0 +1,71 @@
+import traceback
+
+from flask import Blueprint


🚫 [pyright] _{reported by reviewdog 🐶}
Import "flask" could not be resolved (reportMissingImports)

github-actions · 2023-05-26T20:37:48Z

backend/endpoints/dataset_endpoints.py

+import traceback
+
+from flask import Blueprint
+from flask import request


🚫 [pyright] _{reported by reviewdog 🐶}
Import "flask" could not be resolved (reportMissingImports)

karkir0003 · 2023-05-26T20:38:27Z

@noah-iversen u should set target to nextjs, NOT dev

…ayground into tmux-script

github-actions

Remaining comments which cannot be posted as a review comment to avoid GitHub Rate Limit

pyright

backend/driver.py|304 col 14| "boto3" is not defined (reportUndefinedVariable)
backend/driver.py|310 col 15| "traceback" is not defined (reportUndefinedVariable)
backend/driver.py|318 col 35| "request" is not defined (reportUndefinedVariable)
backend/driver.py|324 col 15| "traceback" is not defined (reportUndefinedVariable)
backend/driver.py|331 col 35| "request" is not defined (reportUndefinedVariable)
backend/driver.py|333 col 18| "getAllUserExecutionsData" is not defined (reportUndefinedVariable)
backend/driver.py|336 col 15| "traceback" is not defined (reportUndefinedVariable)
backend/driver.py|343 col 35| "request" is not defined (reportUndefinedVariable)
backend/driver.py|345 col 22| "get_presigned_url_from_exec_file" is not defined (reportUndefinedVariable)
backend/driver.py|346 col 13| "EXECUTION_BUCKET_NAME" is not defined (reportUndefinedVariable)
backend/driver.py|348 col 20| "get_presigned_url_from_exec_file" is not defined (reportUndefinedVariable)
backend/driver.py|349 col 13| "EXECUTION_BUCKET_NAME" is not defined (reportUndefinedVariable)
backend/driver.py|351 col 22| "get_presigned_url_from_exec_file" is not defined (reportUndefinedVariable)
backend/driver.py|352 col 13| "EXECUTION_BUCKET_NAME" is not defined (reportUndefinedVariable)
backend/driver.py|358 col 15| "traceback" is not defined (reportUndefinedVariable)
backend/driver.py|367 col 16| "request" is not defined (reportUndefinedVariable)
backend/driver.py|370 col 45| "secure_filename" is not defined (reportUndefinedVariable)
backend/driver.py|377 col 15| "traceback" is not defined (reportUndefinedVariable)
backend/driver.py|390 col 33| "request" is not defined (reportUndefinedVariable)
backend/driver.py|391 col 30| "add_to_training_queue" is not defined (reportUndefinedVariable)
backend/driver.py|395 col 20| Expression of type "tuple[str, Literal[400]]" cannot be assigned to return type "str"
  "tuple[str, Literal[400]]" is incompatible with "str" (reportGeneralTypeIssues)
backend/driver.py|398 col 20| Expression of type "tuple[str, Literal[200]]" cannot be assigned to return type "str"
  "tuple[str, Literal[200]]" is incompatible with "str" (reportGeneralTypeIssues)
backend/driver.py|402 col 16| Expression of type "tuple[str, Literal[400]]" cannot be assigned to return type "str"
  "tuple[str, Literal[400]]" is incompatible with "str" (reportGeneralTypeIssues)
backend/driver.py|408 col 19| "UserProgressDDBUtil" is not defined (reportUndefinedVariable)
backend/driver.py|408 col 39| "USERPROGRESS_TABLE_NAME" is not defined (reportUndefinedVariable)
backend/driver.py|408 col 64| "AWS_REGION" is not defined (reportUndefinedVariable)
backend/driver.py|409 col 26| "request" is not defined (reportUndefinedVariable)
backend/driver.py|414 col 21| "UserProgressData" is not defined (reportUndefinedVariable)
backend/driver.py|422 col 30| "request" is not defined (reportUndefinedVariable)
backend/driver.py|427 col 19| "UserProgressDDBUtil" is not defined (reportUndefinedVariable)
backend/driver.py|427 col 39| "USERPROGRESS_TABLE_NAME" is not defined (reportUndefinedVariable)
backend/driver.py|427 col 64| "AWS_REGION" is not defined (reportUndefinedVariable)
backend/driver.py|434 col 29| "POINTS_PER_QUESTION" is not defined (reportUndefinedVariable)
backend/driver.py|436 col 34| "POINTS_PER_QUESTION" is not defined (reportUndefinedVariable)
backend/driver.py|437 col 29| "POINTS_PER_QUESTION" is not defined (reportUndefinedVariable)
backend/driver.py|443 col 34| "POINTS_PER_QUESTION" is not defined (reportUndefinedVariable)
backend/driver.py|444 col 29| "POINTS_PER_QUESTION" is not defined (reportUndefinedVariable)
backend/driver.py|446 col 56| "POINTS_PER_QUESTION" is not defined (reportUndefinedVariable)
backend/driver.py|449 col 60| "POINTS_PER_QUESTION" is not defined (reportUndefinedVariable)
backend/driver.py|450 col 66| "POINTS_PER_QUESTION" is not defined (reportUndefinedVariable)
backend/driver.py|453 col 22| "POINTS_PER_QUESTION" is not defined (reportUndefinedVariable)
backend/driver.py|481 col 5| "createUserExecutionsData" is not defined (reportUndefinedVariable)
backend/driver.py|461 col 41| Function with declared type of "dict[Unknown, Unknown]" must return value on all code paths
  Type "None" cannot be assigned to type "dict[Unknown, Unknown]" (reportGeneralTypeIssues)
backend/middleware.py|36 col 49| Operator "in" not supported for types "str" and "Unknown | None"
  Operator "in" not supported for types "str" and "None" (reportGeneralTypeIssues)

github-actions · 2023-05-26T20:44:58Z

backend/driver.py

@@ -1,6 +1,11 @@
 from flask import Blueprint, Flask, send_from_directory
 from flask_cors import CORS
-
+from flasgger import Swagger


🚫 [pyright] _{reported by reviewdog 🐶}
Import "flasgger" could not be resolved (reportMissingImports)

github-actions · 2023-05-26T20:44:58Z

backend/driver.py

@@ -1,6 +1,11 @@
 from flask import Blueprint, Flask, send_from_directory
 from flask_cors import CORS
-
+from flasgger import Swagger
+from flasgger.utils import swag_from


🚫 [pyright] _{reported by reviewdog 🐶}
Import "flasgger.utils" could not be resolved (reportMissingImports)

github-actions · 2023-05-26T20:44:58Z

backend/driver.py

+@swag_from("openapi/tabular-run.openapi.yaml")
+def tabular_run():
+    try:
+        request_data = json.loads(request.data)


🚫 [pyright] _{reported by reviewdog 🐶}
"request" is not defined (reportUndefinedVariable)

github-actions · 2023-05-26T20:44:58Z

backend/driver.py

+            "batch_size": request_data["batch_size"],
+        }
+
+        createUserExecutionsData(


🚫 [pyright] _{reported by reviewdog 🐶}
"createUserExecutionsData" is not defined (reportUndefinedVariable)

github-actions · 2023-05-26T20:44:58Z

backend/driver.py

+        train_loss_results = dl_tabular_drive(
+            user_arch, fileURL, params, json_csv_data_str, customModelName
+        )
+        train_loss_results["user_arch"] = user_arch


🚫 [pyright] _{reported by reviewdog 🐶}
Object of type "None" is not subscriptable (reportOptionalSubscript)

github-actions · 2023-05-26T20:45:01Z

backend/driver.py

+        data = request_data["data"]
+        codeSnippet = request_data["codeSnippet"]
+        payload = json.dumps({"data": data, "code": codeSnippet})
+        resJson = invoke("preprocess_data", payload)


🚫 [pyright] _{reported by reviewdog 🐶}
"invoke" is not defined (reportUndefinedVariable)

github-actions · 2023-05-26T20:45:01Z

backend/driver.py

+            send_error(resJson["message"])
+        return resJson
+    except Exception:
+        print(traceback.format_exc())


🚫 [pyright] _{reported by reviewdog 🐶}
"traceback" is not defined (reportUndefinedVariable)

github-actions · 2023-05-26T20:45:01Z

backend/driver.py

+@swag_from("openapi/getSignedUploadUrl.openapi.yaml")
+def get_signed_upload_url():
+    try:
+        version = request.form.get("version")


🚫 [pyright] _{reported by reviewdog 🐶}
"request" is not defined (reportUndefinedVariable)

github-actions · 2023-05-26T20:45:01Z

backend/driver.py

+def get_signed_upload_url():
+    try:
+        version = request.form.get("version")
+        filename = request.form.get("filename")


🚫 [pyright] _{reported by reviewdog 🐶}
"request" is not defined (reportUndefinedVariable)

github-actions · 2023-05-26T20:45:01Z

backend/driver.py

+    try:
+        version = request.form.get("version")
+        filename = request.form.get("filename")
+        file = request.files.get("file")


🚫 [pyright] _{reported by reviewdog 🐶}
"request" is not defined (reportUndefinedVariable)

karkir0003 and others added 4 commits April 15, 2023 14:08

ECR GitHub Workflow to Push Image (#711)

9225ddd

* attempt at ecr put image: * relaxed ecr rules to push every time * skipped tests * new commands * mini docker * complete example * complete * example * reverted example * fixed Dockerfile

Create tmux script

701303b

Tmux script typo

1a2965f

noah-iversen requested review from karkir0003 and farisdurrani as code owners May 26, 2023 20:37

github-actions bot reviewed May 26, 2023

View reviewed changes

noah-iversen changed the base branch from dev to nextjs May 26, 2023 20:41

Merge branch 'nextjs' of https://github.com/DSGT-DLP/Deep-Learning-Pl…

354bdac

…ayground into tmux-script

github-actions bot reviewed May 26, 2023

View reviewed changes

noah-iversen closed this May 26, 2023

noah-iversen deleted the tmux-script branch May 26, 2023 20:55

noah-iversen restored the tmux-script branch May 26, 2023 20:55

farisdurrani deleted the tmux-script branch June 7, 2023 00:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tmux script #771

Tmux script #771

noah-iversen commented May 26, 2023

github-actions bot left a comment

github-actions bot May 26, 2023

github-actions bot May 26, 2023

github-actions bot May 26, 2023

github-actions bot May 26, 2023

github-actions bot May 26, 2023

github-actions bot May 26, 2023

github-actions bot May 26, 2023

github-actions bot May 26, 2023

github-actions bot May 26, 2023

github-actions bot May 26, 2023

karkir0003 commented May 26, 2023

github-actions bot left a comment

github-actions bot May 26, 2023

github-actions bot May 26, 2023

github-actions bot May 26, 2023

github-actions bot May 26, 2023

github-actions bot May 26, 2023

github-actions bot May 26, 2023

github-actions bot May 26, 2023

github-actions bot May 26, 2023

github-actions bot May 26, 2023

github-actions bot May 26, 2023



		@changevar(
		DataClass=FileUploadData, EnumClass=FileUploadEnums, partition_key=PRIMARY_KEY



		@enumclass(
		DataClass=FileUploadData,

Tmux script #771

Tmux script #771

Conversation

noah-iversen commented May 26, 2023

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot May 26, 2023

Choose a reason for hiding this comment

github-actions bot May 26, 2023

Choose a reason for hiding this comment

github-actions bot May 26, 2023

Choose a reason for hiding this comment

github-actions bot May 26, 2023

Choose a reason for hiding this comment

github-actions bot May 26, 2023

Choose a reason for hiding this comment

github-actions bot May 26, 2023

Choose a reason for hiding this comment

github-actions bot May 26, 2023

Choose a reason for hiding this comment

github-actions bot May 26, 2023

Choose a reason for hiding this comment

github-actions bot May 26, 2023

Choose a reason for hiding this comment

github-actions bot May 26, 2023

Choose a reason for hiding this comment

karkir0003 commented May 26, 2023

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot May 26, 2023

Choose a reason for hiding this comment

github-actions bot May 26, 2023

Choose a reason for hiding this comment

github-actions bot May 26, 2023

Choose a reason for hiding this comment

github-actions bot May 26, 2023

Choose a reason for hiding this comment

github-actions bot May 26, 2023

Choose a reason for hiding this comment

github-actions bot May 26, 2023

Choose a reason for hiding this comment

github-actions bot May 26, 2023

Choose a reason for hiding this comment

github-actions bot May 26, 2023

Choose a reason for hiding this comment

github-actions bot May 26, 2023

Choose a reason for hiding this comment

github-actions bot May 26, 2023

Choose a reason for hiding this comment