Skip to content

Commit

Permalink
Releasing: 1.8.6 (#16134)
Browse files Browse the repository at this point in the history
* chlog continued

* [App] PoC: Add support for Request (#16047)

(cherry picked from commit 592b126)

* Have checkgroup pull the latest runs (#16033)

(cherry picked from commit 4c91dec)

* Update Multinode Warning (#16091)

(cherry picked from commit 3d509f6)

* [App] Serve datatypes with better client code (#16018)

(cherry picked from commit 23013be)

* docs: add PT version (#16010)

* docs: add PT version

* stable

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
(cherry picked from commit c8b4bce)

* add 1.13.1 to adjust versions (#16099)

(cherry picked from commit acd48d3)

* [App] Add display name property to the work (#16095)

Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
(cherry picked from commit 22b254f)

* Fix detection of whether app is running in cloud (#16045)

(cherry picked from commit e54f4f5)

* [App] Add work.delete (#16103)

Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
(cherry picked from commit 80b7faf)

* [App] Improve the autoscaler UI (#16063)

[App] Improve the autoscaler UI (#16063)

(cherry picked from commit 39d27f6)

* [App] Min replica=0 would break autoscaler component (#16092)

* fixing the bug where num_replica=0 would fail

* changelog

(cherry picked from commit aba5f12)

* [App] Scale out/in interval for autoscaler (#16093)

* Adding arguments for scale out/in interval

* Tests

(cherry picked from commit 0fd3d54)

* Set the default work start method to spawn on MacOS (#16089)

(cherry picked from commit f3157f3)

* [App] Add status endpoint, enable `ready` (#16075)

Co-authored-by: thomas chaton <thomas@grid.ai>
(cherry picked from commit 2a85d9b)

* Clarify `work.stop()` limitation (#16073)

(cherry picked from commit 0abd686)

* Update torchvision requirement from <=0.14.0,>=0.11.1 to >=0.11.1,<0.15.0 in /requirements (#16108)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>

(cherry picked from commit e8ac9e9)

* CI: settle file names (#16098)

* CI: settle file names

* rename

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

(cherry picked from commit a203b04)

* [App] Resolve some bugs from the Training Studio scaling (#16114)

Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
(cherry picked from commit 51ec949)

* Update scikit-learn requirement from <1.1.3,>0.22.1 to >0.22.1,<1.2.1 in /requirements (#16107)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
(cherry picked from commit d070e72)

* [App] update app testing (#16000)

Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
(cherry picked from commit 8b68e20)

* [App] Change overwrite to True (#16009)

Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
(cherry picked from commit 085c6b3)

* [App] Improve PythonServer info message on startup (#15989)

* change msg
* update chgl
* show the user's class name

(cherry picked from commit 848ea56)

* [App] Cold start proxy in autoscaler (#16094)

* cold start proxy

* Update src/lightning_app/components/serve/auto_scaler.py

* changelog

* better-doc

Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>
Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>

(cherry picked from commit 7cbdc68)

* Fix endpoint information tab not showing up in AutoScaler UI (#16128)

* .

* why

* Revert "why"

This reverts commit 375d3e8.

* tried api access with fixed values

* Revert "tried api access with fixed values"

This reverts commit f1720f6.

* Fix typo 🎉

* update chglog

* revert removing lines in chlog

* update chglog

Co-authored-by: Akihiro Nitta <akihiro@lightning.ai>
(cherry picked from commit 7b275bc)

* [App] Implement `ready` for components (#16129)

(cherry picked from commit 711aec5)

* [App] Fix e2e tests (#16146)

(cherry picked from commit c98b0c4)

* [App] Remove outdated warning from cloud requirements (#16140)

(cherry picked from commit bf5fa35)

* [App] Fix support for streamlit > 1.14 (#16139)

(cherry picked from commit ec336bc)

* Fix broken link in "Build a Model" section of docs (#16025)

fix build model link

(cherry picked from commit 1c02288)

* drop colossalai from testing as no stable release yet (#16122)

(cherry picked from commit 1e16836)

* [App] Simplify messaging in cloud dispatch (#16160)

(cherry picked from commit 9657671)

* [App] Add annotations endpoint (#16159)

(cherry picked from commit 0630444)

* Small fix in test_cli.py to avoid failure with future version of jsonargparse (#16156)

(cherry picked from commit 50331e0)

* Fix test failing on master due to bad auto-merge (#16118)

(cherry picked from commit 07b775f)

* ver1.8.6

Co-authored-by: Sherin Thomas <sherin@lightning.ai>
Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>
Co-authored-by: Ethan Harris <ethanwharris@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Mauricio Villegas <mauricio_ville@yahoo.com>
  • Loading branch information
10 people committed Dec 21, 2022
1 parent a8a3519 commit bffdc2f
Show file tree
Hide file tree
Showing 87 changed files with 1,427 additions and 427 deletions.
8 changes: 4 additions & 4 deletions .github/checkgroup.yml
Expand Up @@ -6,7 +6,7 @@ subprojects:
- id: "pytorch_lightning: Tests workflow"
paths:
- ".actions/**"
- ".github/workflows/ci-pytorch-tests.yml"
- ".github/workflows/ci-tests-pytorch.yml"
- "requirements/lite/**"
- "src/lightning_lite/**"
- "requirements/pytorch/**"
Expand Down Expand Up @@ -178,7 +178,7 @@ subprojects:
- "src/lightning_lite/**"
- "tests/tests_lite/**"
- "setup.cfg" # includes pytest config
- ".github/workflows/ci-lite-tests.yml"
- ".github/workflows/ci-tests-lite.yml"
- "!requirements/*/docs.txt"
- "!*.md"
- "!**/*.md"
Expand Down Expand Up @@ -221,7 +221,7 @@ subprojects:
- id: "lightning_app: Tests workflow"
paths:
- ".actions/**"
- ".github/workflows/ci-app-tests.yml"
- ".github/workflows/ci-tests-app.yml"
- "src/lightning_app/**"
- "tests/tests_app/**"
- "requirements/app/**"
Expand All @@ -243,7 +243,7 @@ subprojects:
- id: "lightning_app: Examples"
paths:
- ".actions/**"
- ".github/workflows/ci-app-examples.yml"
- ".github/workflows/ci-examples-app.yml"
- "src/lightning_app/**"
- "tests/tests_examples_app/**"
- "examples/app_*/**"
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/README.md
Expand Up @@ -4,10 +4,10 @@

## Unit and Integration Testing

| workflow name | workflow file | action | accelerator\* |
| -------------------------- | ------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------- |
| Test PyTorch full | .github/workflows/ci-pytorch-tests.yml | Run all tests except for accelerator-specific, standalone and slow tests. | CPU |
| Test PyTorch slow | .github/workflows/ci-pytorch-tests-slow.yml | Run only slow tests. Slow tests usually need to spawn threads and cannot be speed up or simplified. | CPU |
| workflow name | workflow file | action | accelerator\* |
| ----------------- | -------------------------------------- | ------------------------------------------------------------------------- | ------------- |
| Test PyTorch full | .github/workflows/ci-tests-pytorch.yml | Run all tests except for accelerator-specific, standalone and slow tests. | CPU |

| pytorch-lightning (IPUs) | .azure-pipelines/ipu-tests.yml | Run only IPU-specific tests. | IPU |
| pytorch-lightning (HPUs) | .azure-pipelines/hpu-tests.yml | Run only HPU-specific tests. | HPU |
| pytorch-lightning (GPUs) | .azure-pipelines/gpu-tests-pytorch.yml | Run all CPU and GPU-specific tests, standalone, and examples. Each standalone test needs to be run in separate processes to avoid unwanted interactions between test cases. | GPU |
Expand Down
File renamed without changes.
Expand Up @@ -9,7 +9,7 @@ on:
types: [opened, reopened, ready_for_review, synchronize] # added `ready_for_review` since draft is skipped
paths:
- ".actions/**"
- ".github/workflows/ci-app-examples.yml"
- ".github/workflows/ci-examples-app.yml"
- "src/lightning_app/**"
- "tests/tests_examples_app/**"
- "examples/app_*/**"
Expand Down
Expand Up @@ -9,7 +9,7 @@ on:
types: [opened, reopened, ready_for_review, synchronize] # added `ready_for_review` since draft is skipped
paths:
- ".actions/**"
- ".github/workflows/ci-app-tests.yml"
- ".github/workflows/ci-tests-app.yml"
- "src/lightning_app/**"
- "tests/tests_app/**"
- "requirements/app/**"
Expand Down
Expand Up @@ -13,7 +13,7 @@ on:
- "src/lightning_lite/**"
- "tests/tests_lite/**"
- "setup.cfg" # includes pytest config
- ".github/workflows/ci-lite-tests.yml"
- ".github/workflows/ci-tests-lite.yml"
- "!requirements/*/docs.txt"
- "!*.md"
- "!**/*.md"
Expand Down
Expand Up @@ -14,7 +14,7 @@ on:
- "tests/tests_pytorch/**"
- "tests/legacy/back-compatible-versions.txt"
- "setup.cfg" # includes pytest config
- ".github/workflows/ci-pytorch-tests.yml"
- ".github/workflows/ci-tests-pytorch.yml"
- "requirements/lite/**"
- "src/lightning_lite/**"
- "!requirements/pytorch/docs.txt"
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/probot-check-group.yml
Expand Up @@ -14,7 +14,7 @@ jobs:
if: github.event.pull_request.draft == false
timeout-minutes: 61 # in case something is wrong with the internal timeout
steps:
- uses: Lightning-AI/probot@v5.1
- uses: Lightning-AI/probot@v5.3
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
Expand Down
4 changes: 2 additions & 2 deletions .gitignore
Expand Up @@ -109,8 +109,8 @@ celerybeat-schedule

# dotenv
.env
.env_staging
.env_local
.env.staging
.env.local

# virtualenv
.venv
Expand Down
18 changes: 9 additions & 9 deletions README.md
Expand Up @@ -90,15 +90,15 @@ Lightning is rigorously tested across multiple CPUs, GPUs, TPUs, IPUs, and HPUs

<center>

| System / PyTorch ver. | 1.9 | 1.10 | 1.12 (latest) |
| :------------------------: | :-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: | :-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: |
| Linux py3.7 \[GPUs\*\*\] | - | - | - |
| Linux py3.7 \[TPUs\*\*\*\] | [![Test](https://github.com/Lightning-AI/lightning/actions/workflows/tpu-tests.yml/badge.svg?branch=master&event=push)](https://github.com/Lightning-AI/lightning/actions/workflows/tpu-tests.yml) | - | - |
| Linux py3.8 \[IPUs\] | [![Build Status](<https://dev.azure.com/Lightning-AI/lightning/_apis/build/status/pytorch-lightning%20(IPUs)?branchName=master>)](https://dev.azure.com/Lightning-AI/lightning/_build/latest?definitionId=25&branchName=master) | - | - |
| Linux py3.8 \[HPUs\] | - | [![Build Status](<https://dev.azure.com/Lightning-AI/lightning/_apis/build/status/pytorch-lightning%20(HPUs)?branchName=master>)](https://dev.azure.com/Lightning-AI/lightning/_build/latest?definitionId=26&branchName=master) | - |
| Linux py3.{7,9} | - | - | [![Test](https://github.com/Lightning-AI/lightning/actions/workflows/ci-pytorch-tests.yml/badge.svg?branch=master&event=push)](https://github.com/Lightning-AI/lightning/actions/workflows/ci-pytorch-tests.yml) |
| OSX py3.{7,9} | - | - | [![Test](https://github.com/Lightning-AI/lightning/actions/workflows/ci-pytorch-tests.yml/badge.svg?branch=master&event=push)](https://github.com/Lightning-AI/lightning/actions/workflows/ci-pytorch-tests.yml) |
| Windows py3.{7,9} | - | - | [![Test](https://github.com/Lightning-AI/lightning/actions/workflows/ci-pytorch-tests.yml/badge.svg?branch=master&event=push)](https://github.com/Lightning-AI/lightning/actions/workflows/ci-pytorch-tests.yml) |
| System / PyTorch ver. | 1.10 | 1.12 |
| :------------------------: | :-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: |
| Linux py3.7 \[GPUs\*\*\] | - | - |
| Linux py3.7 \[TPUs\*\*\*\] | - | - |
| Linux py3.8 \[IPUs\] | - | - |
| Linux py3.8 \[HPUs\] | [![Build Status](<https://dev.azure.com/Lightning-AI/lightning/_apis/build/status/pytorch-lightning%20(HPUs)?branchName=master>)](https://dev.azure.com/Lightning-AI/lightning/_build/latest?definitionId=26&branchName=master) | - |
| Linux py3.{7,9} | - | [![Test](https://github.com/Lightning-AI/lightning/actions/workflows/ci-tests-pytorch.yml/badge.svg?branch=master&event=push)](https://github.com/Lightning-AI/lightning/actions/workflows/ci-tests-pytorch.yml) |
| OSX py3.{7,9} | - | [![Test](https://github.com/Lightning-AI/lightning/actions/workflows/ci-tests-pytorch.yml/badge.svg?branch=master&event=push)](https://github.com/Lightning-AI/lightning/actions/workflows/ci-tests-pytorch.yml) |
| Windows py3.{7,9} | - | [![Test](https://github.com/Lightning-AI/lightning/actions/workflows/ci-tests-pytorch.yml/badge.svg?branch=master&event=push)](https://github.com/Lightning-AI/lightning/actions/workflows/ci-tests-pytorch.yml) |

- _\*\* tests run on two NVIDIA P100_
- _\*\*\* tests run on Google GKE TPUv2/3. TPU py3.7 means we support Colab and Kaggle env._
Expand Down
3 changes: 2 additions & 1 deletion docs/source-app/api_references.rst
Expand Up @@ -45,7 +45,8 @@ ___________________
~multi_node.lite.LiteMultiNode
~multi_node.pytorch_spawn.PyTorchSpawnMultiNode
~multi_node.trainer.LightningTrainerMultiNode
~auto_scaler.AutoScaler
~serve.auto_scaler.AutoScaler
~serve.auto_scaler.ColdStartProxy

----

Expand Down
2 changes: 2 additions & 0 deletions docs/source-pytorch/index.rst
Expand Up @@ -64,6 +64,8 @@ Conda users

Or read the `advanced install guide <starter/installation.html>`_

We are fully compatible with any stable PyTorch version v1.10 and above.

.. raw:: html

<hr class="docutils" style="margin: 50px 0 50px 0">
Expand Down
2 changes: 1 addition & 1 deletion docs/source-pytorch/model/build_model.rst
Expand Up @@ -23,7 +23,7 @@ Build a Model
:header: 2: Validate and test a model
:description: Add a validation and test data split to avoid overfitting.
:col_css: col-md-4
:button_link: validate_model_basic.html
:button_link: ../common/evaluation_basic.html
:height: 150
:tag: basic

Expand Down
4 changes: 4 additions & 0 deletions examples/app_boring/app.py
Expand Up @@ -43,6 +43,10 @@ def __init__(self):
raise_exception=True,
)

@property
def ready(self) -> bool:
return self.dest_work.is_running

def run(self):
self.source_work.run()
if self.source_work.has_succeeded:
Expand Down
1 change: 1 addition & 0 deletions examples/app_display_name/.lightningignore
@@ -0,0 +1 @@
venv/
25 changes: 25 additions & 0 deletions examples/app_display_name/app.py
@@ -0,0 +1,25 @@
import lightning as L


class Work(L.LightningWork):
def __init__(self, start_with_flow=True):
super().__init__(start_with_flow=start_with_flow)

def run(self):
pass


class Flow(L.LightningFlow):
def __init__(self):
super().__init__()
self.w = Work()
self.w1 = Work(start_with_flow=False)
self.w.display_name = "My Custom Name" # Not supported yet
self.w1.display_name = "My Custom Name 1"

def run(self):
self.w.run()
self.w1.run()


app = L.LightningApp(Flow())
17 changes: 7 additions & 10 deletions examples/app_server_with_auto_scaler/app.py
@@ -1,5 +1,5 @@
# ! pip install torch torchvision
from typing import Any, List
from typing import List

import torch
import torchvision
Expand All @@ -8,16 +8,12 @@
import lightning as L


class RequestModel(BaseModel):
image: str # bytecode


class BatchRequestModel(BaseModel):
inputs: List[RequestModel]
inputs: List[L.app.components.Image]


class BatchResponse(BaseModel):
outputs: List[Any]
outputs: List[L.app.components.Number]


class PyTorchServer(L.app.components.PythonServer):
Expand Down Expand Up @@ -79,10 +75,11 @@ def scale(self, replicas: int, metrics: dict) -> int:
# autoscaler specific args
min_replicas=1,
max_replicas=4,
autoscale_interval=10,
scale_out_interval=10,
scale_in_interval=10,
endpoint="predict",
input_type=RequestModel,
output_type=Any,
input_type=L.app.components.Image,
output_type=L.app.components.Number,
timeout_batching=1,
max_batch_size=8,
)
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Expand Up @@ -79,8 +79,8 @@ module = [
"lightning_app.components.serve.types.image",
"lightning_app.components.serve.types.type",
"lightning_app.components.serve.python_server",
"lightning_app.components.serve.auto_scaler",
"lightning_app.components.training",
"lightning_app.components.auto_scaler",
"lightning_app.core.api",
"lightning_app.core.app",
"lightning_app.core.flow",
Expand Down
2 changes: 0 additions & 2 deletions requirements/app/cloud.txt
@@ -1,5 +1,3 @@
# WARNING: this file is not used directly by the backend
# any dependency here needs to be shipped with the base image
redis>=4.0.1, <=4.2.4
docker>=5.0.0, <6.0.2
s3fs>=2022.5.0, <2022.8.3
Expand Down
1 change: 1 addition & 0 deletions requirements/app/test.txt
Expand Up @@ -4,6 +4,7 @@ pytest==7.2.0
pytest-timeout==2.1.0
pytest-cov==4.0.0
pytest-doctestplus>=0.9.0
pytest-asyncio==0.20.3
playwright==1.28.0
httpx
trio<0.22.0
Expand Down
2 changes: 1 addition & 1 deletion requirements/lite/base.txt
Expand Up @@ -2,7 +2,7 @@
# in case you want to preserve/enforce restrictions on the latest compatible version, add "strict" as an in-line comment

numpy>=1.17.2, <1.23.1
torch>=1.9.0, <=1.13.0
torch>=1.9.0, <=1.13.1
fsspec[http]>2021.06.0, <2022.6.0
packaging>=17.0, <=21.3
typing-extensions>=4.0.0, <=4.4.0
Expand Down
2 changes: 1 addition & 1 deletion requirements/lite/examples.txt
@@ -1,4 +1,4 @@
# NOTE: the upper bound for the package version is only set for CI stability, and it is dropped while installing this package
# in case you want to preserve/enforce restrictions on the latest compatible version, add "strict" as an in-line comment

torchvision>=0.10.0, <=0.13.0
torchvision>=0.10.0, <=0.14.1

0 comments on commit bffdc2f

Please sign in to comment.