chore: Add API for storing dependencyMap in page object #33296

AnaghHegde · 2024-05-08T14:57:28Z

Description

The dependencyMap consumes considerable computation on the frontend. We would like to persist that in the backend for every page in an application.

For details on the signature and payload of the api calls please refer this document
Slack thread - https://theappsmith.slack.com/archives/C024GUDM0LT/p1714100372372069

Fixes #33030

Automation

/ok-to-test tags="@tag.Git"

🔍 Cypress test results

Tip

🟢 🟢 🟢 All cypress tests have passed! 🎉 🎉 🎉
Workflow run: https://github.com/appsmithorg/appsmith/actions/runs/9253166755
Commit: 03e9f01
Cypress dashboard url: Click here!

Communication

Should the DevRel and Marketing teams inform users about this change?

Yes
No

coderabbitai · 2024-05-08T14:59:11Z

Walkthrough

The recent changes introduce a new feature to persist the dependency map of a page in the backend. This involves adding a new method updateDependencyMap to various classes and interfaces, updating method signatures, and adding a new field dependencyMap to the PageDTO class. Additionally, tests have been added to validate the new functionality.

Changes

Files	Change Summary
`PageControllerCE.java`	Added imports for `List` and `Map`. Introduced `updateDependencyMap` method.
`NewPage.java`	Added `unpublishedPage_dependencyMap` field to the `Fields` class.
`PageDTO.java`	Added `dependencyMap` field to the `PageDTO` class.
`NewPageServiceCE.java`, `NewPageServiceCEImpl.java`	Added `updateDependencyMap` method. Updated `findBranchedPageId` method signature.
`CustomNewPageRepositoryCE.java`, `CustomNewPageRepositoryCEImpl.java`	Added `updateDependencyMap` and `findBranchedPageId` methods. Added imports for `Map`.
`NewPageServiceTest.java`	Added imports for `HashMap` and `Map`. Added tests for updating dependency maps.

Assessment against linked issues

Objective	Addressed	Explanation
Add dependencyMap in the response of Consolidated view and edit APIs (GET /api/v1/consolidated-api/view)	❓	The summary does not mention changes to consolidated APIs.
New API to update the dependency map of a page (PUT /pages/{pageId}/dependencyMap)	✅
Add dependency map in the response of get page API (edit mode) (GET /api/v1/pages/{pageId})	❓	The summary does not mention changes to get page API in edit mode.
Add dependency map in the response of get page API (view mode) (GET /api/v1/pages/{pageId}/view)	❓	The summary does not mention changes to get page API in view mode.
Publish API changes - Copy the dependency map of a page from unpublished to published	❓	The summary does not mention if the dependency map is copied during publish.

The changes partially address the objectives from the linked issue #33030. The new method for updating the dependency map has been implemented, but the summary does not confirm whether the dependency map is included in the responses of the consolidated and get page APIs, nor whether it's copied during publish.

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

Tips

Chat

There are 3 ways to chat with CodeRabbit:

Review comments: Directly reply to a review comment made by CodeRabbit. Example:
- I pushed a fix in commit <commit_id>.
- Generate unit testing code for this file.
- Open a follow-up GitHub issue for this discussion.
Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
- @coderabbitai generate unit testing code for this file.
- @coderabbitai modularize this function.
PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
- @coderabbitai generate interesting stats about this repository and render them as a table.
- @coderabbitai show all the console.log statements in this repository.
- @coderabbitai read src/utils.ts and generate unit testing code.
- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

@coderabbitai pause to pause the reviews on a PR.
@coderabbitai resume to resume the paused reviews.
@coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
@coderabbitai full review to full the review from scratch and review all the files again.
@coderabbitai summary to regenerate the summary of the PR.
@coderabbitai resolve resolve all the CodeRabbit review comments.
@coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (`.coderabbit.yaml`)

You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
Please see the configuration documentation for more information.
If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

Visit our Documentation for detailed information on how to use CodeRabbit.
Join our Discord Community to get help, request features, and share feedback.
Follow us on X/Twitter for updates and announcements.

AnaghHegde · 2024-05-08T15:01:02Z

/build-deploy-preview skip-tests=true

coderabbitai

Actionable comments posted: 9

...mith-server/src/main/java/com/appsmith/server/repositories/ce/CustomNewPageRepositoryCE.java

app/server/appsmith-server/src/main/java/com/appsmith/server/dtos/PageDTO.java

app/server/appsmith-server/src/main/java/com/appsmith/server/domains/NewPage.java

...server/appsmith-server/src/main/java/com/appsmith/server/newpages/base/NewPageServiceCE.java

...erver/appsmith-server/src/main/java/com/appsmith/server/controllers/ce/PageControllerCE.java

...-server/src/main/java/com/appsmith/server/repositories/ce/CustomNewPageRepositoryCEImpl.java

...er/appsmith-server/src/main/java/com/appsmith/server/newpages/base/NewPageServiceCEImpl.java

dvj1988 · 2024-05-10T09:29:22Z

/build-deploy-preview skip-tests=true

github-actions · 2024-05-10T09:29:43Z

Deploying Your Preview: https://github.com/appsmithorg/appsmith/actions/runs/9030237362.
Workflow: On demand build Docker image and deploy preview.
skip-tests: true.
env: ``.
PR: 33296.
recreate: .

github-actions · 2024-05-10T09:38:05Z

Deploy-Preview-URL: https://ce-33296.dp.appsmith.com

coderabbitai

Actionable comments posted: 1

app/server/appsmith-server/src/test/java/com/appsmith/server/services/NewPageServiceTest.java

AnaghHegde · 2024-05-13T06:22:46Z

/build-deploy-preview skip-tests=true

github-actions · 2024-05-13T06:23:07Z

Deploying Your Preview: https://github.com/appsmithorg/appsmith/actions/runs/9058221426.
Workflow: On demand build Docker image and deploy preview.
skip-tests: true.
env: ``.
PR: 33296.
recreate: .

github-actions · 2024-05-13T06:31:40Z

Deploy-Preview-URL: https://ce-33296.dp.appsmith.com

github-actions · 2024-05-13T06:46:26Z

Failed server tests

com.appsmith.server.solutions.CreateDBTablePageSolutionTests#createPageWithInvalidDatasourceTest

...-server/src/main/java/com/appsmith/server/repositories/ce/CustomNewPageRepositoryCEImpl.java

NilanshBansal · 2024-05-17T07:44:36Z

@AnaghHegde since you are adding a new field dependencyMap in the PageDTO, can you please do a sanity check for the following cases.

Is this field exported to the git repository for git-connected apps?
How will this work for import export? Can consider scenarios - an old JSON imported to the current version, what will be the value of these fields in the DB (I believe this should be null and should be okay but you can still check once).
On exporting the current version JSON, is the dependencyMap field exported?

cc: @rahulbarwal

…o feat/dependency-map-page

NilanshBansal · 2024-05-20T04:10:36Z

@AnaghHegde since you are adding a new field dependencyMap in the PageDTO, can you please do a sanity check for the following cases.

Is this field exported to the git repository for git-connected apps?

How will this work for import export? Can consider scenarios - an old JSON imported to the current version, what will be the value of these fields in the DB (I believe this should be null and should be okay but you can still check once).

On exporting the current version JSON, is the dependencyMap field exported?

cc: @rahulbarwal

@AnaghHegde @rahulbarwal can you please confirm if you have checked this?

mohanarpit · 2024-05-20T05:39:20Z

...erver/appsmith-server/src/main/java/com/appsmith/server/controllers/ce/PageControllerCE.java

@@ -196,4 +199,15 @@ public Mono<ResponseDTO<ApplicationPagesDTO>> getAllPages(
                .findApplicationPages(applicationId, pageId, branchName, mode)
                .map(resources -> new ResponseDTO<>(HttpStatus.OK.value(), resources, null));
    }
+
+    @JsonView(Views.Public.class)
+    @PutMapping("/{defaultPageId}/dependencyMap")


For my information, any reason we chose to do this as a separate endpoint instead of part of update layout API? Would having a separate endpoint cause a race condition between saving the updated page and saving the new dependency map?

This is a valid concern and we don't have a robust approach at the moment. We are reducing the scope to use the dependency map cache only in the view mode for now. When a user publishes the application, we will trigger the updateDependencyMap API followed by the publish API. The dependency map in the unpublished version will reflect the last published version, and it will be updated with each publish.
We will get back to edit mode dependency maps later when we have a better approach for invalidation of older dependency maps.
@AnaghHegde For the time being we can go ahead with this approach.
cc: @rajatagrawal

dvj1988 · 2024-05-22T05:11:36Z

/build-deploy-preview skip-tests=true

github-actions · 2024-05-22T05:11:57Z

Deploying Your Preview: https://github.com/appsmithorg/appsmith/actions/runs/9185693670.
Workflow: On demand build Docker image and deploy preview.
skip-tests: true.
env: ``.
PR: 33296.
recreate: .

github-actions · 2024-05-22T05:19:55Z

Deploy-Preview-URL: https://ce-33296.dp.appsmith.com

AnaghHegde · 2024-05-27T10:16:08Z

@NilanshBansal We are good from git and export point of view.

NilanshBansal

Considering @mohanarpit's comment here is addressed, this LGTM.

dvj1988 · 2024-05-27T11:27:25Z

/build-deploy-preview skip-tests=true

github-actions · 2024-05-27T11:27:48Z

Deploying Your Preview: https://github.com/appsmithorg/appsmith/actions/runs/9254039242.
Workflow: On demand build Docker image and deploy preview.
skip-tests: true.
env: ``.
PR: 33296.
recreate: .

github-actions · 2024-05-27T11:34:14Z

Deploy-Preview-URL: https://ce-33296.dp.appsmith.com

Add API for storing dependencyMap in page object

cf2269d

AnaghHegde self-assigned this May 8, 2024

github-actions bot added Task A simple Todo skip-changelog Adding this label to a PR prevents it from being listed in the changelog labels May 8, 2024

AnaghHegde added the ok-to-test Required label for CI label May 8, 2024

github-actions bot added Templates Templates Product Issues related to Templates labels May 8, 2024

coderabbitai bot reviewed May 8, 2024

View reviewed changes

AnaghHegde marked this pull request as draft May 10, 2024 10:00

add tests

bec496f

AnaghHegde added ok-to-test Required label for CI and removed ok-to-test Required label for CI labels May 10, 2024

AnaghHegde requested review from nidhi-nair and NilanshBansal May 10, 2024 11:23

AnaghHegde marked this pull request as ready for review May 10, 2024 11:23

coderabbitai bot reviewed May 10, 2024

View reviewed changes

app/server/appsmith-server/src/test/java/com/appsmith/server/services/NewPageServiceTest.java Outdated Show resolved Hide resolved

trishaanand added ok-to-test Required label for CI and removed ok-to-test Required label for CI labels May 10, 2024

make dependencyMap optional

98ee8a2

rahulbarwal marked this pull request as draft May 14, 2024 10:30

AnaghHegde marked this pull request as ready for review May 15, 2024 10:04

AnaghHegde added ok-to-test Required label for CI and removed ok-to-test Required label for CI labels May 15, 2024

NilanshBansal requested changes May 17, 2024

View reviewed changes

...-server/src/main/java/com/appsmith/server/repositories/ce/CustomNewPageRepositoryCEImpl.java Outdated Show resolved Hide resolved

use bridge query

6180f98

AnaghHegde requested a review from NilanshBansal May 17, 2024 10:19

Merge branch 'release' of https://github.com/appsmithorg/appsmith int…

ebd8bf5

…o feat/dependency-map-page

mohanarpit reviewed May 20, 2024

View reviewed changes

add sanitisation for git

03e9f01

AnaghHegde added ok-to-test Required label for CI and removed ok-to-test Required label for CI labels May 27, 2024

NilanshBansal approved these changes May 27, 2024

View reviewed changes

AnaghHegde merged commit 1086cdb into release May 31, 2024
43 checks passed

AnaghHegde deleted the feat/dependency-map-page branch May 31, 2024 09:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: Add API for storing dependencyMap in page object #33296

chore: Add API for storing dependencyMap in page object #33296

AnaghHegde commented May 8, 2024 •

edited by github-actions bot

Loading

coderabbitai bot commented May 8, 2024 •

edited

Loading

Chat

CodeRabbit Commands (invoked as PR comments)

CodeRabbit Configration File (`.coderabbit.yaml`)

Documentation and Community

AnaghHegde commented May 8, 2024

coderabbitai bot left a comment

dvj1988 commented May 10, 2024

github-actions bot commented May 10, 2024

github-actions bot commented May 10, 2024

coderabbitai bot left a comment

AnaghHegde commented May 13, 2024

github-actions bot commented May 13, 2024

github-actions bot commented May 13, 2024

github-actions bot commented May 13, 2024

NilanshBansal commented May 17, 2024

NilanshBansal commented May 20, 2024

mohanarpit May 20, 2024 •

edited

Loading

dvj1988 May 24, 2024

dvj1988 commented May 22, 2024

github-actions bot commented May 22, 2024

github-actions bot commented May 22, 2024

AnaghHegde commented May 27, 2024

NilanshBansal left a comment

dvj1988 commented May 27, 2024

github-actions bot commented May 27, 2024

github-actions bot commented May 27, 2024

chore: Add API for storing dependencyMap in page object #33296

chore: Add API for storing dependencyMap in page object #33296

Conversation

AnaghHegde commented May 8, 2024 • edited by github-actions bot Loading

Description

Automation

🔍 Cypress test results

Communication

coderabbitai bot commented May 8, 2024 • edited Loading

Walkthrough

Changes

Assessment against linked issues

Chat

CodeRabbit Commands (invoked as PR comments)

CodeRabbit Configration File (.coderabbit.yaml)

Documentation and Community

AnaghHegde commented May 8, 2024

coderabbitai bot left a comment

Choose a reason for hiding this comment

dvj1988 commented May 10, 2024

github-actions bot commented May 10, 2024

github-actions bot commented May 10, 2024

coderabbitai bot left a comment

Choose a reason for hiding this comment

AnaghHegde commented May 13, 2024

github-actions bot commented May 13, 2024

github-actions bot commented May 13, 2024

github-actions bot commented May 13, 2024

Failed server tests

NilanshBansal commented May 17, 2024

NilanshBansal commented May 20, 2024

mohanarpit May 20, 2024 • edited Loading

Choose a reason for hiding this comment

dvj1988 May 24, 2024

Choose a reason for hiding this comment

dvj1988 commented May 22, 2024

github-actions bot commented May 22, 2024

github-actions bot commented May 22, 2024

AnaghHegde commented May 27, 2024

NilanshBansal left a comment

Choose a reason for hiding this comment

dvj1988 commented May 27, 2024

github-actions bot commented May 27, 2024

github-actions bot commented May 27, 2024

AnaghHegde commented May 8, 2024 •

edited by github-actions bot

Loading

coderabbitai bot commented May 8, 2024 •

edited

Loading

CodeRabbit Configration File (`.coderabbit.yaml`)

mohanarpit May 20, 2024 •

edited

Loading