Separate user code into client and server directories #753

sodic · 2022-10-14T00:45:03Z

Separate client code and server code.

Fixes #522
Fixes #594

Takes a step towards implementing #710.

@Martinsos and @shayneczyzewski, feel free to start reviewing this. I've not yet updated e2e tests, example projects, or the docs. I'll do this once we settle on a structure. The current approach is:

ProjectRoot
├── client
│   └── .waspignore (and client source files)
├── .gitignore
├── main.wasp
├── .env.server
├── .env.client
├── server
│   └── .waspignore (and server source files)
└── .wasproot

I'd probabaly like to move the env files inside client and server

sodic · 2022-10-14T00:47:28Z

waspc/cli/src/Wasp/Cli/Command/CreateNewProject.hs

@@ -6,18 +6,18 @@ where
 import Control.Monad.Except (throwError)


I introduced a lot of changes to this file. The best way to review it is probably to avoid looking at the diff altogether.

The main (non-refactor) changes are:

We now create two separate directories for user code: client and server

Since most of the project scaffolding doesn't depend on anything, we now create a new project mainly by copying a skeleton from our templates and dynamically generate files only when necessary (i.e., for main.wasp file which depends on ProjectInfo).

sodic · 2022-10-14T00:50:04Z

waspc/src/Wasp/Lib.hs

+  maybeDotEnvServerFile <- findDotEnvServer waspDir
+  maybeDotEnvClientFile <- findDotEnvClient waspDir


@shayneczyzewski Do you think it makes sense to move .env files into server and client directories? It might be a bit cleaner.

Yeah, good thinking- I think it would probably make sense for them to be renamed .env inside each respective directory.

Sounds good to me also!

We'll do this in a different issue, I'll make one before merging the PR.

Did this issue get created BTW?

Not yet, I usually go through all unresolved threads and create issues while waiting for the pre-merge CLI to do its thing.

Done: #811.

waspc/src/Wasp/AppSpec.hs

waspc/data/Cli/templates/new/.wasproot

waspc/src/Wasp/CompileOptions.hs

shayneczyzewski

Hey @sodic I like the direction here, nice job! I wanted to take this on in a few passes, so this was my first. :)

At a high level, I think the direction makes sense. A few things I noted:

~~We may want to keep the .env warning around.~~
You are right, that having those two very similar identifiers for compile options and AppSpec is a bit of a code smell for some reason. I assume it would feel worse when we add a "shared" folder. We could try to put them into some additional structure, or we could punt as a TODO and try to solve it when we add "shared". I will think more on this one.
I will do more testing once the example app is updated. Initial new project issues were noted in a comment.

Can't wait to take a second pass, hopefully with some ideas on how to address that similar variable issue you noted. Will think on it and try a few things out. I'll also think about it in the context of multiple wasp files/dirs (so instead of having an explicit client and server, maybe we have a dynamic collection of files/dirs and they are just labeled client/server, for example).

waspc/src/Wasp/Lib.hs

shayneczyzewski · 2022-10-17T15:44:00Z

waspc/src/Wasp/Lib.hs

+  maybeDotEnvServerFile <- findDotEnvServer waspDir
+  maybeDotEnvClientFile <- findDotEnvClient waspDir


Yeah, good thinking- I think it would probably make sense for them to be renamed .env inside each respective directory.

waspc/data/Cli/templates/new/.wasproot

waspc/cli/src/Wasp/Cli/Command/CreateNewProject.hs

Co-authored-by: Shayne Czyzewski <523636+shayneczyzewski@users.noreply.github.com>

Martinsos · 2022-10-19T09:36:42Z

Hey @sodic I like the direction here, nice job! I wanted to take this on in a few passes, so this was my first. :)

At a high level, I think the direction makes sense. A few things I noted:

~~We may want to keep the .env warning around.~~

You are right, that having those two very similar identifiers for compile options and AppSpec is a bit of a code smell for some reason. I assume it would feel worse when we add a "shared" folder. We could try to put them into some additional structure, or we could punt as a TODO and try to solve it when we add "shared". I will think more on this one.

I will do more testing once the example app is updated. Initial new project issues were noted in a comment.

Can't wait to take a second pass, hopefully with some ideas on how to address that similar variable issue you noted. Will think on it and try a few things out. I'll also think about it in the context of multiple wasp files/dirs (so instead of having an explicit client and server, maybe we have a dynamic collection of files/dirs and they are just labeled client/server, for example).

Love that thinking direction, and in generally thinking about this! Dream of modular Wasp is closer and closer :D!

Martinsos

Looking good @sodic ! Left some comments, nothing major.

The big question, I believe, is: what about shared code, that both client and server use?

Additional question: should we organize it all under some kind of src/ dir? I am not sure why I am even asking this, but it feels like it might make sense, I guess I have just seen it on so many places. But then, I can't find a good enough reason for it. I believe src/ is normally used to separate source code from other stuff like some very top lvl config, docs, readmes, ... .
But I guess they can just put all of this stuff under src/ if they wish. How would it look if we added src? I guess client/ and server/ would go under it, and .wasp would remain next to them. But .wasp is also source file! But it is more of a config though. Maybe .wasp also goes into src? Hm. Ok, just a bit of wild thinking, let me know what you think @shayneczyzewski @sodic .

waspc/cli/src/Wasp/Cli/Command/CreateNewProject.hs

waspc/src/Wasp/CompileOptions.hs

waspc/src/Wasp/Lib.hs

Martinsos · 2022-10-21T15:31:29Z

@sodic checked your responses, sounds good, so ping me once you do the changes and are ready for final review!

Martinsos

@sodic checked comments, looking good!
The only non-trivial comment is the one about the name of analyzeWaspProject function.

Btw, what about questions I asked here?
#753 (review)
We discussed "src/" a bit, I think we concluded it might make sense t have it.
What about shared/?
Are those changes coming in this PR, I guess they should?

waspc/cli/src/Wasp/Cli/Command/CreateNewProject.hs

waspc/src/Wasp/Lib.hs

waspc/src/Wasp/Util.hs

…structure

sodic · 2022-11-04T11:34:08Z

Nice catch @shayneczyzewski, I fixed in the next commit. Btw, according to this comment, that globbing quirk should be fixed in next versions of cabal.

sodic · 2022-11-04T16:07:43Z

waspc/src/Wasp/Analyzer/Evaluator/Evaluation/TypedExpr/Combinators.hs

+    mkError ctx msg = Left $ ER.mkEvaluationError ctx $ ER.ParseError $ ER.EvaluationParseError msg
+    stripImportPrefix importPath = stripPrefix serverPrefix importPath <|> stripPrefix clientPrefix importPath
+    serverPrefix = "@server/"
+    clientPrefix = "@client/"


@Martinsos I believe this is the temporary solution we've settled on:

Wasp requires users to import from @client and @server, but works regardless of which one they pick 😅

I'll resolve this comment after I make an issue for it.

sodic · 2022-11-04T16:10:03Z

waspc/src/Wasp/Generator/JsImport.hs

@@ -10,7 +10,7 @@ import Wasp.Generator.ExternalCodeGenerator.Common (GeneratedExternalCodeDir)

 getJsImportDetailsForExtFnImport ::
  -- | Path to generated external code directory, relative to the directory in which file doing the importing is.
-  Path Posix (Rel (Dir a)) (Dir GeneratedExternalCodeDir) ->
+  Path Posix (Rel a) (Dir GeneratedExternalCodeDir) ->


This was an error, there's no need to nest Dirs.

sodic · 2022-11-04T16:10:32Z

waspc/src/Wasp/Generator/ServerGenerator.hs

+extServerCodeDirInServerSrcDirP :: Path Posix (Rel C.ServerSrcDir) (Dir GeneratedExternalCodeDir)
+extServerCodeDirInServerSrcDirP = fromJust $ relDirToPosix extServerCodeDirInServerSrcDir


@Martinsos Is this safe? I do it in several other places.

It will give runtime error if that value is Nothing (if its conversion from rel dir to posix failed). Since we are dealing with our paths here, and not user input, I think it is ok.

I thought so, thanks for confirming!

…structure

waspc/test/Generator/WebAppGeneratorTest.hs

waspc/test/AppSpec/ValidTest.hs

…structure

shayneczyzewski

This is awesome @sodic, amazing work! I love how much nicer the dir structure feels with src and having things split into client/server/shared (and importing with @client etc 🔥). Awesome 👏🏻 And the ExceptT usage to clean a few things up was great.

Just a few small questions/comments, but non are blockers and feel free to ignore them honestly. I think this looks ready.

One thing I would do as part of some PR is update the ChangeLog, however, since this will be a major breaking change worth noting.

waspc/cli/src/Wasp/Cli/Command/CreateNewProject.hs

shayneczyzewski · 2022-11-09T14:31:18Z

waspc/cli/src/Wasp/Cli/Command/CreateNewProject.hs

+  liftIO printGettingStartedInstructions
+  where
+    printGettingStartedInstructions = do
+      putStrLn $ Term.applyStyles [Term.Green] ("Created new Wasp app in ./" ++ projectName ++ " directory!")


I think it is fine to use projectName because the contract of parseProjectInfo means it will fail if invalid, but since we use projectInfo to create, it may also be nice to pull the project name off that struct so everything downstream uses the "validated" thing. But I don't think it warrants a change, just thinking aloud. Only reason it could matter in future is if we did any normalization for them or whatnot.

Yeah, I know what you mean. I'll change it.

shayneczyzewski · 2022-11-09T14:44:51Z

waspc/src/Wasp/Analyzer/Evaluator/Evaluation/TypedExpr/Combinators.hs

+    mkParseError ctx msg = Left $ ER.mkEvaluationError ctx $ ER.ParseError $ ER.EvaluationParseError msg
+    stripImportPrefix importPath = stripPrefix serverPrefix importPath <|> stripPrefix clientPrefix importPath
+    serverPrefix = "@server/"
+    clientPrefix = "@client/"


Will we ever want to allow them to import something from @shared?

We have no such use case at the moment. All declarations have a designated runtime (server or client). I documented this restriction here.

VS Code will help them out with importing JS into Wasp after we split the ExtImport type into ServerImport and ClientImport (and this code will become cleaner too). In the meantime, we're "faking" different types and pretend they exist in the docs.

Sounds good 👍🏻 I will be starting a review of the docs in a bit so it will be fresh in my head :D

shayneczyzewski · 2022-11-09T14:48:42Z

waspc/examples/todoApp/src/server/serverSetup.js

@@ -1,10 +1,12 @@
 import { mySpecialJob } from '@wasp/jobs/mySpecialJob.js'
+import { sayHi } from '../shared/util.js'


This is sweet!

Haha, thanks, but we already pretty much had the same thing with a smaller infrastructure 😄

shayneczyzewski · 2022-11-09T15:10:23Z

waspc/src/Wasp/Lib.hs

+  maybeDotEnvServerFile <- findDotEnvServer waspDir
+  maybeDotEnvClientFile <- findDotEnvClient waspDir


Did this issue get created BTW?

sodic added 3 commits October 12, 2022 15:28

Separate ext code to client and server

4e09b34

Use skeleton in createNewProject and refactor

86e70fe

Refactor Lib.hs to use ExceptT

bb4107e

sodic commented Oct 14, 2022

View reviewed changes

waspc/src/Wasp/AppSpec.hs Show resolved Hide resolved

sodic commented Oct 14, 2022

View reviewed changes

waspc/data/Cli/templates/new/.wasproot Outdated Show resolved Hide resolved

sodic commented Oct 14, 2022

View reviewed changes

waspc/src/Wasp/CompileOptions.hs Show resolved Hide resolved

sodic added 2 commits October 14, 2022 03:03

Fix formatting

cc7f353

Pop up returns

492f247

sodic requested review from Martinsos and shayneczyzewski October 14, 2022 01:26

sodic self-assigned this Oct 14, 2022

shayneczyzewski reviewed Oct 17, 2022

View reviewed changes

waspc/cli/src/Wasp/Cli/Command/CreateNewProject.hs Outdated Show resolved Hide resolved

Extract liftIO and add a do block

d88e204

Co-authored-by: Shayne Czyzewski <523636+shayneczyzewski@users.noreply.github.com>

Martinsos reviewed Oct 19, 2022

View reviewed changes

sodic added 3 commits October 24, 2022 23:36

Address some review comments

51b426a

Add skeleton comment

789dc8f

Extract common CommandError message

6d29601

sodic force-pushed the filip-change-project-structure branch from f6ec33b to 6d29601 Compare October 24, 2022 22:01

sodic added 2 commits October 25, 2022 00:03

Separate skeleton comment into two rows

11846e7

Move server and client dirs into src

4f834ec

Martinsos reviewed Oct 25, 2022

View reviewed changes

waspc/cli/src/Wasp/Cli/Command/CreateNewProject.hs Outdated Show resolved Hide resolved

waspc/src/Wasp/Lib.hs Outdated Show resolved Hide resolved

waspc/src/Wasp/Util.hs Outdated Show resolved Hide resolved

sodic added 4 commits October 25, 2022 15:21

Simplify maybeToEither

385b35c

Further refactor Lib.hs

a7753d2

Further simplify skeleton comment

53dc889

Merge branch 'main' into filip-change-project-structure

8388b0f

sodic force-pushed the filip-change-project-structure branch from 4ef31f8 to 8674887 Compare November 4, 2022 11:18

Merge remote-tracking branch 'origin/main' into filip-change-project-…

997fe72

…structure

Add a shared function demo to todoApp

0cb6369

sodic force-pushed the filip-change-project-structure branch from c0c2480 to 0cb6369 Compare November 4, 2022 11:39

sodic added 2 commits November 4, 2022 15:17

Update waspc and e2e tests

8bb94cc

Fix compiler warnings and rename function

5f47855

sodic commented Nov 4, 2022

View reviewed changes

Rename mkError to mkParserError

54e0b02

sodic commented Nov 4, 2022

View reviewed changes

sodic added 2 commits November 4, 2022 17:14

Remove redundant empty line

0ae2631

Merge remote-tracking branch 'origin/main' into filip-change-project-…

c5c326e

…structure

sodic commented Nov 4, 2022

View reviewed changes

waspc/test/Generator/WebAppGeneratorTest.hs Outdated Show resolved Hide resolved

Fix test warnings

fc76fb6

sodic commented Nov 4, 2022

View reviewed changes

waspc/test/AppSpec/ValidTest.hs Outdated Show resolved Hide resolved

sodic and others added 4 commits November 4, 2022 18:33

Fix formatting

dc29140

Merge remote-tracking branch 'origin/main' into filip-change-project-…

36e5766

…structure

Merge branch 'main' into filip-change-project-structure

cd101ab

Merge branch 'main' into filip-change-project-structure

bed02f1

sodic mentioned this pull request Nov 9, 2022

Update docs to account for recent changes #808

Merged

Fix directory tree watching on wasp start

f384d14

sodic mentioned this pull request Nov 9, 2022

Update example apps to account for recent changes #809

Merged

shayneczyzewski approved these changes Nov 9, 2022

View reviewed changes

Implement review feedback

d4a4c07

This was referenced Nov 10, 2022

Implement different types for SeverImport and ClientImport #810

Open

Move environment variable files to src/server and src/client #811

Closed

sodic merged commit 86e0893 into main Nov 11, 2022

sodic deleted the filip-change-project-structure branch November 11, 2022 16:21

sodic mentioned this pull request Dec 27, 2023

Implement the project install step #1602

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Separate user code into client and server directories #753

Separate user code into client and server directories #753

sodic commented Oct 14, 2022 •

edited

sodic Oct 14, 2022 •

edited

sodic Oct 14, 2022

shayneczyzewski Oct 17, 2022

Martinsos Oct 19, 2022

sodic Oct 24, 2022

shayneczyzewski Nov 9, 2022

sodic Nov 9, 2022

sodic Nov 11, 2022

shayneczyzewski left a comment •

edited

shayneczyzewski Oct 17, 2022

Martinsos commented Oct 19, 2022

Martinsos left a comment

Martinsos commented Oct 21, 2022 •

edited

Martinsos left a comment

sodic commented Nov 4, 2022

sodic Nov 4, 2022

sodic Nov 10, 2022

sodic Nov 4, 2022

sodic Nov 4, 2022

Martinsos Nov 7, 2022

sodic Nov 8, 2022

shayneczyzewski left a comment

shayneczyzewski Nov 9, 2022

sodic Nov 9, 2022

shayneczyzewski Nov 9, 2022

sodic Nov 9, 2022

shayneczyzewski Nov 9, 2022

shayneczyzewski Nov 9, 2022

sodic Nov 9, 2022

shayneczyzewski Nov 9, 2022

		@@ -6,18 +6,18 @@ where
		import Control.Monad.Except (throwError)

		maybeDotEnvServerFile <- findDotEnvServer waspDir
		maybeDotEnvClientFile <- findDotEnvClient waspDir

		extServerCodeDirInServerSrcDirP :: Path Posix (Rel C.ServerSrcDir) (Dir GeneratedExternalCodeDir)
		extServerCodeDirInServerSrcDirP = fromJust $ relDirToPosix extServerCodeDirInServerSrcDir

		@@ -1,10 +1,12 @@
		import { mySpecialJob } from '@wasp/jobs/mySpecialJob.js'
		import { sayHi } from '../shared/util.js'

Separate user code into client and server directories #753

Separate user code into client and server directories #753

Conversation

sodic commented Oct 14, 2022 • edited

sodic Oct 14, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shayneczyzewski left a comment • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Martinsos commented Oct 19, 2022

Martinsos left a comment

Choose a reason for hiding this comment

Martinsos commented Oct 21, 2022 • edited

Martinsos left a comment

Choose a reason for hiding this comment

sodic commented Nov 4, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shayneczyzewski left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sodic commented Oct 14, 2022 •

edited

sodic Oct 14, 2022 •

edited

shayneczyzewski left a comment •

edited

Martinsos commented Oct 21, 2022 •

edited