For dev builds the QtWebEngine directory should be inside the data directory. #1141

michaelsbradleyjr · 2020-10-07T19:12:14Z

The reason for doing this is that we don't really want dev builds to crossover in usage of caches, cookies, etc. that are created by a packaged build in the platform-specific path in the user's home directory, e.g. ~/Library/Application Support/Status on macOS.

We can detect that env var NIM_STATUS_CLIENT_DEV is not empty, as we do for other purposes, and when it's not empty the path-related setters documented here can be used to specify that the persistent storage and cache paths are inside the untracked data directory at the top-level of the repo.

The text was updated successfully, but these errors were encountered:

In the repo: ``` $ bin/nim_status_client --help ``` In the packaged app (macOS example): ``` $ cd /Applications/Status.app/Contents/MacOS $ ./nim_status_client --help ``` Output: ``` Usage: nim_status_client [OPTIONS]... The following options are available: -d, --dataDir Status Desktop data directory. ``` **Using the option** ``` $ cd ~/status-ci-builds/master/Status.app/Contents/MacOS $ ./nim_status_client --dataDir:"${HOME}/status-dirs/master" ``` In another terminal: ``` $ cd ~/status-ci-builds/PR-4242/Status.app/Contents/MacOS $ ./nim_status_client --dataDir:"${HOME}/status-dirs/PR-4242" ``` The path supplied can be relative or absolute, and can be specified with `--dataDir:[path]`, `--dataDir=[path]`, `-d:[path]`, or `-d=[path]`. Either `:` or `=` must be used, i.e. this *will not* work: `--dataDir [path]` or `-d [path]`. It is possible to run the same build in two different terminals by supplying two different `--dataDir`, i.e. this works: ``` $ cd /Applications/Status.app/Contents/MacOS $ ./nim_status_client --dataDir="${HOME}/temp/some1" ``` In another terminal: ``` $ cd /Applications/Status.app/Contents/MacOS $ ./nim_status_client --dataDir="${HOME}/temp/some2" ``` --- Command-line option support is provided by [nim-confutils](https://github.com/status-im/nim-confutils). The environment variable `NIM_STATUS_CLIENT_DEV` has been removed in favor of passing a "define" option to the Nim compiler: `-d:development` for development builds (e.g. `make V=1`) and `-d:production` for packaged builds (e.g. `make V=1 pkg`). Passing the correct option is handled automatically by the Makefile. A make variable named `RELEASE` has been introduced, which defaults to `false`. Presently the `RELEASE` variable should not be set on the command-line or in CI as more work needs to be done to toggle the proper compiler flags. Prior to the changes in this PR, symmetry is somewhat lacking between development and production (packaged) builds with respect to the concept of the "data directory". In development builds the root of the repo effectively serves as the `Status` directory used by production builds, e.g. on macOS `~/Library/Application Support/Status`. Also, there's a bit of confusion as to whether "data directory" refers to a directory for the desktop app's overall data (including status-go data) or to the specific directory used by status-go. This PR attempts to provide symmetry and reduce confusion: * The term "data directory" means the directory used by the desktop app to store multiple kinds of data and is not a reference to the subdirectory used by status-go. * For development builds the "data directory" defaults to `./Status/` relative to the root of the repo. * For production builds the "data directory" default is the same, e.g. on macOS it's ` ~/Library/Application Support/Status/`. The directory used by status-go is `Status/data/`. To be clear, that should be referred to as the "status-go directory" and not the app's "data directory". It would nice if we could rename it from `Status/data/` to `Status/status-go/`. We can do that, I already checked that it works correctly; however, for existing installations it would require that at app launch we check for the presence of `Status/data/` and rename it to `Status/status-go`. While simple enough to do, I was concerned that there might be some edge cases where the directory rename could cause a problem (e.g. if another copy of the app is running) so chose for now to stick with the status-go directory being `Status/data/`. --- **NOTES** More work needs to be done to ensure that all data written by the app is contained in the default or cli-specified data directory. Currently, both development and production (packaged) builds are writing to common directories outside of the data directory, e.g. located within `~/Library/` on macOS. Changing that behavior seems like it will mainly involve changing some defaults related to Qt components such as the web engine. See: #1141. In general, some additional refactoring could be done in the future. For example, implementing `StatusDesktopConfig` in `src/status/libstatus/accounts/constants.nim` (as done in this PR) works fine for now, but better code organization is desirable.

In the repo: ``` $ bin/nim_status_client --help ``` In the packaged app (macOS example): ``` $ cd /Applications/Status.app/Contents/MacOS $ ./nim_status_client --help ``` Output: ``` Usage: nim_status_client [OPTIONS]... The following options are available: -d, --dataDir Status Desktop data directory. ``` **Using the option** ``` $ cd ~/status-ci-builds/master/Status.app/Contents/MacOS $ ./nim_status_client --dataDir:"${HOME}/status-dirs/master" ``` In another terminal: ``` $ cd ~/status-ci-builds/PR-4242/Status.app/Contents/MacOS $ ./nim_status_client --dataDir:"${HOME}/status-dirs/PR-4242" ``` The path supplied can be relative or absolute, and can be specified with `--dataDir:[path]`, `--dataDir=[path]`, `-d:[path]`, or `-d=[path]`. Either `:` or `=` must be used, i.e. this *will not* work: `--dataDir [path]` or `-d [path]`. It is possible to run the same build in two different terminals by supplying two different `--dataDir`, i.e. this works: ``` $ cd /Applications/Status.app/Contents/MacOS $ ./nim_status_client --dataDir="${HOME}/temp/some1" ``` In another terminal: ``` $ cd /Applications/Status.app/Contents/MacOS $ ./nim_status_client --dataDir="${HOME}/temp/some2" ``` --- Command-line option support is provided by [nim-confutils](https://github.com/status-im/nim-confutils). The environment variable `NIM_STATUS_CLIENT_DEV` has been removed in favor of passing a "define" option to the Nim compiler: `-d:development` for development builds (e.g. `make V=1`) and `-d:production` for packaged builds (e.g. `make V=1 pkg`). Passing the correct option is handled automatically by the Makefile. A make variable named `RELEASE` has been introduced, which defaults to `false`. Presently the `RELEASE` variable should not be set on the command-line nor in CI as more work needs to be done to toggle the proper compiler flags. In the case of Status Desktop, "release vs. debug" is a concern orthogonal to "production vs. development". At present, production builds and development builds are all debug builds, but that will likely change in the future: we can have non-release CI production builds and local development builds be debug builds, while release builds in CI would be production builds with `RELEASE=true` (the compiled executable will be fully optimized). Prior to the changes in this PR, symmetry is somewhat lacking between development and production (packaged) builds with respect to the concept of the "data directory". In development builds the root of the repo effectively serves as the `Status` directory used by production builds, e.g. on macOS `~/Library/Application Support/Status`. Also, there's a bit of confusion as to whether "data directory" refers to a directory for the desktop app's overall data (including status-go data) or to the specific directory used by status-go. This PR attempts to provide symmetry and reduce confusion: * The term "data directory" means the directory used by the desktop app to store multiple kinds of data and is not a reference to the subdirectory used by status-go. * For development builds the "data directory" defaults to `./Status/` relative to the root of the repo. * For production builds the "data directory" default is the same as before, e.g. on macOS it's ` ~/Library/Application Support/Status/`. The directory used by status-go is `Status/data/`. To be clear, that should be referred to as the "status-go directory" and not the app's "data directory". It would nice if we could rename it from `Status/data/` to `Status/status-go/`. We can do that, I already checked that it works correctly; however, for existing installations it would require that at app launch we check for the presence of `Status/data/` and rename it to `Status/status-go`. While simple enough to do, I was concerned that there might be some edge cases where the directory rename could cause a problem (e.g. if another copy of the app is running) so chose for now to stick with the status-go directory being `Status/data/`. --- **NOTES** More work needs to be done to ensure that all data written by the app is contained in the default or cli-specified data directory. Currently, both development and production (packaged) builds are writing to common directories outside of the data directory, e.g. located within `~/Library/` on macOS. Changing that behavior seems like it will mainly involve changing some defaults related to Qt components such as the web engine. See: #1141. In general, some additional refactoring could be done in the future. For example, implementing `StatusDesktopConfig` in `src/status/libstatus/accounts/constants.nim` (as done in this PR) works fine for now, but better code organization is desirable. --- Closes #2268

In the repo: ``` $ bin/nim_status_client --help ``` In the packaged app (macOS example): ``` $ cd /Applications/Status.app/Contents/MacOS $ ./nim_status_client --help ``` Output: ``` Usage: nim_status_client [OPTIONS]... The following options are available: -d, --dataDir Status Desktop data directory. ``` **Using the option** ``` $ cd ~/status-ci-builds/master/Status.app/Contents/MacOS $ ./nim_status_client --dataDir:"${HOME}/status-dirs/master" ``` In another terminal: ``` $ cd ~/status-ci-builds/PR-4242/Status.app/Contents/MacOS $ ./nim_status_client --dataDir:"${HOME}/status-dirs/PR-4242" ``` The path supplied can be relative or absolute, and can be specified with `--dataDir:[path]`, `--dataDir=[path]`, `-d:[path]`, or `-d=[path]`. Either `:` or `=` must be used, i.e. this *will not* work: `--dataDir [path]` or `-d [path]`. It is possible to run the same build in two different terminals by supplying two different `--dataDir`, i.e. this works: ``` $ cd /Applications/Status.app/Contents/MacOS $ ./nim_status_client --dataDir="${HOME}/temp/some1" ``` In another terminal: ``` $ cd /Applications/Status.app/Contents/MacOS $ ./nim_status_client --dataDir="${HOME}/temp/some2" ``` --- Command-line option support is provided by [nim-confutils](https://github.com/status-im/nim-confutils). The environment variable `NIM_STATUS_CLIENT_DEV` has been removed in favor of passing a "define" option to the Nim compiler: `-d:development` for development builds (e.g. `make V=1`) and `-d:production` for packaged builds (e.g. `make V=1 pkg`). Passing the correct option is handled automatically by the Makefile. A make variable named `RELEASE` has been introduced, which defaults to `false`. Presently the `RELEASE` variable should not be set on the command-line nor in CI as more work needs to be done to toggle the proper compiler flags. In the case of Status Desktop, "release vs. debug" is a concern orthogonal to "production vs. development". At present, production builds and development builds are all debug builds, but that will likely change in the future: we can have non-release CI production builds and local development builds be debug builds, while release builds in CI would be production builds with `RELEASE=true` (the compiled executable will be fully optimized). Prior to the changes in this PR, symmetry is somewhat lacking between development and production (packaged) builds with respect to the concept of the "data directory". In development builds the root of the repo effectively serves as the `Status` directory used by production builds, e.g. on macOS `~/Library/Application Support/Status`. Also, there's a bit of confusion as to whether "data directory" refers to a directory for the desktop app's overall data (including status-go data) or to the specific directory used by status-go. This PR attempts to provide symmetry and reduce confusion: * The term "data directory" means the directory used by the desktop app to store multiple kinds of data and is not a reference to the subdirectory used by status-go. * For development builds the "data directory" defaults to `./Status/` relative to the root of the repo. * For production builds the "data directory" default is the same as before, e.g. on macOS it's ` ~/Library/Application Support/Status/`. The directory used by status-go is `Status/data/`. To be clear, that should be referred to as the "status-go directory" and not the app's "data directory". It would nice if we could rename it from `Status/data/` to `Status/status-go/`. We can do that, I already checked that it works correctly; however, for existing installations it would require that at app launch we check for the presence of `Status/data/` and rename it to `Status/status-go`. While simple enough to do, I was concerned that there might be edge cases where the directory rename could cause a problem (e.g. if another copy of the app is running) so chose for now to stick with the status-go directory being `Status/data/`. --- **NOTES** More work needs to be done to ensure that all data written by the app is contained in the default or cli-specified data directory. Currently, both development and production (packaged) builds are writing to common directories outside of the data directory, e.g. located within `~/Library/` on macOS. Changing that behavior seems like it will mainly involve changing defaults related to Qt components such as the web engine. See: #1141. In general, additional refactoring could be done in the future. For example, implementing `StatusDesktopConfig` in `src/status/libstatus/accounts/constants.nim` (as done in this PR) works fine for now, but better code organization is desirable. --- Closes #2268

In the repo: ``` $ bin/nim_status_client --help ``` In the packaged app (macOS example): ``` $ cd /Applications/Status.app/Contents/MacOS $ ./nim_status_client --help ``` Output: ``` Usage: nim_status_client [OPTIONS]... The following options are available: -d, --dataDir Status Desktop data directory. ``` **Using the option** ``` $ cd ~/status-ci-builds/master/Status.app/Contents/MacOS $ ./nim_status_client --dataDir:"${HOME}/status-dirs/master" ``` In another terminal: ``` $ cd ~/status-ci-builds/PR-4242/Status.app/Contents/MacOS $ ./nim_status_client --dataDir:"${HOME}/status-dirs/PR-4242" ``` The path supplied can be relative or absolute, and can be specified with `--dataDir:[path]`, `--dataDir=[path]`, `-d:[path]`, or `-d=[path]`. Either `:` or `=` must be used, i.e. this *will not* work: `--dataDir [path]` or `-d [path]`. The name of the option follows Nim's partial case-insensitivity rules, so `--dataDir`, `--datadir`, and `--data_dir` are all equivalent. See [Identifier equality][ieq] in the Nim Manual. It is possible to run the same build in multiple terminals by supplying different `--dataDir`, i.e. this works: ``` $ cd /Applications/Status.app/Contents/MacOS $ ./nim_status_client --dataDir="${HOME}/temp/some1" ``` In another terminal: ``` $ cd /Applications/Status.app/Contents/MacOS $ ./nim_status_client --dataDir="${HOME}/temp/some2" ``` **Windows** It is recommended to use a Git Bash or MSYS2 terminal when invoking `bin/nim_status_client.exe` (development build) or `bin/Status.exe` (production build) on the command-line. The reason is that if the exe is invoked in a session of `cmd.exe` it will return to the prompt immediately; the app will run but there will be no output in the terminal. In any case, the `--dataDir` option will take effect whether the exe is invoked in `cmd.exe` or a recommended terminal. For development builds, when invoking `bin/nim_status_client.exe` directly instead of via `make run`, because e.g. you wish to use the `--dataDir` option, it is required to first setup the `PATH` environment variable correctly. See the `run-windows` target in this repo's Makefile for more information. **Linux** The `--dataDir` option may be passed to command-line invocation of a production (AppImage) build in the same way as passing it to a development build: ``` $ Status.AppImage --dataDir:/path/to/wherever ``` For development builds, when invoking `bin/nim_status_client` directly instead of via `make run`, because e.g. you wish to use the `--dataDir` option, it is required to setup the `LD_LIBRARY_PATH` environment variable correctly. See the `run-linux` target in this repo's Makefile for more information. --- BREAKING CHANGE: The `qt` subdir of the app's data directory is now a sibling of the status-go directory rather than a subdir of the status-go directory: ``` Status (app data directory) ├── data (status-go directory) ├── qt └── tmp ``` Because app settings are stored in the `qt` directory that means that existing installations will lose their customized settings. At app startup, it would be possible to detect `Status/data/qt` and if `Status/qt` doesn't exist yet then copy `Status/data/qt` to `Status/qt`. However, there was some concern that behavior could lead to problems later on if we forget the workaround is in place. So for now that settings preservation strategy has not been implemented, but it might be before this commit is merged pending full team awareness/consensus. --- Command-line option support is provided by [nim-confutils](https://github.com/status-im/nim-confutils). The environment variable `NIM_STATUS_CLIENT_DEV` has been removed in favor of passing a "define" option to the Nim compiler: `-d:development` for development builds (e.g. `make V=1`) and `-d:production` for packaged builds (e.g. `make V=1 pkg`). Passing the correct option is handled automatically by the Makefile. A make variable named `RELEASE` has been introduced, which defaults to `false`. Presently the `RELEASE` variable should not be set on the command-line nor in CI as more work needs to be done to toggle the proper compiler flags. In the case of Status Desktop, "release vs. debug" is a concern orthogonal to "production vs. development". At present, production builds and development builds are all debug builds, but that will likely change in the future: we can have non-release CI production builds and local development builds be debug builds, while release builds in CI would be production builds with `RELEASE=true` (the compiled executable will be fully optimized). Prior to the changes in this PR, symmetry is somewhat lacking between development and production (packaged) builds with respect to the concept of the "data directory". In development builds the root of the repo effectively serves as the `Status` directory used by production builds, e.g. on macOS `~/Library/Application Support/Status`. Also, there's a bit of confusion as to whether "data directory" refers to a directory for the desktop app's overall data (including status-go data) or to the specific directory used by status-go. This PR attempts to provide symmetry and reduce confusion: * The term "data directory" means the directory used by the desktop app to store multiple kinds of data and is not a reference to the subdirectory used by status-go. * For development builds the "data directory" defaults to `./Status/` relative to the root of the repo. * For production builds the "data directory" default is the same as before, e.g. on macOS it's ` ~/Library/Application Support/Status/`. The directory used by status-go is `Status/data/`. To be clear, that should be referred to as the "status-go directory" and not the app's "data directory". It would nice if we could rename it from `Status/data/` to `Status/status-go/`. We can do that, I already checked that it works correctly; however, for existing installations it would require that at app launch we check for the presence of `Status/data/` and rename it to `Status/status-go`. While simple enough to do, I was concerned that there might be edge cases where the directory rename could cause a problem (e.g. if another copy of the app is running) so chose for now to stick with the status-go directory being `Status/data/`. --- **NOTES** More work needs to be done to ensure that all data written by the app is contained in the default or cli-specified data directory. Currently, both development and production (packaged) builds are writing to common directories outside of the data directory, e.g. located within `~/Library/` on macOS. Changing that behavior seems like it will mainly involve changing defaults related to Qt components such as the web engine. See: #1141. In general, additional refactoring could be done in the future. For example, implementing `StatusDesktopConfig` in `src/status/libstatus/accounts/constants.nim` (as done in this PR) works fine for now, but better code organization is desirable. --- Closes #2268 [ieq]: https://nim-lang.org/docs/manual.html#lexical-analysis-identifier-equality

iurimatias · 2022-05-15T22:48:28Z

closed in favour of #5747

michaelsbradleyjr added the Browser label Oct 7, 2020

richard-ramos added the enhancement label Oct 21, 2020

michaelsbradleyjr mentioned this issue Apr 20, 2021

feat: command-line option can be used to specify app's data directory #2316

Merged

iurimatias added chore idea labels Jul 5, 2021

iurimatias added the E:Browser label Mar 7, 2022

iurimatias closed this as completed May 15, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

For dev builds the QtWebEngine directory should be inside the data directory. #1141

For dev builds the QtWebEngine directory should be inside the data directory. #1141

michaelsbradleyjr commented Oct 7, 2020 •

edited

Loading

iurimatias commented May 15, 2022

For dev builds the QtWebEngine directory should be inside the data directory. #1141

For dev builds the QtWebEngine directory should be inside the data directory. #1141

Comments

michaelsbradleyjr commented Oct 7, 2020 • edited Loading

iurimatias commented May 15, 2022

michaelsbradleyjr commented Oct 7, 2020 •

edited

Loading