Skip to content

Commit

Permalink
enable support for javascript-based search filters, closes #257
Browse files Browse the repository at this point in the history
* filter function typed in TypeScript
* Monaco Editor used as a code editor
  • Loading branch information
vladimiry committed Mar 21, 2021
1 parent 0dd8a28 commit aaf0235
Show file tree
Hide file tree
Showing 34 changed files with 961 additions and 347 deletions.
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,8 @@ Some Linux package types are available for installing from the repositories (`Pa
- :octocat: **Open Source**.
- :gear: **Reproducible builds**. See details in [#183](https://github.com/vladimiry/ElectronMail/issues/183).
- :gear: **Cross platform**. The app works on Linux/OSX/Windows platforms. Binary installation packages located [here](https://github.com/vladimiry/ElectronMail/releases).
- :mag_right: **Full-text search**. Including email body content scanning capability. Enabled with [v2.2.0](https://github.com/vladimiry/ElectronMail/releases/tag/v2.2.0) release. See the respective [issue](https://github.com/vladimiry/ElectronMail/issues/92) for details.
- :mag_right: **Full-text search**. Including email **body content** scanning capability. Enabled with [v2.2.0](https://github.com/vladimiry/ElectronMail/releases/tag/v2.2.0) release. See the respective [issue](https://github.com/vladimiry/ElectronMail/issues/92) for details.
- :mag_right: **JavaScript-based/unlimited messages filtering**. Enabled since [v4.11.0](https://github.com/vladimiry/ElectronMail/releases/tag/v4.11.0) release. See the respective [#257](https://github.com/vladimiry/ElectronMail/issues/257) for details. Requires [local store](https://github.com/vladimiry/ElectronMail/wiki/FAQ) feature to be enabled.
- :package: **Offline access to the email messages** (attachments content not stored locally, but emails body content). The [local store](https://github.com/vladimiry/ElectronMail/wiki/FAQ) feature enables storing your messages in the encrypted `database.bin` file (see [FAQ](https://github.com/vladimiry/ElectronMail/wiki/FAQ) for file purpose details). So the app allows you to view your messages offline, running full-text search against them, exporting them to EML files. etc. Enabled since [v2.0.0](https://github.com/vladimiry/ElectronMail/releases/tag/v2.0.0) release.
- :mailbox: **Multi accounts** support including supporting individual [API entry points](https://github.com/vladimiry/ElectronMail/issues/29). For example, you can force the specific email account added in the app connect to the email provider via the [Tor](https://www.torproject.org/) only by selecting the `https://protonirockerxow.onion/` API entry point in the dropdown list and configuring a proxy as described in [this](https://github.com/vladimiry/ElectronMail/issues/113#issuecomment-529130116) message.
- :unlock: **Automatic login into the app** with a remembered the system keychain remembered master ([keep me signed in](images/keep-me-signed-in.png) feature). Integration with as a system keychain is done with the [keytar](https://github.com/atom/node-keytar) module. By the way, on Linux [KeePassXC](https://github.com/keepassxreboot/keepassxc) implements the [Secret Service](https://specifications.freedesktop.org/secret-service/latest/) interface and so it can be acting as a system keychain (for details, see the "automatic login into the app"-related point in the [FAQ](https://github.com/vladimiry/ElectronMail/wiki/FAQ)).
Expand All @@ -47,7 +48,7 @@ Some Linux package types are available for installing from the repositories (`Pa
- :bell: **Native notifications** for individual accounts clicking on which focuses the app window and selects respective account in the accounts list.
- :calendar: **Calendar notifications / alarms** regardless of the open page (mail/calendar/settings/account/drive). The opt-in feature has been enabled since [v4.9.0](https://github.com/vladimiry/ElectronMail/releases/tag/v4.9.0). See [#229](https://github.com/vladimiry/ElectronMail/issues/229) for details.
- :sunglasses: **Making all email "read"** in a single mouse click. Enabled since [v3.8.0](https://github.com/vladimiry/ElectronMail/releases/tag/v3.8.0). Requires [local store](https://github.com/vladimiry/ElectronMail/wiki/FAQ) feature to be enabled.
- :sunglasses: **Routing images through proxy**. The opt-in feature has been enabled since [v4.9.0](https://github.com/vladimiry/ElectronMail/releases/tag/v4.9.0). See [#312](https://github.com/vladimiry/ElectronMail/issues/312) for details. Requires [local store](https://github.com/vladimiry/ElectronMail/wiki/FAQ) feature to be enabled.
- :sunglasses: **Routing images through proxy**. The opt-in feature has been enabled since [v4.9.0](https://github.com/vladimiry/ElectronMail/releases/tag/v4.9.0). See [#312](https://github.com/vladimiry/ElectronMail/issues/312) for details.
- :sunglasses: **Batch mails removing** bypassing the trash. Enabled since [v4.9.0](https://github.com/vladimiry/ElectronMail/releases/tag/v4.9.0). Requires [local store](https://github.com/vladimiry/ElectronMail/wiki/FAQ) feature to be enabled.
- :sunglasses: **Batch mails moving between folders**. Enabled since [v4.5.0](https://github.com/vladimiry/ElectronMail/releases/tag/v4.5.0). Requires [local store](https://github.com/vladimiry/ElectronMail/wiki/FAQ) feature to be enabled.

Expand Down
14 changes: 9 additions & 5 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "electron-mail",
"description": "Unofficial ProtonMail Desktop App",
"version": "4.10.3",
"version": "4.11.0",
"author": "Vladimir Yakovlev <desktop-app@protonmail.ch>",
"license": "MIT",
"homepage": "https://github.com/vladimiry/ElectronMail",
Expand Down Expand Up @@ -67,9 +67,9 @@
"assets:dev": "npm-run-all assets:copy:dev assets:webclient:dev",
"assets:copy": "cpx \"./src/assets/dist/**/*\" ./app/assets",
"assets:copy:dev": "cpx \"./src/assets/dist/**/*\" ./app-dev/assets",
"assets:webclient": "yarn assets:webclient:base ./app/webclient",
"assets:webclient:dev": "yarn assets:webclient:base ./app-dev/webclient",
"assets:webclient:base": "yarn ts-node:shortcut ./scripts/prepare-webclient/protonmail.ts",
"assets:webclient": "yarn assets:webclient:base ./app",
"assets:webclient:dev": "yarn assets:webclient:base ./app-dev",
"assets:webclient:base": "yarn ts-node:shortcut ./scripts/prepare-webclient/index.ts",
"electron-builder:install-app-deps": "electron-builder install-app-deps --arch=x64",
"electron-builder:dir": "electron-builder --x64 --dir",
"electron-builder:dist": "npm exec --package=ts-node -- ts-node --files --require tsconfig-paths/register ./scripts/electron-builder/run-with-default-evn-vars.ts --x64 --publish never",
Expand Down Expand Up @@ -101,7 +101,7 @@
"scripts/download-tray-icon-font": "yarn ts-node:shortcut ./scripts/download-tray-icon-font.ts",
"scripts/transfer": "yarn ts-node:shortcut ./scripts/transfer/index.ts",
"ts-node:shortcut": "cross-env TS_NODE_FILES=true ts-node --require tsconfig-paths/register",
"webpack:shortcut": "cross-env TS_NODE_FILES=true npm exec --package=webpack-cli --node-options=\"--require tsconfig-paths/register\" -- webpack"
"webpack:shortcut": "cross-env TS_NODE_PROJECT=./webpack-configs/tsconfig.json TS_NODE_FILES=true npm exec --package=webpack-cli --node-options=\"--require tsconfig-paths/register\" -- webpack"
},
"ava": {
"extensions": [
Expand Down Expand Up @@ -145,6 +145,7 @@
"proxy-agent": "4.0.1",
"pure-uuid": "1.6.2",
"pureimage": "0.2.7",
"quickjs-emscripten": "0.11.0",
"reflect-metadata": "0.1.13",
"remeda": "0.0.27",
"rxjs": "6.6.6",
Expand Down Expand Up @@ -212,6 +213,7 @@
"cpx2": "3.0.0",
"cross-env": "7.0.3",
"css-loader": "5.1.2",
"dts-generator": "3.0.0",
"electron": "12.0.1",
"electron-builder": "22.10.5",
"escape-string-regexp": "4.0.0",
Expand All @@ -227,6 +229,7 @@
"immer": "8.0.1",
"import-sort-cli": "6.0.0",
"import-sort-parser-typescript": "6.0.0",
"imports-loader": "2.0.0",
"jasmine": "3.6.4",
"jsdom": "16.5.1",
"karma": "6.2.0",
Expand All @@ -235,6 +238,7 @@
"karma-webpack": "5.0.0",
"lint-staged": "10.5.4",
"mini-css-extract-plugin": "1.3.9",
"monaco-editor": "0.23.0",
"ndx": "1.0.2",
"ndx-query": "1.0.1",
"ngx-bootstrap": "6.2.0",
Expand Down
13 changes: 13 additions & 0 deletions scripts/lib.ts
Original file line number Diff line number Diff line change
Expand Up @@ -223,3 +223,16 @@ export const catchTopLeventAsync = (asyncFn: () => Promise<unknown>): void => {
process.exit(1);
});
};

export const applyPatch = async ({patchFile, cwd}: { patchFile: string; cwd: string }): Promise<void> => {
await execShell([
"git",
[
"apply",
"--ignore-whitespace",
"--reject",
patchFile,
],
{cwd},
]);
};
22 changes: 22 additions & 0 deletions scripts/prepare-webclient/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
import path from "path";
import pathIsInside from "path-is-inside";

import {CWD_ABSOLUTE_DIR} from "scripts/const";
import {buildProtonClients} from "scripts/prepare-webclient/protonmail";
import {catchTopLeventAsync} from "scripts/lib";
import {resolveProtonMetadata} from "scripts/prepare-webclient/monaco-editor-dts";

const [, , appDestDir] = process.argv;

if (!appDestDir) {
throw new Error("Empty base destination directory argument");
}

if (!pathIsInside(path.resolve(CWD_ABSOLUTE_DIR, appDestDir), CWD_ABSOLUTE_DIR)) {
throw new Error(`Invalid base destination directory argument value: ${appDestDir}`);
}

catchTopLeventAsync(async () => {
await buildProtonClients({destDir: path.join(appDestDir, "./webclient")});
await resolveProtonMetadata({destDir: path.join(appDestDir)});
});
81 changes: 81 additions & 0 deletions scripts/prepare-webclient/monaco-editor-dts.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
import fs from "fs";
import fsExtra from "fs-extra";
import path from "path";

import {CONSOLE_LOG} from "scripts/lib";
import {IpcMainServiceScan} from "src/shared/api/main";
import {PROTON_MONACO_EDITOR_DTS_ASSETS_LOCATION} from "src/shared/constants";
import {PROTON_SHARED_MESSAGE_INTERFACE} from "src/shared/proton-apps-constants";

// TODO "require/var-requires"-based import
export const dtsGenerator: { // eslint-disable-line @typescript-eslint/no-unsafe-assignment
default: (options: { baseDir: string, files: string[], out: string }) => Promise<string>
} = require("dts-generator"); // eslint-disable-line @typescript-eslint/no-var-requires

export const resolveProtonMetadata = async (
{destDir}: { destDir: string },
): Promise<IpcMainServiceScan["ApiImplReturns"]["staticInit"]["monacoEditorExtraLibArgs"]> => {
const options = {
system: {
base: "./node_modules/typescript/lib",
in: "./node_modules/typescript/lib/lib.esnext.d.ts",
out: path.join(destDir, PROTON_MONACO_EDITOR_DTS_ASSETS_LOCATION.system),
},
protonMessage: {
base: "./output/git/proton-mail/node_modules/proton-shared",
in: path.join("./output/git/proton-mail/node_modules/proton-shared", PROTON_SHARED_MESSAGE_INTERFACE.projectRelativeFile),
out: path.join(destDir, PROTON_MONACO_EDITOR_DTS_ASSETS_LOCATION.protonMessage),
},
} as const;

// TODO replace "dts-generator" dependency with something capable to combine "./node_modules/typescript/lib/lib.esnext.d.ts"
for (const key of [/* "system", */ "protonMessage"] as const) {
const sourceFile = options[key].in;
if (!fsExtra.pathExistsSync(sourceFile)) {
throw new Error(`The source "${sourceFile}" file doesn't exits.`);
}
await dtsGenerator.default({baseDir: options[key].base, files: [sourceFile], out: options[key].out});
CONSOLE_LOG(`Merged "${options[key].in}" to "${options[key].out}"`);
}

// TODO drop custom "./node_modules/typescript/lib/lib.esnext.d.ts" combining when "dts-generator" gets replaced
{
const referenceTagRe = /\/\/\/[\s\t]+<reference[\s\t]+lib=["']+(.*)["']+[\s\t]+\/>/;
const mergedFiles: string[] = [];
const extractContent = (file: string): string => {
const lines = fs.readFileSync(file).toString().split("\n");
const resultLines: string[] = [`// === file: ${file}`];
for (const line of lines) {
const match = referenceTagRe.exec(line);
const libName = match && match[1];
if (libName) {
resultLines.push(
extractContent(path.join(path.dirname(file), `lib.${libName}.d.ts`)),
);
} else {
resultLines.push(line);
}
}
mergedFiles.push(file);
return resultLines.join("\n");
};
fsExtra.ensureDirSync(path.dirname(options.system.out));
fs.writeFileSync(options.system.out, extractContent(options.system.in));
CONSOLE_LOG(`Merged to "${options.system.out}" files:`, mergedFiles);
}

return {
system: [
fs.readFileSync(options.system.out).toString(),
`in-memory:${options.system.in}`,
],
protonMessage: [
(
fs.readFileSync(options.protonMessage.out).toString()
+
`declare const mail: Omit<import("lib/interfaces/mail/Message").Message, "Body"> & {Body: string};`
),
`in-memory:${PROTON_SHARED_MESSAGE_INTERFACE.url}`,
],
};
};
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@
import fs from "fs";
import fsExtra from "fs-extra";
import path from "path";
import pathIsInside from "path-is-inside";

import {CONSOLE_LOG, execShell, resolveGitCommitInfo, resolveGitOutputBackupDir} from "scripts/lib";
import {CWD_ABSOLUTE_DIR, GIT_CLONE_ABSOLUTE_DIR} from "scripts/const";
import {GIT_CLONE_ABSOLUTE_DIR} from "scripts/const";
import {PROVIDER_REPO_MAP} from "src/shared/proton-apps-constants";
import {RUNTIME_ENV_CI_PROTON_CLIENTS_ONLY, WEB_CLIENTS_BLANK_HTML_FILE_NAME} from "src/shared/constants";

Expand All @@ -26,16 +25,6 @@ const reposOnlyFilter: DeepReadonly<{ value: Array<keyof typeof PROVIDER_REPO_MA
return {value: result, envVariableName};
})();

const [, , BASE_DEST_DIR] = process.argv;

if (!BASE_DEST_DIR) {
throw new Error("Empty base destination directory argument");
}

if (!pathIsInside(path.resolve(CWD_ABSOLUTE_DIR, BASE_DEST_DIR), CWD_ABSOLUTE_DIR)) {
throw new Error(`Invalid base destination directory argument value: ${BASE_DEST_DIR}`);
}

export interface FolderAsDomainEntry<T extends any = any> { // eslint-disable-line @typescript-eslint/no-explicit-any
folderNameAsDomain: string;
options: T;
Expand Down Expand Up @@ -75,18 +64,20 @@ export async function executeBuildFlow<T extends FolderAsDomainEntry[], O = Unpa
repoType,
folderAsDomainEntries,
repoRelativeDistDir = PROVIDER_REPO_MAP[repoType].repoRelativeDistDir,
destDir,
destSubFolder,
flow,
}: {
repoType: keyof typeof PROVIDER_REPO_MAP;
folderAsDomainEntries: T;
repoRelativeDistDir?: string;
destSubFolder: string;
repoType: keyof typeof PROVIDER_REPO_MAP
folderAsDomainEntries: T
repoRelativeDistDir?: string
destDir: string
destSubFolder: string
flow: {
postClone?: Flow<O>;
install?: Flow<O>;
build: Flow<O>;
};
postClone?: Flow<O>
install?: Flow<O>
build: Flow<O>
}
},
): Promise<void> {
if (
Expand All @@ -101,7 +92,7 @@ export async function executeBuildFlow<T extends FolderAsDomainEntry[], O = Unpa
const repoDir = path.join(GIT_CLONE_ABSOLUTE_DIR, repoType);

for (const folderAsDomainEntry of folderAsDomainEntries) {
const targetDistDir = path.resolve(BASE_DEST_DIR as string, folderAsDomainEntry.folderNameAsDomain, destSubFolder);
const targetDistDir = path.resolve(destDir, folderAsDomainEntry.folderNameAsDomain, destSubFolder);

CONSOLE_LOG(
`Prepare web client build [${repoType}]:`,
Expand Down Expand Up @@ -157,19 +148,29 @@ export async function executeBuildFlow<T extends FolderAsDomainEntry[], O = Unpa

const repoDistBackupDir = resolveGitOutputBackupDir({repoType, suffix: `dist-${folderAsDomainEntry.folderNameAsDomain}`});

if (fsExtra.pathExistsSync(repoDistBackupDir)) { // taking dist from the backup
const src = repoDistBackupDir;
const dest = repoDistDir;
CONSOLE_LOG(`Copying backup ${src} to ${dest}`);
await fsExtra.copy(src, dest);
} else { // executing the build
const installModules = async (): Promise<void> => {
if (fsExtra.pathExistsSync(path.resolve(repoDir, "node_modules"))) {
CONSOLE_LOG("Skip dependencies installing");
} else if (flow.install) {
await flow.install(flowOptions);
} else {
await execShell(["npm", ["ci"], {cwd: repoDir}]);
}
};

if (fsExtra.pathExistsSync(repoDistBackupDir)) { // taking dist from the backup
const src = repoDistBackupDir;
const dest = repoDistDir;

CONSOLE_LOG(`Copying backup ${src} to ${dest}`);
await fsExtra.copy(src, dest);

if (repoType === "proton-mail") {
// installing modules since required by "./scripts/prepare-webclient/monaco-editor-dts.ts"
await installModules();
}
} else { // executing the build
await installModules();

if (shouldFailOnBuild) {
throw new Error(`Halting since "${shouldFailOnBuildEnvVarName}" env var has been enabled`);
Expand Down
22 changes: 5 additions & 17 deletions scripts/prepare-webclient/protonmail.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@ import fsExtra from "fs-extra";
import path from "path";

import {BINARY_NAME} from "src/shared/constants";
import {catchTopLeventAsync, execShell} from "scripts/lib";
import {CWD_ABSOLUTE_DIR} from "scripts/const";
import {executeBuildFlow, FolderAsDomainEntry, printAndWriteFile} from "./lib";
import {FolderAsDomainEntry, executeBuildFlow, printAndWriteFile} from "./protonmail-lib";
import {PROVIDER_REPO_MAP, PROVIDER_REPO_NAMES} from "src/shared/proton-apps-constants";
import {applyPatch, execShell} from "scripts/lib";

const folderAsDomainEntries: Array<FolderAsDomainEntry<{
configApiParam:
Expand Down Expand Up @@ -178,24 +178,12 @@ function resolveWebpackConfigPatchingCode(
return result;
}

const applyPatch = async ({patchFile, cwd}: { patchFile: string; cwd: string }): Promise<void> => {
await execShell([
"git",
[
"apply",
"--ignore-whitespace",
"--reject",
patchFile,
],
{cwd},
]);
};

catchTopLeventAsync(async () => {
export const buildProtonClients = async ({destDir}: { destDir: string }): Promise<void> => {
for (const repoType of PROVIDER_REPO_NAMES) {
await executeBuildFlow({
repoType,
folderAsDomainEntries,
destDir,
destSubFolder: PROVIDER_REPO_MAP[repoType].baseDirName,
flow: {
async install({repoDir}) {
Expand Down Expand Up @@ -277,4 +265,4 @@ catchTopLeventAsync(async () => {
},
});
}
});
};

0 comments on commit aaf0235

Please sign in to comment.