Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Meteor 3 / NodeJS 18 upgrade #29112

Draft
wants to merge 100 commits into
base: develop
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
100 commits
Select commit Hold shift + click to select a range
c18c84b
Initial implementation of deasync on outgoing integrations
rodrigok Apr 27, 2023
e02504d
Remove fibers from incoming integration
rodrigok Apr 29, 2023
3b88bb9
Merge branch 'develop' into refactor/deasync
rodrigok Apr 29, 2023
1829571
Merge remote-tracking branch 'origin/develop' into refactor/deasync
rodrigok May 2, 2023
0392a18
Internalize tap
rodrigok Apr 19, 2023
1976293
replay release-3.0 branch
KevLehman May 2, 2023
6d046a2
assets
KevLehman May 2, 2023
5eacdcd
add promise.await for now
KevLehman May 2, 2023
e32f655
ignore intern package
KevLehman May 3, 2023
09bff08
Merge remote-tracking branch 'origin/develop' into refactor/deasync
rodrigok May 4, 2023
9720acf
Implement deasync for apps engine
rodrigok May 5, 2023
3adb358
Merge remote-tracking branch 'origin/develop' into refactor/deasync
rodrigok May 5, 2023
02d4851
update apps-engine version to 1.39.0-deasync
rodrigok May 8, 2023
250bc1e
Merge branch 'develop' into refactor/deasync
rodrigok May 8, 2023
b61be12
Merge remote-tracking branch 'origin/develop' into refactor/deasync
rodrigok May 9, 2023
d053565
Merge remote-tracking branch 'origin/develop' into wip-release-3.0.x
rodrigok May 9, 2023
97c7e37
Some fixes
rodrigok May 9, 2023
9b1fdcb
Remove some uncessary changes
rodrigok May 9, 2023
16e1cdf
More fixes
rodrigok May 10, 2023
ab061a5
Update do meteor 2.12
rodrigok May 10, 2023
f5aa6ac
Merge branch 'refactor/deasync' into wip-release-3.0.x
rodrigok May 10, 2023
bf3cba7
Enable async local storage again
rodrigok May 11, 2023
a9db74f
Merge remote-tracking branch 'origin/develop' into wip-release-3.0.x
rodrigok May 11, 2023
b5301bb
Upgrade meteor to 3.0-alpha.6
rodrigok May 11, 2023
131e475
Ignore eslint for internalized packages
rodrigok May 11, 2023
ac081f7
Merge remote-tracking branch 'origin/develop' into wip-release-3.0.x
rodrigok May 11, 2023
cf711e4
Merge remote-tracking branch 'origin/develop' into wip-release-3.0.x
rodrigok May 15, 2023
4ca389f
Fix ts
rodrigok May 15, 2023
18c01dd
Skip problematic tests
rodrigok May 15, 2023
a3458d8
Fix password checks
rodrigok May 16, 2023
f4e36ba
Fix some tests
rodrigok May 17, 2023
ff8f2c2
Merge remote-tracking branch 'origin/develop' into wip-release-3.0.x
rodrigok May 18, 2023
3943ea6
Fix intermitent test
rodrigok May 18, 2023
101d7e8
Merge remote-tracking branch 'origin/develop' into wip-release-3.0.x
rodrigok May 18, 2023
02d7821
Merge remote-tracking branch 'origin/develop' into wip-release-3.0.x
rodrigok May 19, 2023
1648f9c
fix ts
ggazzo May 19, 2023
e6e2358
remove old streamer client
ggazzo May 19, 2023
84b013a
Merge remote-tracking branch 'origin/develop' into wip-release-3.0.x
rodrigok May 19, 2023
9eaf84b
Merge remote-tracking branch 'origin/develop' into wip-release-3.0.x
rodrigok May 22, 2023
0b0a325
Use async Assets API
rodrigok May 22, 2023
6196582
Merge remote-tracking branch 'origin/develop' into wip-release-3.0.x
rodrigok May 22, 2023
625c499
Try to use meteor dev bundle
rodrigok May 23, 2023
621785b
Merge remote-tracking branch 'origin/develop' into wip-release-3.0.x
rodrigok May 26, 2023
f1294de
Fix TS
rodrigok May 26, 2023
567c9c4
Try to build
rodrigok May 26, 2023
f14c24d
Try to fix build
rodrigok May 27, 2023
3004533
Merge branch 'develop' into wip-release-3.0.x
rodrigok Jun 16, 2023
afa72bc
Upgrade meteor to 3.0-alpha.10
rodrigok Jun 21, 2023
094a1cc
Merge branch 'develop' into wip-release-3.0.x
rodrigok Jun 21, 2023
c47c35b
Back to normal meteor install
rodrigok Jun 21, 2023
b87cf60
Revert asset async
rodrigok Jun 22, 2023
47c75a8
Fix ui breaking
rodrigok Jun 22, 2023
6bfff6c
Try to store all test traces
rodrigok Jun 22, 2023
2797672
Fix anonymous login method
rodrigok Jun 22, 2023
ce0cb81
Fix room load
rodrigok Jun 22, 2023
4494eff
Fix intermittent error executing onFreshInstall
rodrigok Jun 22, 2023
1914100
Fix ts
rodrigok Jun 22, 2023
c6dc6a7
Merge branch 'develop' into wip-release-3.0.x
rodrigok Jun 23, 2023
e8c07fa
Merge branch 'develop' into wip-release-3.0.x
rodrigok Jul 3, 2023
f3e61f2
Merge remote-tracking branch 'origin/develop' into wip-release-3.0.x
rodrigok Sep 30, 2023
c94ddfd
Update meteor to alpha 17
rodrigok Oct 27, 2023
c82d542
Merge remote-tracking branch 'origin/develop' into wip-release-3.0.x
rodrigok Oct 27, 2023
c19e356
Update node to 18.16.1
rodrigok Oct 27, 2023
b1c6e6f
Merge remote-tracking branch 'origin/develop' into wip-release-3.0.x
rodrigok Nov 2, 2023
91c7e59
Fix error when build i18n package
rodrigok Nov 2, 2023
d441e58
Fix TS error
rodrigok Nov 2, 2023
a38ed52
Fix lint
rodrigok Nov 2, 2023
9cc9e11
Add missing dependency
rodrigok Nov 2, 2023
1b6ae8b
Update Meteor Build - coverage ubuntu image version
rodrigok Nov 2, 2023
6eb1b83
Update all CI to ubuntu-22.04
rodrigok Nov 3, 2023
35903bd
Update docker images
rodrigok Nov 3, 2023
b10aa11
Try to fix docker build
rodrigok Nov 3, 2023
c674590
Use python 3 on docker-mongo
rodrigok Nov 3, 2023
a60098f
chore: bump to the latest alpha + node 20
MarcosSpessatto Dec 19, 2023
743228e
chore: fix to support latest alpha
MarcosSpessatto Dec 19, 2023
de07da0
chore: bump isolated-vm
MarcosSpessatto Dec 19, 2023
b8e1f64
fix: change upsert by upserAsync
MarcosSpessatto Dec 19, 2023
b494b5e
fix: resolve import loading order
MarcosSpessatto Dec 20, 2023
39a6a29
Merge branch 'develop' into wip-release-3.0.x
MarcosSpessatto Dec 21, 2023
69c4758
fix: update autoupdate to reflect meteor 3 release changes
MarcosSpessatto Dec 21, 2023
dfdcd79
chore: fix lint
MarcosSpessatto Dec 21, 2023
e3600ab
ci: update node to 20
MarcosSpessatto Dec 21, 2023
2c203e8
ci: fix isolated vm install
MarcosSpessatto Dec 21, 2023
20f5705
fix: absolute url workaround to circumvent meteor`s weird impl
MarcosSpessatto Dec 22, 2023
820d84c
fix: trying stuff
MarcosSpessatto Dec 28, 2023
0793ea7
Merge branch 'develop' into wip-release-3.0.x
MarcosSpessatto Dec 28, 2023
b460925
fix: wrong imports
MarcosSpessatto Dec 29, 2023
e1eeb1f
chore: bump meteor
MarcosSpessatto Dec 29, 2023
c2c1574
ci: bump playwright
MarcosSpessatto Dec 29, 2023
b29e7d6
ci: fix playwright version
MarcosSpessatto Dec 29, 2023
36759b5
fix: undo callAsync changes
MarcosSpessatto Dec 29, 2023
b085cac
fix: try to fix settings
MarcosSpessatto Dec 29, 2023
c3fea12
Merge branch 'develop' into wip-release-3.0.x
MarcosSpessatto Jan 4, 2024
1792c02
chore: bump meteor to beta
MarcosSpessatto Jan 4, 2024
551771d
Merge branch 'develop' into wip-release-3.0.x
MarcosSpessatto Jan 5, 2024
de563c6
fix: prevent settings to be created twice
MarcosSpessatto Jan 5, 2024
2217c7f
chore: testing
MarcosSpessatto Jan 5, 2024
9000a39
fix: wtf
MarcosSpessatto Jan 8, 2024
1e25f48
Merge branch 'develop' into wip-release-3.0.x
MarcosSpessatto Jan 10, 2024
b5ccae4
fix: fix lint and merge
MarcosSpessatto Jan 10, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
7 changes: 7 additions & 0 deletions RELEASE-3.0.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Instructions to run the meteor development branch of 3.0 version

1. Clone the meteor project locally following the instructions at https://github.com/meteor/meteor/blob/devel/DEVELOPMENT.md
2. Change to the branch `release-3.0`
3. Add a alias for the cloned meteor project named `md` (example `alias md=/Users/rocket.chat/Projects/meteor/meteor`). Instructions on link above as well.
4. Checkout the Rocket.Chat's branch named `meteor-3.0`
5. Run the project `NODE_OPTIONS="--trace-warnings --unhandled-rejections=warn" DISABLE_FIBERS=1 IGNORE_ASYNC_PLUGIN=1 md`
73 changes: 36 additions & 37 deletions apps/meteor/.meteor/packages
Original file line number Diff line number Diff line change
Expand Up @@ -15,36 +15,35 @@ rocketchat:streamer
rocketchat:version
rocketchat:user-presence

accounts-base@2.2.7
accounts-facebook@1.3.3
accounts-github@1.5.0
accounts-google@1.4.0
accounts-meteor-developer@1.5.0
accounts-oauth@1.4.2
accounts-password@2.3.4
accounts-twitter@1.5.0
accounts-base
accounts-facebook
accounts-github
accounts-google
accounts-meteor-developer
accounts-oauth
accounts-password
accounts-twitter

pauli:accounts-linkedin
google-oauth@1.4.3
oauth@2.2.0
oauth2@1.3.2
google-oauth
oauth
oauth2

check@1.3.2
ddp-rate-limiter@1.1.1
rate-limit@1.0.9
email@2.2.4
http@2.0.0
check
ddp-rate-limiter
rate-limit
email

meteor-base@1.5.1
ddp-common@1.4.0
webapp@1.13.4
meteor-base
ddp-common
webapp

mongo@1.16.5
mongo

reload@1.3.1
service-configuration@1.3.1
session@1.2.1
shell-server@0.5.0
reload
service-configuration
session
shell-server

dispatch:run-as-user
ostrio:cookies
Expand All @@ -53,23 +52,23 @@ kadira:flow-router

meteorhacks:inject-initial

routepolicy@1.1.1
routepolicy

webapp-hashing@1.1.1
facts-base@1.0.1
webapp-hashing
facts-base

tracker@1.3.1
reactive-dict@1.3.1
reactive-var@1.0.12
tracker
reactive-dict
reactive-var

babel-compiler@7.10.3
standard-minifier-js@2.8.1
standard-minifier-css@1.9.0
dynamic-import@0.7.2
ecmascript@0.16.6
typescript@4.9.4
babel-compiler
standard-minifier-js
standard-minifier-css
dynamic-import
ecmascript
typescript

autoupdate@1.8.0
autoupdate

# photoswipe

Expand Down
4 changes: 2 additions & 2 deletions apps/meteor/app/2fa/client/overrideMeteorCall.ts
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ const callAsyncWithTotp =
(methodName: string, args: unknown[]) =>
async (twoFactorCode: string, twoFactorMethod: string): Promise<unknown> => {
try {
const result = await callAsync(methodName, ...args, { twoFactorCode, twoFactorMethod });
const result = await callAsync(methodName, { returnServerPromise: true }, ...args, { twoFactorCode, twoFactorMethod });

return result;
} catch (error: unknown) {
Expand All @@ -59,7 +59,7 @@ Meteor.call = function (methodName: string, ...args: unknown[]): unknown {
};

Meteor.callAsync = async function _callAsyncWithTotp(methodName: string, ...args: unknown[]): Promise<unknown> {
const promise = callAsync(methodName, ...args);
const promise = callAsync(methodName, { returnServerPromise: true }, ...args);

return process2faAsyncReturn({
promise,
Expand Down
8 changes: 4 additions & 4 deletions apps/meteor/app/authentication/server/startup/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -228,7 +228,7 @@ const onCreateUserAsync = async function (options, user = {}) {

Accounts.onCreateUser(function (...args) {
// Depends on meteor support for Async
return Promise.await(onCreateUserAsync.call(this, ...args));
return onCreateUserAsync.call(this, ...args);
});

const { insertUserDoc } = Accounts;
Expand Down Expand Up @@ -263,7 +263,7 @@ const insertUserDocAsync = async function (options, user) {
};
}

const _id = insertUserDoc.call(Accounts, options, user);
const _id = await insertUserDoc.call(Accounts, options, user);

user = await Users.findOne({
_id,
Expand Down Expand Up @@ -313,7 +313,7 @@ const insertUserDocAsync = async function (options, user) {

Accounts.insertUserDoc = function (...args) {
// Depends on meteor support for Async
return Promise.await(insertUserDocAsync.call(this, ...args));
return insertUserDocAsync.call(this, ...args);
};

const validateLoginAttemptAsync = async function (login) {
Expand Down Expand Up @@ -385,7 +385,7 @@ const validateLoginAttemptAsync = async function (login) {

Accounts.validateLoginAttempt(function (...args) {
// Depends on meteor support for Async
return Promise.await(validateLoginAttemptAsync.call(this, ...args));
return validateLoginAttemptAsync.call(this, ...args);
});

Accounts.validateNewUser(function (user) {
Expand Down
7 changes: 1 addition & 6 deletions apps/meteor/app/cas/server/cas_server.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ import { Accounts } from 'meteor/accounts-base';
import { WebApp } from 'meteor/webapp';
import { RoutePolicy } from 'meteor/routepolicy';
import _ from 'underscore';
import fiber from 'fibers';
import { CredentialTokens, Rooms, Users } from '@rocket.chat/models';
import { validate } from '@rocket.chat/cas-validate';

Expand Down Expand Up @@ -99,11 +98,7 @@ const middleware = function (req, res, next) {

// Listen to incoming OAuth http requests
WebApp.connectHandlers.use(function (req, res, next) {
// Need to create a fiber since we're using synchronous http calls and nothing
// else is wrapping this in a fiber automatically
fiber(function () {
middleware(req, res, next);
}).run();
middleware(req, res, next);
});

/*
Expand Down
181 changes: 94 additions & 87 deletions apps/meteor/app/file-upload/server/lib/proxy.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,92 +13,99 @@ import { isDocker } from '../../../utils/server';

const logger = new Logger('UploadProxy');

WebApp.connectHandlers.stack.unshift({
route: '',
// eslint-disable-next-line @typescript-eslint/no-misused-promises
async handle(req: createServer.IncomingMessage, res: http.ServerResponse, next: NextFunction) {
// Quick check to see if request should be catch
if (!req.url?.includes(`/${UploadFS.config.storesPath}/`)) {
return next();
}

logger.debug({ msg: 'Upload URL:', url: req.url });

if (req.method !== 'POST') {
return next();
}

// Remove store path
const parsedUrl = URL.parse(req.url);
const path = parsedUrl.pathname?.substr(UploadFS.config.storesPath.length + 1) || '';

// Get store
const regExp = new RegExp('^/([^/?]+)/([^/?]+)$');
const match = regExp.exec(path);

// Request is not valid
if (match === null) {
res.writeHead(400);
res.end();
return;
}

// Get store
const store = UploadFS.getStore(match[1]);
if (!store) {
res.writeHead(404);
res.end();
return;
}

// Get file
const fileId = match[2];
const file = await store.getCollection().findOne({ _id: fileId });
if (!file) {
res.writeHead(404);
res.end();
return;
}

if (!file.instanceId || file.instanceId === InstanceStatus.id()) {
logger.debug('Correct instance');
return next();
}

// Proxy to other instance
const instance = await InstanceStatusModel.findOneById(file.instanceId);

if (instance == null) {
res.writeHead(404);
res.end();
return;
}

if (instance.extraInformation.host === process.env.INSTANCE_IP && isDocker() === false) {
instance.extraInformation.host = 'localhost';
}

logger.debug(`Wrong instance, proxing to ${instance.extraInformation.host}:${instance.extraInformation.port}`);

const options = {
hostname: instance.extraInformation.host,
port: instance.extraInformation.port,
path: req.originalUrl,
method: 'POST',
};

logger.warn(
'UFS proxy middleware is deprecated as this upload method is not being used by Web/Mobile Clients. See this: https://docs.rocket.chat/api/rest-api/methods/rooms/upload',
);
// eslint-disable-next-line @typescript-eslint/naming-convention
const proxy = http.request(options, function (proxy_res) {
proxy_res.pipe(res, {
end: true,
});
});

req.pipe(proxy, {
// @ts-expect-error - l
const dummyRouter = WebApp.connectHandlers._router;

// Create a dummy route
dummyRouter.route('*');
// fetch the newly created "layer"
const stackedRoute = dummyRouter.stack.pop();
stackedRoute.handle = async (req: createServer.IncomingMessage, res: http.ServerResponse, next: NextFunction) => {
// Quick check to see if request should be catch
if (!req.url?.includes(`/${UploadFS.config.storesPath}/`)) {
return next();
}

logger.debug({ msg: 'Upload URL:', url: req.url });

if (req.method !== 'POST') {
return next();
}

// Remove store path
const parsedUrl = URL.parse(req.url);
const path = parsedUrl.pathname?.substr(UploadFS.config.storesPath.length + 1) || '';

// Get store
const regExp = new RegExp('^/([^/?]+)/([^/?]+)$');
const match = regExp.exec(path);

// Request is not valid
if (match === null) {
res.writeHead(400);
res.end();
return;
}

// Get store
const store = UploadFS.getStore(match[1]);
if (!store) {
res.writeHead(404);
res.end();
return;
}

// Get file
const fileId = match[2];
const file = await store.getCollection().findOne({ _id: fileId });
if (!file) {
res.writeHead(404);
res.end();
return;
}

if (!file.instanceId || file.instanceId === InstanceStatus.id()) {
logger.debug('Correct instance');
return next();
}

// Proxy to other instance
const instance = await InstanceStatusModel.findOneById(file.instanceId);

if (instance == null) {
res.writeHead(404);
res.end();
return;
}

if (instance.extraInformation.host === process.env.INSTANCE_IP && isDocker() === false) {
instance.extraInformation.host = 'localhost';
}

logger.debug(`Wrong instance, proxing to ${instance.extraInformation.host}:${instance.extraInformation.port}`);

const options = {
hostname: instance.extraInformation.host,
port: instance.extraInformation.port,
path: req.originalUrl,
method: 'POST',
};

logger.warn(
'UFS proxy middleware is deprecated as this upload method is not being used by Web/Mobile Clients. See this: https://docs.rocket.chat/api/rest-api/methods/rooms/upload',
);
// eslint-disable-next-line @typescript-eslint/naming-convention
const proxy = http.request(options, function (proxy_res) {
proxy_res.pipe(res, {
end: true,
});
},
});
});

req.pipe(proxy, {
end: true,
});
};

// Move the layer to the top :)
// @ts-expect-error - l
WebApp.connectHandlers._router.stack.unshift(stackedRoute);
2 changes: 1 addition & 1 deletion apps/meteor/app/livechat/server/livechat.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import { WebApp } from 'meteor/webapp';
import { settings } from '../../settings/server';
import { addServerUrlToIndex } from '../lib/Assets';

const indexHtmlWithServerURL = addServerUrlToIndex(Assets.getText('livechat/index.html') || '');
const indexHtmlWithServerURL = addServerUrlToIndex((await Assets.getText('livechat/index.html')) || '');

WebApp.connectHandlers.use('/livechat', (req, res, next) => {
if (!req.url) {
Expand Down