Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a new config for strict typescript checks #10433

Merged
merged 7 commits into from
Aug 27, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,10 @@ jobs:
name: Run typescript tests
command: |
python -m scripts.typescript_checks
- run:
name: Run typescript tests in strict mode
command: |
python -m scripts.typescript_checks --strict_checks
Comment on lines 117 to +123
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding is that we want the typescript checks to pass before running e2e tests, that's why we have these in CircleCI. Is strict check tests required to pass for the e2e tests to run? Or is it specific to typescript checks itself?
If its the latter, can we move it to Github Actions? @nithusha21 FYI

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know, actually do we need both typescript checks and the strict typescript checks? What's the difference? Would just the strict checks suffice?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually the thing is that we want to make the typescript checks more strict so we are enabling the strict flag in tsconfig. But the strict checks can't be enabled in the complete codebase due to a lot of ts errors. You can refer the discussion at #10327

So, only some files are compiled using the strict config and we plan to cover the complete codebase gradually.

I think we can enable the strict checks in circle ci given that they take a less amount of time (currently only 3s) and the checks are also present in pre_push_hoooks.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah okay, sounds good. Thanks for the explanation!

- persist_to_workspace:
root: /home/circleci/
paths:
Expand Down
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@
/core/templates/services/UpgradedServices.ts @bansalnitish @srijanreddy98
/typings/ @ankita240796
/tsconfig.json @ankita240796
/tsconfig-strict.json @nishantwrp @vojtechjelinek


# Answer classification team.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ import { AdminRouterService } from
'pages/admin-page/services/admin-router.service.ts';

describe('Admin router service', () => {
let ars: AdminRouterService = null;
let ars: AdminRouterService;

beforeEach(() => {
ars = new AdminRouterService();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,9 @@ export class AdminRouterService {
AdminPageConstants.ADMIN_TAB_URLS.ACTIVITIES);

getTabNameByHash(tabHash: string): string | null {
for (var tabName in AdminPageConstants.ADMIN_TAB_URLS) {
if (AdminPageConstants.ADMIN_TAB_URLS[tabName] === tabHash) {
for (const [tabName, tabUrl] of Object.entries(
AdminPageConstants.ADMIN_TAB_URLS)) {
if (tabUrl === tabHash) {
return tabName;
}
}
Expand Down
12 changes: 12 additions & 0 deletions scripts/pre_push_hook.py
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,8 @@
TRAVIS_CI_PROTRACTOR_CHECK_CMDS = [
PYTHON_CMD, '-m', 'scripts.check_e2e_tests_are_captured_in_ci']
TYPESCRIPT_CHECKS_CMDS = [PYTHON_CMD, '-m', 'scripts.typescript_checks']
STRICT_TYPESCRIPT_CHECKS_CMDS = [
PYTHON_CMD, '-m', 'scripts.typescript_checks', '--strict_checks']
GIT_IS_DIRTY_CMD = 'git status --porcelain --untracked-files=no'


Expand Down Expand Up @@ -453,6 +455,16 @@ def main(args=None):
'Push aborted due to failing typescript checks.')
sys.exit(1)

strict_typescript_checks_status = 0
if does_diff_include_ts_files(files_to_lint):
strict_typescript_checks_status = run_script_and_get_returncode(
STRICT_TYPESCRIPT_CHECKS_CMDS)
Copy link
Member

@DubeySandeep DubeySandeep Aug 25, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to run the typescript checks twice? If yes, will it be possible to run the "normal/simple" checks as well using the --strict_mode?

Also, if we are running the script twice are we compiling files twice?

One more: How much time will it add for pre_push checks?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually the thing is that we want to make the typescript checks more strict so we are enabling the strict flag in tsconfig. But the strict checks can't be enabled in the complete codebase due to a lot of ts errors. You can refer the discussion at #10327

So, only some files are compiled using the strict config and we plan to cover the complete codebase gradually.

Also, I think it would be better to run only the strict checks when --strict_checks flag is used.

The time added to the pre_push_hooks is about 3 seconds.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, so in the future, we will only have strict checks. Thanks for the info!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, correct!

if strict_typescript_checks_status != 0:
python_utils.PRINT(
'Push aborted due to failing typescript checks in '
'strict mode.')
sys.exit(1)

frontend_status = 0
travis_ci_check_status = 0
if does_diff_include_js_or_ts_files(files_to_lint):
Expand Down
20 changes: 20 additions & 0 deletions scripts/pre_push_hook_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -601,6 +601,26 @@ def mock_run_script_and_get_returncode(unused_script):
self.assertTrue(
'Push aborted due to failing typescript checks.' in self.print_arr)

def test_strict_typescript_check_failiure(self):
self.does_diff_include_ts_files = True
def mock_run_script_and_get_returncode(script):
if script == pre_push_hook.STRICT_TYPESCRIPT_CHECKS_CMDS:
return 1
return 0

run_script_and_get_returncode_swap = self.swap(
pre_push_hook, 'run_script_and_get_returncode',
mock_run_script_and_get_returncode)
with self.get_remote_name_swap, self.get_refs_swap, self.print_swap:
with self.collect_files_swap, self.uncommitted_files_swap:
with self.check_output_swap, self.start_linter_swap:
with self.ts_swap, run_script_and_get_returncode_swap:
with self.assertRaisesRegexp(SystemExit, '1'):
pre_push_hook.main(args=[])
self.assertTrue(
'Push aborted due to failing typescript checks in '
'strict mode.' in self.print_arr)

def test_frontend_test_failure(self):
self.does_diff_include_js_or_ts_files = True
def mock_run_script_and_get_returncode(unused_script):
Expand Down
37 changes: 33 additions & 4 deletions scripts/typescript_checks.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
from __future__ import absolute_import # pylint: disable=import-only-modules
from __future__ import unicode_literals # pylint: disable=import-only-modules

import argparse
import json
import os
import shutil
Expand All @@ -26,8 +27,21 @@
import python_utils
from . import common

_PARSER = argparse.ArgumentParser(
description="""
Run the script from the oppia root folder:
python -m scripts.typescript_checks
Note that the root folder MUST be named 'oppia'.
""")

_PARSER.add_argument(
'--strict_checks',
help='optional; if specified, compiles typescript using strict config.',
action='store_true')

COMPILED_JS_DIR = os.path.join('local_compiled_js_for_test', '')
TSCONFIG_FILEPATH = 'tsconfig.json'
STRICT_TSCONFIG_FILEPATH = 'tsconfig-strict.json'


def validate_compiled_js_dir():
Expand All @@ -41,8 +55,13 @@ def validate_compiled_js_dir():
'in %s: %s' % (COMPILED_JS_DIR, TSCONFIG_FILEPATH, out_dir))


def compile_and_check_typescript():
"""Compiles typescript files and checks the compilation errors."""
def compile_and_check_typescript(config_path):
"""Compiles typescript files and checks the compilation errors.

Args:
config_path: str. The config that should be used to run the typescript
checks.
"""
node_path = common.NODE_PATH
os.environ['PATH'] = '%s/bin:' % node_path + os.environ['PATH']

Expand All @@ -54,7 +73,7 @@ def compile_and_check_typescript():
python_utils.PRINT('Compiling and testing typescript...')
cmd = [
'./node_modules/typescript/bin/tsc', '--project',
TSCONFIG_FILEPATH]
config_path]
process = subprocess.Popen(cmd, stdout=subprocess.PIPE)
error_messages = []
for line in iter(process.stdout.readline, ''):
Expand All @@ -70,7 +89,17 @@ def compile_and_check_typescript():
python_utils.PRINT('Compilation successful!')


def main(args=None):
"""Run the typescript checks."""

parsed_args = _PARSER.parse_args(args=args)
compile_and_check_typescript(
STRICT_TSCONFIG_FILEPATH
if parsed_args.strict_checks else
TSCONFIG_FILEPATH)


# The 'no coverage' pragma is used as this line is un-testable. This is because
# it will only be called when typescript_checks.py is used as a script.
if __name__ == '__main__': # pragma: no cover
compile_and_check_typescript()
main()
41 changes: 35 additions & 6 deletions scripts/typescript_checks_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,8 @@ def test_compiled_js_dir_validation(self):
with outDir in typescript_checks.TSCONFIG_FILEPATH.
"""
with self.popen_swap:
typescript_checks.compile_and_check_typescript()
typescript_checks.compile_and_check_typescript(
typescript_checks.TSCONFIG_FILEPATH)
out_dir = ''
with python_utils.open_file(
typescript_checks.TSCONFIG_FILEPATH, 'r') as f:
Expand All @@ -60,7 +61,8 @@ def test_compiled_js_dir_validation(self):
'in %s: %s' % (
MOCK_COMPILED_JS_DIR, typescript_checks.TSCONFIG_FILEPATH,
out_dir)):
typescript_checks.compile_and_check_typescript()
typescript_checks.compile_and_check_typescript(
typescript_checks.TSCONFIG_FILEPATH)

def test_compiled_js_dir_is_deleted_before_compilation(self):
"""Test that compiled_js_dir is deleted before a fresh compilation."""
Expand All @@ -76,7 +78,8 @@ def mock_validate_compiled_js_dir():
if not os.path.exists(os.path.dirname(MOCK_COMPILED_JS_DIR)):
os.mkdir(os.path.dirname(MOCK_COMPILED_JS_DIR))

typescript_checks.compile_and_check_typescript()
typescript_checks.compile_and_check_typescript(
typescript_checks.TSCONFIG_FILEPATH)
self.assertFalse(
os.path.exists(os.path.dirname(MOCK_COMPILED_JS_DIR)))

Expand All @@ -99,14 +102,16 @@ def mock_popen_for_deletion(unused_cmd, stdout): # pylint: disable=unused-argum
with popen_swap, compiled_js_dir_swap, validate_swap:
if not os.path.exists(os.path.dirname(MOCK_COMPILED_JS_DIR)):
os.mkdir(os.path.dirname(MOCK_COMPILED_JS_DIR))
typescript_checks.compile_and_check_typescript()
typescript_checks.compile_and_check_typescript(
typescript_checks.TSCONFIG_FILEPATH)
self.assertFalse(
os.path.exists(os.path.dirname(MOCK_COMPILED_JS_DIR)))

def test_no_error_is_produced_for_valid_compilation(self):
"""Test that no error is produced if stdout is empty."""
with self.popen_swap:
typescript_checks.compile_and_check_typescript()
typescript_checks.compile_and_check_typescript(
typescript_checks.TSCONFIG_FILEPATH)

def test_error_is_produced_for_invalid_compilation(self):
"""Test that error is produced if stdout is not empty."""
Expand All @@ -116,4 +121,28 @@ def mock_popen_for_errors(unused_cmd, stdout): # pylint: disable=unused-argumen

with self.swap(subprocess, 'Popen', mock_popen_for_errors):
with self.assertRaisesRegexp(SystemExit, '1'):
typescript_checks.compile_and_check_typescript()
typescript_checks.compile_and_check_typescript(
typescript_checks.TSCONFIG_FILEPATH)

def test_config_path_when_no_arg_is_used(self):
"""Test if the config path is correct when no arg is used."""
def mock_compile_and_check_typescript(config_path):
self.assertEqual(config_path, typescript_checks.TSCONFIG_FILEPATH)
compile_and_check_typescript_swap = self.swap(
typescript_checks, 'compile_and_check_typescript',
mock_compile_and_check_typescript)

with compile_and_check_typescript_swap:
typescript_checks.main(args=[])

def test_config_path_when_strict_checks_arg_is_used(self):
"""Test if the config path is correct when strict checks arg is used."""
def mock_compile_and_check_typescript(config_path):
self.assertEqual(
config_path, typescript_checks.STRICT_TSCONFIG_FILEPATH)
compile_and_check_typescript_swap = self.swap(
typescript_checks, 'compile_and_check_typescript',
mock_compile_and_check_typescript)

with compile_and_check_typescript_swap:
typescript_checks.main(args=['--strict_checks'])
38 changes: 38 additions & 0 deletions tsconfig-strict.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
{
"compilerOptions": {
"allowJs": true,
"downlevelIteration": true,
"lib": ["es2017", "dom", "webworker"],
"outDir": "local_compiled_js_for_test",
"rootDir": ".",
"skipLibCheck": true,
"target": "es5",
"sourceMap": true,
"strict": true,
"typeRoots": ["./node_modules/@types"],
"experimentalDecorators": true,
"emitDecoratorMetadata": true,
"allowSyntheticDefaultImports": true,
"esModuleInterop": true,
"baseUrl": ".",
"paths": {
"app.constants": ["core/templates/app.constants"],
"app-type.constants": ["core/templates/app-type.constants"],
"components/*": ["core/templates/components/*"],
"domain/*": ["core/templates/domain/*"],
"expressions/*": ["core/templates/expressions/*"],
"pages/*": ["core/templates/pages/*"],
"services/*": ["core/templates/services/*"],
"classifiers/*": ["extensions/classifiers/*"],
"interactions/*": ["extensions/interactions/*"],
"filters/*": ["core/templates/filters/*"],
"static/*": ["third_party/static/*"],
"tests/*": ["core/templates/tests/*"]
}
},
"files": [
"core/templates/pages/admin-page/services/admin-router.service.ts",
"core/templates/pages/admin-page/services/admin-router.service.spec.ts"
],
"include": ["typings/*.ts"]
}