Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stable/v0 0 1 #35

Merged
merged 20 commits into from
May 23, 2024
Merged

Stable/v0 0 1 #35

merged 20 commits into from
May 23, 2024

Conversation

iNewLegend
Copy link
Member

@iNewLegend iNewLegend commented May 21, 2024

No description provided.

…ad handling

This commit includes a number of changes in the console involving new classes and the removal of old ones all aimed at improving thread handling in the console. It introduces new classes like `ConsoleThreadSend`, `ConsoleBuildBase`, `ConsoleThreadFormat`, `ConsoleWatchBase`, `ConsoleThreadReceive` and removes old ones like `thread-console.ts` and `watch-console.ts`. These changes make the console output thread aware, ensuring messages from different threads are outputted correctly, which markedly improves the clarity and orderliness of the console output.
The code changes improved the behavior and error handling of Typescript and Rollup build tasks within the CLI. Notably, the handling of dependencies in multi-threaded builds was advanced. Unnecessary/unused import was also removed in "common.ts". Other updates include improvements to console logging and error handling for clarity and better debugging. The 'zTSPreDiagnostics' function was further refactored for improved accuracy.
The commit includes a new plugin that enforces no relative import in the Zenflux Eslint configuration. This plugin encourages more consistent and maintainable code by restricting the use of local file paths in import statements. Import statements using relative paths are flagged as errors, and a message is provided to guide users towards the preferred path alias syntax.
…guides

Adjusted some TypeScript rules: changed `typescript_import_prefer_absolute_path` to false and `typescript_import_sort_module_name` to true, as well as updated `typescript_import_use_node_resolution` to global. Also, `typescript_use_import_type` is now always used. Furthermore, whitespace in `ij_visual_guides` and `ij_property_prefix` for both JavaScript and TypeScript has been removed for cleaner code.
Updated dependencies for several packages including react-x-env and react-noop-renderer to include '@zenflux/react-scheduler'. Removed 'bun-types' from the types array in the zenflux-cli tsconfig.json. Modified scripts in the main package.json including updating the version and packageManager properties, and adding a new "@z-flux:dev" command.
This workflow is responsible for building and running tests for React packages. It runs on each push and pull request, checking out the code, setting up the environment, installing dependencies, building the code, and finally running the tests.
@iNewLegend iNewLegend force-pushed the stable/v0-0-1 branch 2 times, most recently from 05a07f0 to 179e397 Compare May 21, 2024 14:23
Updates the testing environment for react packages in GitHub Actions, allowing tests to run either using GitHub's host or self-hosted runners.
@iNewLegend
Copy link
Member Author

/describe
--pr_description.publish_labels=false
--pr_description.publish_description_as_comment=false
--pr_description.generate_ai_title=true
--pr_description.use_description_markers=true
--pr_description.final_update_message=false

@iNewLegend
Copy link
Member Author

/describe

Copy link
Contributor

PR Review 🔍

⏱️ Estimated effort to review [1-5]

4, because the PR introduces a significant amount of changes across multiple files, involving complex modifications in threading, console management, and TypeScript utilities. The changes impact core functionalities and require a thorough review to ensure they integrate well without causing regressions or bugs.

🧪 Relevant tests

No

⚡ Possible issues

Possible Bug: The use of dynamic imports in threading might lead to race conditions or errors if not handled properly.

Performance Concern: Frequent file system operations in 'zApiExporter' and 'findNearestPackageJson' could lead to performance bottlenecks.

🔒 Security concerns

No

Code feedback:
relevant filepackages/zenflux-cli/src/core/typescript.ts
suggestion      

Consider implementing caching for 'zTSGetPackageByConfig' to avoid redundant filesystem operations which can degrade performance. [important]

relevant lineconst packagesCache = new Map();

relevant filepackages/zenflux-cli/src/console/console-thread-receive.ts
suggestion      

Ensure that all possible event types are handled in 'ConsoleThreadReceive.connect'. Currently, only 'DEFAULT_WORKER_EVENTS' are considered. You might want to add a fallback or error handling for unexpected event types. [medium]

relevant lineDEFAULT_WORKER_EVENTS.forEach( ( event ) => {

relevant filepackages/zenflux-cli/src/core/api-extractor.ts
suggestion      

To improve the readability and maintainability of the logging within 'zApiExporter', consider creating a separate function to handle the different types of log messages (error, warning, verbose, info). This will make the main function cleaner and easier to understand. [medium]

relevant lineactiveConsole.verbose( () => [ zApiExporter.name, util.inspect( {

relevant filepackages/zenflux-cli/src/console/console-thread-send.ts
suggestion      

Add error handling for the case where 'args[0]' is not an instance of Error in 'ConsoleThreadSend.output'. This will prevent potential runtime errors if 'args[0]' is expected to be an error but is not. [important]

relevant lineif ( args[ 0 ] instanceof Error ) {

…ptions to `readme.md`

This update adds two new options to the command build base: `--no-diagnostic`, which disables typescript diagnostics, and `--no-declaration`, which disables declaration file generation and stops using the api-exporter. Both options and their behaviors have also been documented in the README file.
@iNewLegend iNewLegend force-pushed the stable/v0-0-1 branch 2 times, most recently from ff58c12 to cbd94d0 Compare May 23, 2024 04:03
…ames in workflows

The environment variable names in workflows `1-main-github-hosted.yml` and `2-manage-runners.yml` have been updated to be more explicit about their purpose. This improves readability and maintainability of the code. The logic remains the same, only the variable names have been changed to reflect their functionalities better.
…console-thread-format.ts`

Detailed docs were added to the `ConsoleThreadFormat` class in `console-thread-format.ts`. They provide information on the expected output of several abstract methods, the class's purpose, and the behavior of the constructor function. This enhances code clarity and promotes proper implementation of the class methods.
This commit adds valuable explanatory comment describing the purpose and function of the `ConsoleThreadSend` class in `console-thread-send.ts`. This class manages sending operations in a multithreaded console environment, using the functionalities provided by ConsoleManager.
Comment on lines 29 to 76
class SimpleOutputStrategy implements OutputStrategy {
private $: ConsoleWatchBase;

public constructor( base: ConsoleWatchBase ) {
this.$ = base;
}

public output( method: TConsoleLoggerMethod, args: any[] ) {
this.$.getLogWidget().log( args.join( " " ) );
}
}

class DebugOutputStrategy implements OutputStrategy {
private $: ConsoleWatchBase;

public constructor( base: ConsoleWatchBase ) {
this.$ = base;
}

public output( method: TConsoleLoggerMethod, args: any[] ) {
if ( method.name === this.$.debug.name ) {

logWidgetDebugBuffer.push( args.join( " " ) );

return this.addDebouncedLog();
}

this.$.getLogWidget().log( args.join( " " ) );
}

private addDebouncedLog() {
// Use debounce to prevent too many logs.
if ( logWidgetDebugBuffer.length > LOG_MAX_BUFFER_SIZE ) {
this.createLogAndClearBuffer();
}

zDebounce( `console-${ this.$.getName() }-debug-buffer`, () => {
if ( logWidgetDebugBuffer.length ) {
this.createLogAndClearBuffer();
}
}, LOG_DEBUG_DEBOUNCE_DELAY );
}

private createLogAndClearBuffer() {
logWidgetDebug!.log( logWidgetDebugBuffer.join( "\n" ) );
logWidgetDebugBuffer.length = 0;
}
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/ask Can you explain what is the difference between the strategies?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The SimpleOutputStrategy and DebugOutputStrategy classes implement the OutputStrategy interface but differ in how they handle logging:

SimpleOutputStrategy

  • Purpose: Directly logs the provided arguments.
  • Method:
    • output(method: TConsoleLoggerMethod, args: any[]): Joins the args array into a single string and logs it using the log method of the log widget obtained from this.$.getLogWidget().

DebugOutputStrategy

  • Purpose: Adds a layer of buffering and debouncing for debug logs to prevent excessive logging.
  • Methods:
    • output(method: TConsoleLoggerMethod, args: any[]):
      • Checks if the method name matches this.$.debug.name.
      • If it matches, it pushes the joined args into logWidgetDebugBuffer and calls addDebouncedLog().
      • If it doesn't match, it logs the arguments directly like SimpleOutputStrategy.
    • addDebouncedLog():
      • Uses a debounce mechanism to limit the frequency of log updates.
      • If the buffer exceeds LOG_MAX_BUFFER_SIZE, it calls createLogAndClearBuffer().
      • Uses zDebounce to schedule the buffer to be logged and cleared after LOG_DEBUG_DEBOUNCE_DELAY.
    • createLogAndClearBuffer(): Joins the buffer into a single string, logs it using logWidgetDebug, and then clears the buffer.

Summary

  • SimpleOutputStrategy: Direct logging without any additional logic.
  • DebugOutputStrategy: Buffered and debounced logging specifically for debug messages to manage log volume and frequency.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/ask can you suggest /** comment **/ to explain it?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Certainly! Here are suggested comments to explain the SimpleOutputStrategy and DebugOutputStrategy classes:

/**
 * SimpleOutputStrategy class implements the OutputStrategy interface.
 * It provides a simple logging mechanism by directly logging the provided arguments.
 */
class SimpleOutputStrategy implements OutputStrategy {
    private $: ConsoleWatchBase;

    /**
     * Constructor for SimpleOutputStrategy.
     * @param base - An instance of ConsoleWatchBase.
     */
    public constructor( base: ConsoleWatchBase ) {
        this.$ = base;
    }

    /**
     * Outputs the log by joining the arguments and logging them to the log widget.
     * @param method - The logging method.
     * @param args - The arguments to log.
     */
    public output( method: TConsoleLoggerMethod, args: any[] ) {
        this.$.getLogWidget().log( args.join( " " ) );
    }
}

/**
 * DebugOutputStrategy class implements the OutputStrategy interface.
 * It provides a logging mechanism with a debounce feature to prevent too many logs.
 */
class DebugOutputStrategy implements OutputStrategy {
    private $: ConsoleWatchBase;

    /**
     * Constructor for DebugOutputStrategy.
     * @param base - An instance of ConsoleWatchBase.
     */
    public constructor( base: ConsoleWatchBase ) {
        this.$ = base;
    }

    /**
     * Outputs the log with a debounce mechanism if the method is debug.
     * Otherwise, it logs the arguments directly.
     * @param method - The logging method.
     * @param args - The arguments to log.
     */
    public output( method: TConsoleLoggerMethod, args: any[] ) {
        if ( method.name === this.$.debug.name ) {
            logWidgetDebugBuffer.push( args.join( " " ) );
            return this.addDebouncedLog();
        }
        this.$.getLogWidget().log( args.join( " " ) );
    }

    /**
     * Adds a debounced log to prevent too many logs.
     * If the buffer exceeds the maximum size, it creates a log and clears the buffer.
     */
    private addDebouncedLog() {
        if ( logWidgetDebugBuffer.length > LOG_MAX_BUFFER_SIZE ) {
            this.createLogAndClearBuffer();
        }

        zDebounce( `console-${ this.$.getName() }-debug-buffer`, () => {
            if ( logWidgetDebugBuffer.length ) {
                this.createLogAndClearBuffer();
            }
        }, LOG_DEBUG_DEBOUNCE_DELAY );
    }

    /**
     * Creates a log from the buffer and clears the buffer.
     */
    private createLogAndClearBuffer() {
        logWidgetDebug!.log( logWidgetDebugBuffer.join( "\n" ) );
        logWidgetDebugBuffer.length = 0;
    }
}

These comments provide a clear explanation of the purpose and functionality of each class and method.

…watch.ts`

This new class is responsible for managing the output of debug log messages in a batch to prevent overwhelming logs. The log buffer flushes when the maximum buffer size is attained or after a certain delay period. This strategy is employed when the debug flag is enabled.
Copy link
Contributor

Ask

can you create new issue for this todo?

Answer:

Certainly! Here is the information extracted from the PR for creating a new issue:

Issue Title:

Avoid using useless imports from thread, use dynamic imports instead

Description:

There is a TODO comment in the file packages/zenflux-cli/src/core/build.ts that suggests avoiding useless imports from threads and using dynamic imports instead. This change is intended to optimize the code by loading modules only when they are needed.

File and Line Number:

  • File: packages/zenflux-cli/src/core/build.ts
  • Line Number: 3

Code Snippet:

/**
 * @TODO: Avoid using useless imports from thread, use dynamic imports instead.
 */

Suggested Steps:

  1. Identify the imports that are currently being loaded unnecessarily.
  2. Replace these imports with dynamic imports to ensure they are only loaded when needed.
  3. Test the changes to ensure that the dynamic imports work as expected and do not introduce any new issues.

Would you like me to include any additional details or context in the issue?

@@ -1,5 +1,7 @@
/**
* @author Leonid Vinikov <leonidvinikov@gmail.com>
*
* @TODO: Avoid using useless imports from thread, use dynamic imports instead.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/ask can you create new issue for this todo?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Certainly! Here is a suggested issue description for the TODO:

Title: Refactor Imports to Use Dynamic Imports

Description:
In the file packages/zenflux-cli/src/core/build.ts, there is a TODO comment indicating that we should avoid using useless imports from the thread and instead use dynamic imports. This will help in optimizing the code and potentially improve performance.

TODO Comment:

/**
 * @TODO: Avoid using useless imports from thread, use dynamic imports instead.
 */

Tasks:

  • Identify and remove any unnecessary static imports from the thread.
  • Implement dynamic imports where applicable.
  • Test to ensure that the functionality remains intact after the changes.

File:
packages/zenflux-cli/src/core/build.ts

References:
PR: 'Stable/v0 0 1'
Branch: 'stable/v0-0-1'

Feel free to adjust the description as needed.


host.sendMessage( "message", "build", "prepare",
config.outputName,
// TODO: Sending formated message to the host, not the best practice.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/ask can you create new issue for this todo?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Certainly! Here is a draft for the new issue based on the TODO comment:

Title: Refactor message sending to host in zRollupBuildInWorker

Description:
The current implementation in zRollupBuildInWorker sends a formatted message to the host, which is not considered best practice. This needs to be refactored to improve code quality and maintainability.

File:
packages/zenflux-cli/src/core/build.ts

Current Code:

// TODO: Sending formated message to the host, not the best practice.
host.sendMessage("message", "build", "prepare", config.outputName);

Suggested Improvement:

  • Refactor the code to use a more appropriate method for sending messages to the host.
  • Consider using sendVerbose or sendMessage with a different approach that does not involve passing formatted messages directly.

Additional Context:
This change is aimed at improving the clarity and maintainability of the code. The current approach of sending formatted messages directly is not ideal and should be revised.

Feel free to adjust the issue details as needed!

}
} );

// TODO: It should favor skipping external dependencies.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/ask can you create new issue for this todo?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Certainly! Here is a suggested issue description for the TODO comment:

Title: Skip External Dependencies in zBuildThreadWaitForDependencies

Description:
In the zBuildThreadWaitForDependencies function within packages/zenflux-cli/src/core/build.ts, there is a TODO comment indicating that the function should favor skipping external dependencies. This needs to be implemented to improve the build process.

File: packages/zenflux-cli/src/core/build.ts

Line: 231 (approximate, based on the provided diff)

Current TODO Comment:

// TODO: It should favor skipping external dependencies.

Tasks:

  • Analyze the current handling of dependencies in the zBuildThreadWaitForDependencies function.
  • Implement logic to skip external dependencies.
  • Add appropriate tests to ensure the new logic works as expected.
  • Update documentation if necessary.

Additional Context:
The function currently processes dependencies from options.otherConfigs and checks them against packagesDependencies. The new logic should identify external dependencies and skip them during this process.

Feel free to adjust the issue description as needed.

import fs from "node:fs";
import path from "node:path";

// TODO: Use cache
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/ask create issue for this

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To create an issue for the // TODO: Use cache comment, you can follow these steps:

  1. Title: Implement Caching in plugin.js

  2. Description:

    • Context: In the packages/zenflux-eslint/plugin.js file, there is a TODO comment indicating the need to use caching.
    • Details: The current implementation does not utilize caching, which could potentially improve performance and efficiency.
    • Location: The TODO comment is located at the beginning of the plugin.js file.
    • Task: Implement a caching mechanism in the plugin.js file to optimize performance.
  3. Labels: Enhancement, Performance, TODO

  4. Assignee: (Assign to the relevant developer or team)

By creating this issue, it will be tracked and prioritized accordingly.

fs.readFileSync( globalThis.__Z_ESLINT_CONFIG__.zRootPackagePath, 'utf8' )
);

// TODO Move out of "@zenflux/eslint" package
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/ask create issue for this

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To create an issue for the TODO comment, you can use the following template:

Title: Move code out of "@zenflux/eslint" package

Description:
There is a TODO comment in the plugin.js file indicating that some code needs to be moved out of the @zenflux/eslint package. This task involves identifying the specific code that should be relocated and determining the appropriate new location for it.

File: packages/zenflux-eslint/plugin.js

Line: 12

Current TODO Comment:

// TODO Move out of "@zenflux/eslint" package

Tasks:

  • Identify the code that needs to be moved.
  • Determine the appropriate new location for the code.
  • Update the codebase to reflect this change.
  • Ensure all references to the moved code are updated accordingly.
  • Test to ensure no functionality is broken due to this change.

Priority: Medium

Labels: refactoring, enhancement

This issue will help track the progress of this TODO item and ensure it gets addressed in a timely manner.

import type {
TZCreateDeclarationWorkerOptions,
TZPreDiagnosticsOptions,
TZPreDiagnosticsWorkerOptions
} from "@zenflux/cli/src/definitions/typescript";

import type { Thread, ThreadHost } from "@zenflux/cli/src/modules/threading/thread";
import type { Worker } from "@zenflux/cli/src/modules/threading/worker";
import type { IZConfigInternal } from "@zenflux/cli/src/definitions/config";

// TODO: Avoid this, create threadPool with max threads = cpu cores.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/ask Create issue from todo

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To create an issue from the TODO comment, you can use the following details:

Title: Create threadPool with max threads equal to CPU cores

Description:
There is a TODO comment in the packages/zenflux-cli/src/core/typescript.ts file that suggests creating a thread pool with a maximum number of threads equal to the number of CPU cores. This enhancement aims to optimize the threading model by avoiding the creation of an excessive number of threads, which can lead to performance degradation.

Location in Code:

// TODO: Avoid this, create threadPool with max threads = cpu cores.

Proposed Solution:
Implement a thread pool that dynamically adjusts the number of threads based on the available CPU cores. This can be achieved using Node.js's os module to determine the number of CPU cores and then configuring the thread pool accordingly.

Benefits:

  • Improved performance by efficiently utilizing CPU resources.
  • Reduced overhead from managing too many threads.
  • Better scalability and resource management.

Priority: Medium

This issue will help track the progress of this enhancement and ensure it is addressed in a future update.

@iNewLegend iNewLegend merged commit 5b83208 into ZenFlux:main May 23, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant