Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve error warning log pollution + composer recipes (Part 3) #100

Merged
merged 32 commits into from
Oct 24, 2023

Conversation

melroy89
Copy link
Member

@melroy89 melroy89 commented Oct 22, 2023

  • Recipe updates (doctrine-migrations-bundle, flystem-bundle, monolog-bundle, stimulus-bundle, translation)
  • Enable log rotation on production (max 7 files, one per dag)
  • Enable log rotation in development mode as well (max 10 files)
  • Full stacktraces in development mode, strict logging on action level, buffer size, and deprecation to stderr
  • Change default development log level from debug to info, with a remark the developer can always enable debug. Warning: debug is very very verbose, hence info as default.

@melroy89 melroy89 changed the title Improve error warning log pollution 3 Improve error warning log pollution + docs (Part 3) Oct 22, 2023
@melroy89 melroy89 force-pushed the improve_error_warning_log_pollution_3 branch from cc2c73c to 558c2ab Compare October 22, 2023 22:54
@ghost ghost added documentation Improvements or additions to documentation backend Backend related issues and pull requests enhancement New feature or request labels Oct 23, 2023
@melroy89 melroy89 force-pushed the improve_error_warning_log_pollution_3 branch from 886c2bb to 0982da1 Compare October 23, 2023 16:20
@melroy89

This comment was marked as outdated.

@melroy89

This comment was marked as off-topic.

@melroy89 melroy89 marked this pull request as ready for review October 23, 2023 22:51
@melroy89 melroy89 changed the title Improve error warning log pollution + docs (Part 3) Improve error warning log pollution (Part 3) Oct 23, 2023
@melroy89 melroy89 changed the title Improve error warning log pollution (Part 3) Improve error warning log pollution + composer recipes (Part 3) Oct 23, 2023
@ghost
Copy link

ghost commented Oct 24, 2023

I got this info/warning during the migration step, is this expected?

[2023-10-24T13:27:10.041008+00:00] deprecation.INFO: User Deprecated: In ORM 3.0, the AttributeDriver will report fields for the classes where they are declared. This may uncover invalid mapping configurations. To opt into the new mode today, set the "reportFieldsWhereDeclared" constructor parameter to true. (AttributeDriver.php:78 called by App_KernelProdContainer.php:530, https://github.com/doctrine/orm/pull/10455, package doctrine/orm) {"exception":"[object] (ErrorException(code: 0): User Deprecated: In ORM 3.0, the AttributeDriver will report fields for the classes where they are declared. This may uncover invalid mapping configurations. To opt into the new mode today, set the \"reportFieldsWhereDeclared\" constructor parameter to true. (AttributeDriver.php:78 called by App_KernelProdContainer.php:530, https://github.com/doctrine/orm/pull/10455, package doctrine/orm) at /var/www/kbin/vendor/doctrine/deprecations/lib/Doctrine/Deprecations/Deprecation.php:210)"} []

@ghost
Copy link

ghost commented Oct 24, 2023

Also, my prod log file is already at 40M about a second after updating and growing about 1 MB every second...lol.

@melroy89
Copy link
Member Author

Ps. Sorry about 30 commits. I can rewrite git history if you wish. Then again we will use squash merge so it will be a single commit anyway..

@melroy89
Copy link
Member Author

Also, my prod log file is already at 40M about a second after updating and growing about 1 MB every second...lol.

Ow that is maybe not cool.. I did enable log rotation. But it shouldn't spam that much to prod.log. Dev.log is a different story of course. Let me check

@melroy89
Copy link
Member Author

melroy89 commented Oct 24, 2023

I got this info/warning during the migration step, is this expected?

[2023-10-24T13:27:10.041008+00:00] deprecation.INFO: User Deprecated: In ORM 3.0, the AttributeDriver will report fields for the classes where they are declared. This may uncover invalid mapping configurations. To opt into the new mode today, set the "reportFieldsWhereDeclared" constructor parameter to true. (AttributeDriver.php:78 called by App_KernelProdContainer.php:530, https://github.com/doctrine/orm/pull/10455, package doctrine/orm) {"exception":"[object] (ErrorException(code: 0): User Deprecated: In ORM 3.0, the AttributeDriver will report fields for the classes where they are declared. This may uncover invalid mapping configurations. To opt into the new mode today, set the \"reportFieldsWhereDeclared\" constructor parameter to true. (AttributeDriver.php:78 called by App_KernelProdContainer.php:530, https://github.com/doctrine/orm/pull/10455, package doctrine/orm) at /var/www/kbin/vendor/doctrine/deprecations/lib/Doctrine/Deprecations/Deprecation.php:210)"} []

Is the deprecation message pointing to a file in our code base? I don't think so?

@asdfzdfj
Copy link
Contributor

asdfzdfj commented Oct 24, 2023

Also, my prod log file is already at 40M about a second after updating and growing about 1 MB every second...lol.

Ow that is maybe not cool.. I did enable log rotation. But it shouldn't spam that much to prod.log. Dev.log is a different story of course. Let me check

it looks like you also axed fingers_crossed log type in the process (from my understanding it would only print out logs when message with higher configured severity appears, along with some previous log lines for context)

so now it would just log all that's enabled

@ghost
Copy link

ghost commented Oct 24, 2023

I got this info/warning during the migration step, is this expected?

[2023-10-24T13:27:10.041008+00:00] deprecation.INFO: User Deprecated: In ORM 3.0, the AttributeDriver will report fields for the classes where they are declared. This may uncover invalid mapping configurations. To opt into the new mode today, set the "reportFieldsWhereDeclared" constructor parameter to true. (AttributeDriver.php:78 called by App_KernelProdContainer.php:530, https://github.com/doctrine/orm/pull/10455, package doctrine/orm) {"exception":"[object] (ErrorException(code: 0): User Deprecated: In ORM 3.0, the AttributeDriver will report fields for the classes where they are declared. This may uncover invalid mapping configurations. To opt into the new mode today, set the \"reportFieldsWhereDeclared\" constructor parameter to true. (AttributeDriver.php:78 called by App_KernelProdContainer.php:530, https://github.com/doctrine/orm/pull/10455, package doctrine/orm) at /var/www/kbin/vendor/doctrine/deprecations/lib/Doctrine/Deprecations/Deprecation.php:210)"} []

Is the deprecation message pointing to a file in our code base? I don't think so?

Good point... just pointing out the obvious if you will. Doesn't bother me if it doesn't bother you.

@melroy89
Copy link
Member Author

melroy89 commented Oct 24, 2023

it looks like you also axed fingers_crossed log type in the process (from my understanding it would only print out logs when message with higher configured severity appears, along with some previous log lines for context)

so now it would just log all that's enabled

It was not very well documented on https://symfony.com/doc/current/logging.html#how-to-rotate-your-log-files, but we now use rotating_file in the nested section. Resolving the issue big files due to rotating_files in the main section of production. But instead of type stream in dev, I now use the rotating_files type in the main section of the development config.

This last config allows people to run development mode on a federated instance without worrying to get a 100GB dev.log files (I have been there.. done that 😢 ).

Copy link
Contributor

@asdfzdfj asdfzdfj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think I see anything particularly standing out, and all seems to be looking fine. @nobodyatroot ?

@ghost ghost self-requested a review October 24, 2023 15:21
Copy link

@ghost ghost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Much better, no more exploding logfile size!

@melroy89
Copy link
Member Author

Much better, no more exploding logfile size!

sorry about that ^^ I made it worse instead of better. But now it should be even better than before.

@melroy89 melroy89 merged commit 159f426 into main Oct 24, 2023
3 checks passed
@melroy89 melroy89 deleted the improve_error_warning_log_pollution_3 branch October 24, 2023 15:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend Backend related issues and pull requests enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants