Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reason for execution log and detailed log being completely separated? #478

Closed
bertfrees opened this issue May 20, 2016 · 1 comment
Closed

Reason for execution log and detailed log being completely separated? #478

bertfrees opened this issue May 20, 2016 · 1 comment
Labels
question
Milestone
v1.12.0

Comments

@bertfrees
Copy link
Member

bertfrees commented May 20, 2016 •
edited
Loading

This came up during the face-to-face meeting.

Job messages (execution log) go through the EventBus, are dispatched based on their job ID, and pushed to the client (PushNotifier.java). Currently only the Calabash pipelines (created for each job) are hooked up to the EventBus (CalabashXProcPipeline.java#L199). I have made use of this to get access to the EventBus from within some of my Java code (daisy/pipeline-mod-braille@7d92111), but this trick only works for Java code directly invoked within a custom Calabash step (not within a Saxon extension function for example).

The detailed log is a file that collects logging messages using org.slf4j.MDC and logback (DefaultJobExecutionService.java#L100, JobURIUtils.java#L39, config-logback.xml#L5) and that can be downloaded by the client on demand (LogResource.java#L77).

It makes sense that some messages are sent immediately to the web UI as soon as they are triggered within the job, while debug messages are only sent to a file that can be obtained later. However does it also make sense to have them completely separated and use a different mechanism for each?

We could unify the two kinds of messages by using one and the same mechanism, and dispatching them to execution log and/or detailed log based on e.g. the log level (in addition to job ID for dispatching to right job). This could have several advantages:

  • It is easier to understand for user
  • It is easier to understand for maintainer
  • If the log level is used for dispatching, the developer doesn't have to think about where the message will end up, he only needs to decide upon the importance of the message.
  • If a mechanism such as MDC is used, it becomes possible to get messages from within Java into the execution log. It also becomes possible to get messages from used libraries into the execution log.
@bertfrees bertfrees changed the title Reason for job messages and detailed log being completely separated? Reason for execution log and detailed log being completely separated? May 20, 2016
@bertfrees bertfrees mentioned this issue May 20, 2016
Open
bertfrees added a commit to daisy/pipeline-framework that referenced this issue Jun 6, 2016
@bertfrees
Mechanism for sending messages to execution log from Java
8e088a6 
using a custom Logback appender that sents log messages to the
EventBus. Attaching the right job ID is done using the same MDC based
principle that was already used in IgnoreSiftAppender.

This fixes issue daisy/pipeline#477

Note that this also follows the proposal of unifying the two types of
messages (execution log and detailed log) through one and the same
mechanism, except that XProc/XSLT messages are still sent directly to
the EventBus: see issue
daisy/pipeline#478.

In order to support messages belonging to one job coming from several
sources, the message sequence numbers are now computed automatically
within MessageBuilder, and the withSequence method has been removed.
@bertfrees bertfrees mentioned this issue Jun 6, 2016
Closed
@bertfrees bertfrees mentioned this issue Jun 15, 2016
Closed
bertfrees added a commit to sbsdev/pipeline that referenced this issue Mar 30, 2017
@bertfrees
[framework] Mechanism for sending messages to execution log from Java
81a607c 
using a custom Logback appender that sents log messages to the
EventBus. Attaching the right job ID is done using the same MDC based
principle that was already used in IgnoreSiftAppender.

This fixes issue daisy/pipeline#477

Note that this also follows the proposal of unifying the two types of
messages (execution log and detailed log) through one and the same
mechanism, except that XProc/XSLT messages are still sent directly to
the EventBus: see issue
daisy/pipeline#478.

In order to support messages belonging to one job coming from several
sources, the message sequence numbers are now computed automatically
within MessageBuilder, and the withSequence method has been removed.
bertfrees added a commit to daisy/pipeline-framework that referenced this issue Apr 11, 2017
@bertfrees
[framework] Mechanism for sending messages to execution log from Java
3ba602d 
using a custom Logback appender that sents log messages to the
EventBus. Attaching the right job ID is done using the same MDC based
principle that was already used in IgnoreSiftAppender.

This fixes issue daisy/pipeline#477

Note that this also follows the proposal of unifying the two types of
messages (execution log and detailed log) through one and the same
mechanism, except that XProc/XSLT messages are still sent directly to
the EventBus: see issue
daisy/pipeline#478.

In order to support messages belonging to one job coming from several
sources, the message sequence numbers are now computed automatically
within MessageBuilder, and the withSequence method has been removed.
bertfrees added a commit to daisy/pipeline-framework that referenced this issue Jul 31, 2017
@bertfrees
[framework] Mechanism for sending messages to execution log from Java
a8b468a 
using a custom Logback appender that sents log messages to the
EventBus. Attaching the right job ID is done using the same MDC based
principle that was already used in IgnoreSiftAppender.

This fixes issue daisy/pipeline#477

Note that this also follows the proposal of unifying the two types of
messages (execution log and detailed log) through one and the same
mechanism, except that XProc/XSLT messages are still sent directly to
the EventBus: see issue
daisy/pipeline#478.

In order to support messages belonging to one job coming from several
sources, the message sequence numbers are now computed automatically
within MessageBuilder, and the withSequence method has been removed.
bertfrees added a commit to daisy/pipeline-framework that referenced this issue Dec 13, 2017
@bertfrees
Support posting messages to EventBus through SLF4J
6bad04c 
without needing direct access to the EventBus. This happens via a
custom Logback appender.

This feature is intended mainly for channelling log messages from
external libraries to the execution log.

It also follows the proposal of unifying the two types of
messages (execution log and detailed log) through one and the same
mechanism, except that XProc/XSLT messages are still sent directly to
the EventBus: see issue daisy/pipeline#478.
bertfrees added a commit to daisy/pipeline-framework that referenced this issue Jan 8, 2018
@bertfrees
[framework] Mechanism for sending messages to execution log from Java
66ce899 
using a custom Logback appender that sents log messages to the
EventBus. Attaching the right job ID is done using the same MDC based
principle that was already used in IgnoreSiftAppender.

This fixes issue daisy/pipeline#477

Note that this also follows the proposal of unifying the two types of
messages (execution log and detailed log) through one and the same
mechanism, except that XProc/XSLT messages are still sent directly to
the EventBus: see issue
daisy/pipeline#478.

In order to support messages belonging to one job coming from several
sources, the message sequence numbers are now computed automatically
within MessageBuilder, and the withSequence method has been removed.
bertfrees added a commit to daisy/pipeline-framework that referenced this issue Jan 9, 2018
@bertfrees
[framework] Mechanism for sending messages to execution log from Java
1f27727 
using a custom Logback appender that sents log messages to the
EventBus. Attaching the right job ID is done using the same MDC based
principle that was already used in IgnoreSiftAppender.

This fixes issue daisy/pipeline#477

Note that this also follows the proposal of unifying the two types of
messages (execution log and detailed log) through one and the same
mechanism, except that XProc/XSLT messages are still sent directly to
the EventBus: see issue
daisy/pipeline#478.

In order to support messages belonging to one job coming from several
sources, the message sequence numbers are now computed automatically
within MessageBuilder, and the withSequence method has been removed.
@bertfrees bertfrees mentioned this issue Feb 5, 2018
Merged
bertfrees added a commit to daisy/pipeline-framework that referenced this issue Jul 5, 2018
@bertfrees
[framework] Mechanism for sending messages to execution log from Java
066269c 
using a custom Logback appender that sents log messages to the
EventBus. Attaching the right job ID is done using the same MDC based
principle that was already used in IgnoreSiftAppender.

This fixes issue daisy/pipeline#477

Note that this also follows the proposal of unifying the two types of
messages (execution log and detailed log) through one and the same
mechanism, except that XProc/XSLT messages are still sent directly to
the EventBus: see issue
daisy/pipeline#478.

In order to support messages belonging to one job coming from several
sources, the message sequence numbers are now computed automatically
within MessageBuilder, and the withSequence method has been removed.
bertfrees added a commit to daisy/pipeline-framework that referenced this issue Dec 21, 2018
@bertfrees
Support posting messages to EventBus through SLF4J
1c0c1fc 
without needing direct access to the EventBus. This happens via a
custom Logback appender.

This feature is intended mainly for channelling log messages from
external libraries to the execution log.

It also follows the proposal of unifying the two types of
messages (execution log and detailed log) through one and the same
mechanism, except that XProc/XSLT messages are still sent directly to
the EventBus: see issue daisy/pipeline#478.
@bertfrees bertfrees added this to the v1.12.0 milestone Apr 6, 2022
@bertfrees
Copy link
Member Author

Fixed by daisy/pipeline-framework#127

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question
Projects
None yet
Milestone
v1.12.0
Development

No branches or pull requests
1 participant
@bertfrees