Skip to content

Implement python logging for output/debugging instead of print(). #175

Open

Description

@alexlyttle

A possible enhancement to this package would be to use python logging for output and logging errors etc. instead of using print. Logging allows you to:

  • See when and where in the code a logging call has been made
  • Assign categories to messages which the user can configure what to output/not (great for debugging)
  • Can configure output from the module to be sent to specific files - great when running on a remote machine
  • Can configure messages to be sent straight to stdout and stderr (print doesn't do this immediately unless you run the script with the python -u option.

I am new to logging myself, but it appears to be a much better way of logging messages within a large python package than print() (which doesn't give much info about where it came from etc.)

Maybe someone with more knowledge on this could implement it?

Activity

  1. grd349 commented on Oct 30, 2019

    @grd349
    Collaborator

    Hi @alexlyttle,

    This is an excellent idea. let's have a chat about how we could manage this. In the mean time, if anyone wants this for themselves, just shout.

  2. changed the title [-]Enhancement: Implement python logging for output/debugging instead of print().[/-] [+]Implement python logging for output/debugging instead of print().[/+] on Nov 7, 2019
  3. nielsenmb commented on Nov 21, 2019

    @nielsenmb
    Collaborator

    Ideas:

    1. Use the built-in logging module
    2. silent logging at every function call? Set logging level to INFO
    3. time stamp at each log step
    4. PBjam version logged at beginning of run
    5. filename star_date_time.log
    6. verbose settings to set amount of messages to print statements (silent, brief, letsjam). silent for cluster runs, brief by default, letsjam for debugging a single star.
  4. nielsenmb commented on Feb 26, 2020

    @nielsenmb
    Collaborator

    This is a problem description for the PBjam Hackday:

    Problem: Currently PBjam just dumps all messages whether progress or error related to the terminal. For debugging purposes this is not the most optimal way of doing things. Furthermore, the error messages that PBjam produces may not always be very meaningful.

    Possible Solution: A good first step is to make a small class or module that just handles writing statements to a file. This can possibly involve the Python built-in modules warnings and logging.

    Fully integrating error handling and messaging at all points in PBjam is beyond the scope of a hack day, but maybe an intermediate step is to research and possibly set up a 'best practice' framework for ensuring that PBjam fails gracefully. This can then later be integrated more completely.

  5. grd349 commented on Jul 1, 2020

    @grd349
    Collaborator

    Of course we lost the hack day. This is still something that should be done. @nielsenmb @alexlyttle I invite your thoughts on how to proceed :)

  6. nielsenmb commented on Jul 21, 2020

    @nielsenmb
    Collaborator

    Just trying to list a few options that I can think of.

    1. One log for the whole run, stored in cwd. This might get cluttered.
    2. One log for the whole run, copied to all the star directories in that session. No log file in cwd. This seems neater to me.
    3. A log for each star stored in the star directories. With log file for just the session part in cwd.

    My idea is to print all logging levels to the file(s), function enter/exits, progress etc. So do we still want verbosity arguments in various functions? Don't think it'll be that difficult, but might be overkill?

  7. grd349 commented on Jul 21, 2020

    @grd349
    Collaborator

    We still want warnings sent to terminal no? We should have logs too but I think 1 log for the whole run should be a starting point. We can always iterate later.

  8. nielsenmb commented on Jul 21, 2020

    @nielsenmb
    Collaborator
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions