Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Profile code #19

Open
wants to merge 9 commits into
base: master
Choose a base branch
from
Open

Profile code #19

wants to merge 9 commits into from

Conversation

prateek-sumologic
Copy link
Collaborator

Have added the profiling code for the classes and methods of the api and the main file. The code profiles each line of each of the function and generates html (flamegraph profile) for each of the functions.

payload, state = self.transform_data(content)
#Todo Make this atomic if after sending -> Ctrl - C happens then it fails to save state
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why are we removing existing comments ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have restore the comments

self.log.info(f'''No results window LogType: {log_type} kwargs: {kwargs} status: {fetch_success}''')
is_move_fetch_window, new_state = self.check_move_fetch_window(kwargs)
if is_move_fetch_window:
self.save_state(**new_state)
self.log.debug(f'''Moving fetched window newstate: {new_state}''')
else:
self.log.error(f'''Error LogType: {log_type} status: {fetch_success} reason: {content}''')
except Exception as e:
Copy link
Collaborator

@himanshu219 himanshu219 May 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove try and except, the exceptions should be propagated to base class where these are caught.


msg['project_id'] = self.api_config['PROJECT_ID']
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

revert back this change, the older if conditions were there for different mongodb versions which have different log format.

@himanshu219 himanshu219 self-requested a review May 21, 2024 10:52
Copy link
Collaborator

@himanshu219 himanshu219 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use separate commits for fixing indentatins / cosmetic changes
and changes needed for improving memory footprint.

Also describe the changes done to improve the memory footprint and how much it improved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants