Looking for issues to get started with contributing

[kikiscodedeliveryservice]

@regulartim

Hey, I'm Rohit (a backend developer with 1.5 years of experience in Django Python) and I'm looking to contribute to GreedyBear. The framework, models and endpoint structure are similar to what I've worked on before and I'm eager to translate those skills into contributions here.

I am a bit confused as to where to begin. I've looked at open, unassigned issues and identified something I would love to start working on: issue #1509. I know that The Honeynet Project participates in GSoC and this issue could just be something raised by @opbot-xd to self-assign later on/organise their work for GSoC. I wanted to ask if that was the case or if this was open for other contributors since the coding period for GSoC seems to be at an end for most projects.

Asking here because I didn't want to pollute the issue thread assuming @opbot-xd had their own plans for it. If not, I'd be happy to reply in that thread with my questions and alternative approaches for fulfilling the feature request.

If the issue is not open to others, I would really appreciate any guidance on first issues or any areas of improvement you want to target, so that I can look at them and potentially come up with my own issue. I have gone through the community and contribution guidelines as well as set up the dev environment locally, so I'm primed to start.

Thank you for your time and consideration!

[kikiscodedeliveryservice]

Bumping this — any word on whether #1509's up for grabs, @regulartim @opbot-xd?

[opbot-xd]

Hi @RohitPadala, first of all thanks for taking the time to contribute to GreedyBear!

Talking about #1509, its one of the future works which we can continue from my GSoC project. There is a blog also which can help you understand what has been done till now on the project. And yes according to my understanding and discussions with @mlodic in the PR thread, yes this issue is out of GSoC scope but However, let’s wait for confirmation from @regulartim and @mlodic as well.

Thanks again for your interest in contributing!

[kikiscodedeliveryservice]

Thank you so much for the context, @opbot-xd! I'm going through the blog post, it's good context on what's already built and it makes the changes more intuitive to understand. Will wait for @regulartim / @mlodic to confirm before starting anything.

[kikiscodedeliveryservice]

Hi @opbot-xd — really enjoyed the write-up, went through it in depth, side-by-side with the code. The work seems very thorough, deliberate and well-done.

I only had a minor query: PayloadExtractionJob._deduplicate treats any existing HoneypotPayload row with a matching hash as "already handled," whether it actually has a downloaded file or is just a metadata-only stub — which happens when an event reports a payload hash without the file itself attached. So if a real file for that hash later becomes available, it looks like it's already been handled and never gets downloaded. Was that a known tradeoff, or is it worth its own small follow-up issue?

One other small thing worth flagging: the blog describes hitting the quarantine cap as samples getting "tracked in the database by metadata only" and remaining retrievable later — but the actual loop just breaks with no metadata-only record created. And since /recent's time windows are strictly sequential based on EXTRACTION_INTERVAL, with no overlap or backfill logic, each payload's window only ever occurs once. Does a quota hit during that window mean the automated pipeline never picks it up again, or is there something else — like a scheduler timing, a manual recovery path — that I'm missing?

[opbot-xd]

Hi! Thank you so much for reading through the write-up and for taking the time to look so closely at the code. You’ve caught two excellent edge cases here, and you are completely right on both counts.

1. The Deduplication Edge Case
You are spot on. PayloadExtractionJob._deduplicate currently just checks for the existence of the sha256 hash in the database. If an event previously reported a payload hash and triggered _process_payload_hashes (creating a metadata-only stub with no payload_file attached), _deduplicate will view that hash as "already handled." When the actual file later becomes available for download, the job will skip it. This is a bug and wasn't an intentional tradeoff.

To fix this, _deduplicate should only exclude hashes that have a valid payload_file attached. Additionally, _download_and_store needs to be updated to get and update an existing stub rather than blindly attempting to create a new row (which would otherwise trigger an IntegrityError due to the unique constraint on sha256). I will open a follow-up issue for this.

2. The Quarantine Cap Behavior
Great catch here as well. The implementation drifted from the original design intention described in the blog post. Currently, when the cap is reached, it simply breaks out of the loop without creating any metadata-only records for the remaining files. And because the /recent endpoint strictly relies on sequential, non-overlapping time windows (start_ts to end_ts), any payloads that were skipped due to the quota limit are indeed skipped forever. There is no automated backfill or retry path.

The description in the blog post is inaccurate based on what actually shipped. I need to fix this by either ensuring we still save the metadata-only stubs before breaking the loop, or by introducing a backfill mechanism for payloads that were skipped. I'll create an issue for this as well.

Thank you again for the thorough review and for pointing these out! I really appreciate the feedback.

[regulartim]

Hi @kikiscodedeliveryservice ! Yes, you can work on #1509 if you want to, but I have to say that this is not a very beginner friendly issue. If you still want to work on it, could you please provide a concise description of your proposed solution and post it as a comment in #1509 ?

[kikiscodedeliveryservice]

Hi @regulartim, I have posted a response in issue #1509's thread. PTAL and thank you for your time!