Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Properly filter Relationships to assemble a Feed View #1421

Merged
merged 1 commit into from
Apr 8, 2024

Conversation

gbuisson
Copy link
Contributor

@gbuisson gbuisson commented Apr 8, 2024

Epic #
Close #
Related https://github.com/advthreat/iroh/issues/9111

Here are the steps to render a Feed in CTIA:

  1. Fetch the feed document
  2. Fetch the referenced indicator
  3. Fetch all the relationships of the indicator
  4. Fetch all the objects referenced by the source_ref field of the relationships
  5. extract the observables from the Judgements and render the response

The query on step 3 is too naive as it attempts to list any kind of relationship even though we're looking only for indicators with an "element-of" relationship_type, some indicators are linked to a very large number of Sightings, retrieving them consumes a huge amount of resources and ultimately leads to a failure.

This PR updates the filter so we only fetch the proper indicators relevant for a Feed.

§ QA

Describe the steps to test your PR.

  1. Create and Query a Feed
  2. The functionality should still work as expected and there should be no regression

§ Release Notes

public: Feed performance improvements

Copy link
Contributor

@msprunck msprunck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@gbuisson gbuisson merged commit 52e19a4 into master Apr 8, 2024
14 checks passed
johnejardine pushed a commit that referenced this pull request Apr 8, 2024
<!-- Specify linked issues and REMOVE THE UNUSED LINES -->

> **Epic** #
> Close #
> Related advthreat/iroh#9111

Here are the steps to render a Feed in CTIA:

1. Fetch the feed document
2. Fetch the referenced indicator
3. Fetch all the relationships of the indicator
4. Fetch all the objects referenced by the `source_ref` field of the relationships
5. extract the observables from the Judgements and render the response  

The query on step 3 is too naive as it attempts to list any kind of relationship even though we're looking only for indicators with an "element-of" relationship_type, some indicators are linked to a very large number of Sightings, retrieving them consumes a huge amount of resources and ultimately leads to a failure.

This PR updates the filter so we only fetch the proper indicators relevant for a Feed.

<!-- UNCOMMENT THIS SECTION IF NEEDED
<a name="iroh-services-clients">[§](#iroh-services-clients)</a> IROH Services Clients
=====================================================================================

Put all informations that need to be communicated to IROH Services Clients.
Typically IROH-UI, ATS Integration, Orbital, etc...
 -->

<a name="qa">[§](#qa)</a> QA
============================


Describe the steps to test your PR.

1. Create and Query a Feed
2. The functionality should still work as expected and there should be no regression


<!-- UNCOMMENT THIS SECTION IF NEEDED
<a name="ops">[§](#ops)</a> Ops
===============================

  Specify Ops related issues and documentation
- Config change needed: threatgrid/tenzin#
- Migration needed: threatgrid/tenzin#
- How to enable/disable that feature: (ex remove service from `bootstrap.cfg`, add scope to org)
-->

<!-- UNCOMMENT THIS SECTION IF NEEDED
<a name="documentation">[§](#documentation)</a> Documentation
=============================================================

  Public Facing documentation section;
- Public documentation updated needed: threatgrid/iroh-ui#
  See internal [doc file](./services/iroh-auth/doc/public-doc.org)
 -->

<a name="release-notes">[§](#release-notes)</a> Release Notes
=============================================================

<!-- REMOVE UNUSED LINES -->

```
public: Feed performance improvements
```
@NeelSaini
Copy link

Verified on TEST in rel v2.17.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants