Dev-crawler into dev

[Mammutor]

This is the newest state of the crawler, but there are still issues with the writing into the DB

[RobinGummels]

Seems fine, i guess... Maybe change the point with you .env

[RobinGummels]

Classical perfomance-error Problem in CI/CD-Pipeline. Probably because you are running the Crawler again, the responsetime for basic Querys is greater then 5 Seconds.

  ✕ query executes in reasonable time (<5s for small dataset) (7380 ms)

     Integration: Collection Search with Aggregated Fields › Performance and Cardinality › query executes in reasonable time (<5s for small dataset)

     expect(received).toBeLessThan(expected)

     Expected: < 5000
     Received:   7379

 317 |
 318 |       // Should complete within 5 seconds for typical test datasets
> 319 |       expect(duration).toBeLessThan(5000);
     |                        ^
 320 |     }, 10000); // 10s timeout for Jest
 321 |   });
 322 | });

 at Object.toBeLessThan (__tests__/buildCollectionSearchQuery.integration.test.js:319:24)

If you wan't to, i can just force the merge and skip tests.

[Mammutor]

Classical perfomance-error Problem in CI/CD-Pipeline. Probably because you are running the Crawler again, the responsetime for basic Querys is greater then 5 Seconds.

  ✕ query executes in reasonable time (<5s for small dataset) (7380 ms)

     Integration: Collection Search with Aggregated Fields › Performance and Cardinality › query executes in reasonable time (<5s for small dataset)

     expect(received).toBeLessThan(expected)

     Expected: < 5000
     Received:   7379

 317 |
 318 |       // Should complete within 5 seconds for typical test datasets
> 319 |       expect(duration).toBeLessThan(5000);
     |                        ^
 320 |     }, 10000); // 10s timeout for Jest
 321 |   });
 322 | });

 at Object.toBeLessThan (__tests__/buildCollectionSearchQuery.integration.test.js:319:24)

If you wan't to, i can just force the merge and skip tests.

I dont want to just push with no test. This is not a timely matter, so i would suggest that we just wait

[RobinGummels]

The tests are running. But exactly one test fails, which tracks the perfomance of the response-time. So i would say it would be fair to merge. But i'm also fine with not merging yet.