chore: app for warehouse

[achettyiitr]

Description

• Introduce the warehouse app.
• Remove globals from the warehouse package.
• Added test cases.

Linear Ticket

• https://linear.app/rudderstack/issue/PIPE-218/introduce-warehouse-app

Security

• The code changed/added as part of this pull request won't create any security issues with how the software is being used.

[codecov]

Codecov Report

Attention: 49 lines in your changes are missing coverage. Please review.

Comparison is base (85adbbf) 68.96% compared to head (3a3721f) 69.19%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3862      +/-   ##
==========================================
+ Coverage   68.96%   69.19%   +0.23%     
==========================================
  Files         357      357              
  Lines       53082    53132      +50     
==========================================
+ Hits        36607    36764     +157     
+ Misses      14155    14051     -104     
+ Partials     2320     2317       -3     

Files	Coverage Δ
warehouse/admin.go	9.72% <100.00%> (+2.57%)	⬆️
warehouse/encoding/encoding.go	100.00% <100.00%> (ø)
warehouse/grpc.go	81.04% <100.00%> (+1.55%)	⬆️
warehouse/http.go	78.72% <100.00%> (+1.14%)	⬆️
warehouse/integrations/deltalake/deltalake.go	74.49% <100.00%> (-0.31%)	⬇️
warehouse/internal/loadfiles/loadfiles.go	72.95% <100.00%> (ø)
warehouse/internal/repo/schema.go	88.88% <100.00%> (ø)
warehouse/internal/repo/upload.go	88.95% <100.00%> (ø)
warehouse/mode.go	100.00% <100.00%> (ø)
warehouse/router_scheduling.go	95.20% <100.00%> (-0.04%)	⬇️
... and 15 more

... and 8 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[achettyiitr]

We can probably skip starting in case rudder-core is in degraded mode as we are already skipping the start for the warehouse service later in here.

[fracasula]

Great work so far @achettyiitr, it's looking good 👍

[achettyiitr]

It still has nothing to do with triggers though so maybe we can give it a generic name or remove it completely.
Thoughts?

Yeah, we can remove this and just pass the dependency for triggerStore: syncs.Map. But I don't want to expose Store, Delete, and Load to other modules. Currently, it hides the implementation details. Thoughts?

[achettyiitr]

It still has nothing to do with triggers though so maybe we can give it a generic name or remove it completely.
Thoughts?

Yeah, we can remove this and just pass the dependency for triggerStore: syncs.Map. But I don't want to expose Store, Delete, and Load to other modules. Currently, it hides the implementation details. Thoughts?

Thanks, @fracasula, for pointing it out regarding triggerStore. No need to have it, since it is just proxying to syncs.Map. Removing it.

[atzoum]

@achettyiitr this could be a good chance to fix deprecated usages of the config api in warehouse, e.g. config.RegisterXxxConfigVariable.

All other packages have been adapted except for reporting, event_schema (soon to be removed) and warehouse

[achettyiitr]

@achettyiitr this could be a good chance to fix deprecated usages of the config api in warehouse, e.g. config.RegisterXxxConfigVariable.

All other packages have been adapted except for reporting, event_schema (soon to be removed) and warehouse

Picking it up just after these changes get merged.

[fracasula]

Approving with the last batch of minor comments. Thanks @achettyiitr for the effort! This is good work 👍