Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(deps): fix batch of missing production dependencies v2.0.0-rc.1 #3344

Closed
petermetz opened this issue Jun 21, 2024 · 0 comments · Fixed by #3345
Closed

fix(deps): fix batch of missing production dependencies v2.0.0-rc.1 #3344

petermetz opened this issue Jun 21, 2024 · 0 comments · Fixed by #3345
Assignees
Labels
bug Something isn't working dependencies Pull requests that update a dependency file good-first-issue Good for newcomers good-first-issue-400-expert Hacktoberfest Hacktoberfest participants are welcome to take a stab at issues marked with this label. P1 Priority 1: Highest
Milestone

Comments

@petermetz
Copy link
Contributor

High level scope for this issue

Make sure all the missing production dependencies are specified in the package.json as needed.

Description / Context

It turns out that the missing dependency check that we have in place does not work as well as it should because some edge cases are not covered.

Quoting the work that was done already:

https://github.com/petermetz/cacti/tree/ci-fix-check-missing-node-deps =>

  1. The depcheck tool we use have not correctly discovered some of the
    missing dependencies that we have because it only verifies that the
    imported dependency is present SOMEwhere in the package.json file, not that
    it is specifically present in the production dependencies section which
    leads to crashes and broken packages due to the API server not installing
    dev dependencies when instantiating a plugin and therefore missing a few
    of the dependencies that are otherwise very much needed at runtime in
    production.
  2. The solution to the problem was to implement our own typescript parsing
    with babel and then double check the work of depcheck to make sure that
    the dependencies that it marks as "no issues" are actually OK and have no
    issues.
  3. The hardest edge case was type imports e.g. import type { Express } from "express";
    because the import was there, but we did not actually need that dependency
    in the production dependencies as long as ALL of the imports to it in the
    given package were type imports. To robustly verify this being the case or not
    we had to pull out the big guns and parse all the typescript code per package
    to make sure that we've looked at every single import of the dependency in
    question at every single code file of the package in question.
[tools/check-missing-node-deps.ts] ERROR - MISSING production dependency uuid from ./packages/cactus-test-verifier-client/package.json. Found usage in 
	./packages/cactus-test-verifier-client/src/main/typescript/verifier-with-go-eth-stress-check.ts


[tools/check-missing-node-deps.ts] ERROR - MISSING production dependency express from ./extensions/cactus-plugin-htlc-coordinator-besu/package.json. Found usage in 
	./extensions/cactus-plugin-htlc-coordinator-besu/src/main/typescript/plugin-htlc-coordinator-besu.ts
	./extensions/cactus-plugin-htlc-coordinator-besu/src/main/typescript/web-services/counterparty-htlc-endpoint.ts
	./extensions/cactus-plugin-htlc-coordinator-besu/src/main/typescript/web-services/own-htlc-endpoint.ts


[tools/check-missing-node-deps.ts] ERROR - MISSING production dependency @aries-framework/askar from ./examples/cactus-example-discounted-asset-trade-client/package.json. Found usage in 
	./examples/cactus-example-discounted-asset-trade-client/src/main/typescript/lib/agent-setup.ts


[tools/check-missing-node-deps.ts] ERROR - MISSING production dependency @aries-framework/core from ./examples/cactus-example-discounted-asset-trade-client/package.json. Found usage in 
	./examples/cactus-example-discounted-asset-trade-client/src/main/typescript/lib/agent-setup.ts
	./examples/cactus-example-discounted-asset-trade-client/src/main/typescript/lib/connections.ts
	./examples/cactus-example-discounted-asset-trade-client/src/main/typescript/lib/credentials.ts
	./examples/cactus-example-discounted-asset-trade-client/src/main/typescript/lib/listeners.ts
	./examples/cactus-example-discounted-asset-trade-client/src/main/typescript/lib/proofs.ts


[tools/check-missing-node-deps.ts] ERROR - MISSING production dependency @aries-framework/node from ./examples/cactus-example-discounted-asset-trade-client/package.json. Found usage in 
	./examples/cactus-example-discounted-asset-trade-client/src/main/typescript/lib/agent-setup.ts


[tools/check-missing-node-deps.ts] ERROR - MISSING production dependency @aries-framework/indy-vdr from ./examples/cactus-example-discounted-asset-trade-client/package.json. Found usage in 
	./examples/cactus-example-discounted-asset-trade-client/src/main/typescript/lib/agent-setup.ts


[tools/check-missing-node-deps.ts] ERROR - MISSING production dependency @aries-framework/anoncreds from ./examples/cactus-example-discounted-asset-trade-client/package.json. Found usage in 
	./examples/cactus-example-discounted-asset-trade-client/src/main/typescript/lib/agent-setup.ts


[tools/check-missing-node-deps.ts] ERROR - MISSING production dependency @aries-framework/anoncreds-rs from ./examples/cactus-example-discounted-asset-trade-client/package.json. Found usage in 
	./examples/cactus-example-discounted-asset-trade-client/src/main/typescript/lib/agent-setup.ts


[tools/check-missing-node-deps.ts] ERROR - MISSING production dependency express from ./packages/cactus-plugin-keychain-azure-kv/package.json. Found usage in 
	./packages/cactus-plugin-keychain-azure-kv/src/main/typescript/web-services/delete-keychain-entry-endpoint.ts
	./packages/cactus-plugin-keychain-azure-kv/src/main/typescript/web-services/get-keychain-entry-endpoint.ts
	./packages/cactus-plugin-keychain-azure-kv/src/main/typescript/web-services/has-keychain-entry-endpoint.ts
	./packages/cactus-plugin-keychain-azure-kv/src/main/typescript/web-services/set-keychain-entry-endpoint.ts


[tools/check-missing-node-deps.ts] ERROR - MISSING production dependency socket.io from ./packages/cacti-plugin-ledger-connector-stellar/package.json. Found usage in 
	./packages/cacti-plugin-ledger-connector-stellar/src/main/typescript/plugin-ledger-connector-stellar.ts
	./packages/cacti-plugin-ledger-connector-stellar/src/main/typescript/web-services/watch-blocks-v1-endpoint.ts


[tools/check-missing-node-deps.ts] ERROR - MISSING production dependency express from ./packages/cactus-plugin-keychain-aws-sm/package.json. Found usage in 
	./packages/cactus-plugin-keychain-aws-sm/src/main/typescript/webservices/get-prometheus-exporter-metrics-endpoint-v1.ts


[tools/check-missing-node-deps.ts] ERROR - MISSING production dependency web3-eth from ./packages/cactus-plugin-ledger-connector-xdai/package.json. Found usage in 
	./packages/cactus-plugin-ledger-connector-xdai/src/main/typescript/plugin-ledger-connector-xdai.ts


[tools/check-missing-node-deps.ts] ERROR - MISSING production dependency socket.io from ./packages/cactus-plugin-ledger-connector-aries/package.json. Found usage in 
	./packages/cactus-plugin-ledger-connector-aries/src/main/typescript/plugin-ledger-connector-aries.ts
	./packages/cactus-plugin-ledger-connector-aries/src/main/typescript/web-services/watch-connection-state-v1-endpoint.ts
	./packages/cactus-plugin-ledger-connector-aries/src/main/typescript/web-services/watch-proof-state-v1-endpoint.ts


[tools/check-missing-node-deps.ts] ERROR - MISSING production dependency express from ./packages/cactus-plugin-keychain-vault/package.json. Found usage in 
	./packages/cactus-plugin-keychain-vault/src/main/typescript/plugin-keychain-vault-remote-adapter.ts
	./packages/cactus-plugin-keychain-vault/src/main/typescript/web-services/get-prometheus-exporter-metrics-endpoint-v1.ts


[tools/check-missing-node-deps.ts] ERROR - MISSING production dependency axios from ./packages/cactus-plugin-ledger-connector-polkadot/package.json. Found usage in 
	./packages/cactus-plugin-ledger-connector-polkadot/src/main/typescript/generated/openapi/typescript-axios/api.ts
	./packages/cactus-plugin-ledger-connector-polkadot/src/main/typescript/generated/openapi/typescript-axios/base.ts
	./packages/cactus-plugin-ledger-connector-polkadot/src/main/typescript/generated/openapi/typescript-axios/common.ts


[tools/check-missing-node-deps.ts] ERROR - MISSING production dependency socket.io from ./packages/cactus-plugin-ledger-connector-ethereum/package.json. Found usage in 
	./packages/cactus-plugin-ledger-connector-ethereum/src/main/typescript/plugin-ledger-connector-ethereum.ts
	./packages/cactus-plugin-ledger-connector-ethereum/src/main/typescript/web-services/watch-blocks-v1-endpoint.ts


[tools/check-missing-node-deps.ts] ERROR - MISSING production dependency ethers from ./packages/cactus-plugin-ledger-connector-ethereum/package.json. Found usage in 
	./packages/cactus-plugin-ledger-connector-ethereum/src/main/typescript/plugin-ledger-connector-ethereum.ts


[tools/check-missing-node-deps.ts] ERROR - MISSING production dependency socket.io from ./packages/cactus-plugin-ledger-connector-quorum/package.json. Found usage in 
	./packages/cactus-plugin-ledger-connector-quorum/src/main/typescript/plugin-ledger-connector-quorum.ts
	./packages/cactus-plugin-ledger-connector-quorum/src/main/typescript/web-services/watch-blocks-v1-endpoint.ts


[tools/check-missing-node-deps.ts] ERROR - MISSING production dependency web3-eth from ./packages/cactus-plugin-ledger-connector-quorum/package.json. Found usage in 
	./packages/cactus-plugin-ledger-connector-quorum/src/main/typescript/plugin-ledger-connector-quorum.ts


[tools/check-missing-node-deps.ts] ERROR - MISSING production dependency ethers from ./packages/cactus-plugin-ledger-connector-quorum/package.json. Found usage in 
	./packages/cactus-plugin-ledger-connector-quorum/src/main/typescript/plugin-ledger-connector-quorum.ts


[tools/check-missing-node-deps.ts] ERROR - MISSING production dependency @bufbuild/protobuf from ./packages/cactus-cmd-api-server/package.json. Found usage in 
	./packages/cactus-cmd-api-server/src/main/typescript/api-server.ts


[tools/check-missing-node-deps.ts] ERROR - MISSING production dependency @bufbuild/protobuf from ./packages/cactus-plugin-keychain-memory/package.json. Found usage in 
	./packages/cactus-plugin-keychain-memory/src/main/typescript/plugin-keychain-memory.ts
	./packages/cactus-plugin-keychain-memory/src/main/typescript/generated/crpc/services/default_service_connect.ts
	./packages/cactus-plugin-keychain-memory/src/main/typescript/generated/crpc/services/default_service_pb.ts
	./packages/cactus-plugin-keychain-memory/src/main/typescript/generated/crpc/models/delete_keychain_entry_request_v1_pb_pb.ts
	./packages/cactus-plugin-keychain-memory/src/main/typescript/generated/crpc/models/delete_keychain_entry_response_v1_pb_pb.ts
	./packages/cactus-plugin-keychain-memory/src/main/typescript/generated/crpc/models/get_keychain_entry_request_v1_pb_pb.ts
	./packages/cactus-plugin-keychain-memory/src/main/typescript/generated/crpc/models/get_keychain_entry_response_v1_pb_pb.ts
	./packages/cactus-plugin-keychain-memory/src/main/typescript/generated/crpc/models/has_keychain_entry_request_v1_pb_pb.ts
	./packages/cactus-plugin-keychain-memory/src/main/typescript/generated/crpc/models/has_keychain_entry_response_v1_pb_pb.ts
	./packages/cactus-plugin-keychain-memory/src/main/typescript/generated/crpc/models/set_keychain_entry_request_v1_pb_pb.ts
	./packages/cactus-plugin-keychain-memory/src/main/typescript/generated/crpc/models/set_keychain_entry_response_v1_pb_pb.ts
	./packages/cactus-plugin-keychain-memory/src/main/typescript/crpc-services/keychain-memory-crpc-svc-openapi.ts


[tools/check-missing-node-deps.ts] ERROR - MISSING production dependency @connectrpc/connect from ./packages/cactus-plugin-keychain-memory/package.json. Found usage in 
	./packages/cactus-plugin-keychain-memory/src/main/typescript/crpc-services/keychain-memory-crpc-svc-openapi.ts


[tools/check-missing-node-deps.ts] ERROR - MISSING production dependency socket.io from ./packages/cactus-plugin-ledger-connector-fabric/package.json. Found usage in 
	./packages/cactus-plugin-ledger-connector-fabric/src/main/typescript/plugin-ledger-connector-fabric.ts
	./packages/cactus-plugin-ledger-connector-fabric/src/main/typescript/watch-blocks/watch-blocks-v1-endpoint.ts


[tools/check-missing-node-deps.ts] ERROR - MISSING production dependency @hyperledger/cactus-test-tooling from ./packages/cactus-plugin-ledger-connector-fabric/package.json. Found usage in 
	./packages/cactus-plugin-ledger-connector-fabric/src/main/typescript/deploy-contract-go-source/deploy-contract-go-source-impl-fabric-v2-5-6.ts


[tools/check-missing-node-deps.ts] ERROR - MISSING production dependency express from ./packages/cactus-plugin-ledger-connector-corda/package.json. Found usage in 
	./packages/cactus-plugin-ledger-connector-corda/src/main/typescript/plugin-ledger-connector-corda.ts
	./packages/cactus-plugin-ledger-connector-corda/src/main/typescript/web-services/deploy-contract-jars-endpoint.ts
	./packages/cactus-plugin-ledger-connector-corda/src/main/typescript/web-services/diagnose-node-endpoint-v1.ts
	./packages/cactus-plugin-ledger-connector-corda/src/main/typescript/web-services/get-prometheus-exporter-metrics-endpoint-v1.ts
	./packages/cactus-plugin-ledger-connector-corda/src/main/typescript/web-services/invoke-contract-endpoint-v1.ts
	./packages/cactus-plugin-ledger-connector-corda/src/main/typescript/web-services/list-flows-endpoint-v1.ts
	./packages/cactus-plugin-ledger-connector-corda/src/main/typescript/web-services/network-map-endpoint-v1.ts


[tools/check-missing-node-deps.ts] ERROR - MISSING production dependency @grpc/grpc-js from ./packages/cactus-core-api/package.json. Found usage in 
	./packages/cactus-core-api/src/main/typescript/plugin/grpc-service/i-plugin-grpc-service.ts
	./packages/cactus-core-api/src/main/typescript/generated/proto/protoc-gen-ts/weaver/common/protos/relay/datatransfer.ts
	./packages/cactus-core-api/src/main/typescript/generated/proto/protoc-gen-ts/weaver/common/protos/networks/networks.ts
	./packages/cactus-core-api/src/main/typescript/generated/proto/protoc-gen-ts/weaver/common/protos/driver/driver.ts


[tools/check-missing-node-deps.ts] ERROR - MISSING production dependency google-protobuf from ./packages/cactus-core-api/package.json. Found usage in 
	./packages/cactus-core-api/src/main/typescript/generated/proto/protoc-gen-ts/weaver/common/protos/relay/datatransfer_pb.d.ts
	./packages/cactus-core-api/src/main/typescript/generated/proto/protoc-gen-ts/weaver/common/protos/networks/networks.ts
	./packages/cactus-core-api/src/main/typescript/generated/proto/protoc-gen-ts/weaver/common/protos/networks/networks_pb.d.ts
	./packages/cactus-core-api/src/main/typescript/generated/proto/protoc-gen-ts/weaver/common/protos/driver/driver_pb.d.ts
	./packages/cactus-core-api/src/main/typescript/generated/proto/protoc-gen-ts/weaver/common/protos/corda/view_data.ts
	./packages/cactus-core-api/src/main/typescript/generated/proto/protoc-gen-ts/weaver/common/protos/corda/view_data_pb.d.ts
	./packages/cactus-core-api/src/main/typescript/generated/proto/protoc-gen-ts/weaver/common/protos/common/access_control.ts
	./packages/cactus-core-api/src/main/typescript/generated/proto/protoc-gen-ts/weaver/common/protos/common/access_control_pb.d.ts
	./packages/cactus-core-api/src/main/typescript/generated/proto/protoc-gen-ts/weaver/common/protos/common/ack.ts
	./packages/cactus-core-api/src/main/typescript/generated/proto/protoc-gen-ts/weaver/common/protos/common/ack_pb.d.ts
	./packages/cactus-core-api/src/main/typescript/generated/proto/protoc-gen-ts/weaver/common/protos/common/asset_locks.ts
	./packages/cactus-core-api/src/main/typescript/generated/proto/protoc-gen-ts/weaver/common/protos/common/asset_locks_pb.d.ts
	./packages/cactus-core-api/src/main/typescript/generated/proto/protoc-gen-ts/weaver/common/protos/common/asset_transfer.ts
	./packages/cactus-core-api/src/main/typescript/generated/proto/protoc-gen-ts/weaver/common/protos/common/asset_transfer_pb.d.ts
	./packages/cactus-core-api/src/main/typescript/generated/proto/protoc-gen-ts/weaver/common/protos/common/interop_payload.ts
	./packages/cactus-core-api/src/main/typescript/generated/proto/protoc-gen-ts/weaver/common/protos/common/interop_payload_pb.d.ts
	./packages/cactus-core-api/src/main/typescript/generated/proto/protoc-gen-ts/weaver/common/protos/common/membership.ts
	./packages/cactus-core-api/src/main/typescript/generated/proto/protoc-gen-ts/weaver/common/protos/common/membership_pb.d.ts
	./packages/cactus-core-api/src/main/typescript/generated/proto/protoc-gen-ts/weaver/common/protos/common/proofs.ts
	./packages/cactus-core-api/src/main/typescript/generated/proto/protoc-gen-ts/weaver/common/protos/common/proofs_pb.d.ts
	./packages/cactus-core-api/src/main/typescript/generated/proto/protoc-gen-ts/weaver/common/protos/common/query.ts
	./packages/cactus-core-api/src/main/typescript/generated/proto/protoc-gen-ts/weaver/common/protos/common/query_pb.d.ts
	./packages/cactus-core-api/src/main/typescript/generated/proto/protoc-gen-ts/weaver/common/protos/common/state.ts
	./packages/cactus-core-api/src/main/typescript/generated/proto/protoc-gen-ts/weaver/common/protos/common/state_pb.d.ts
	./packages/cactus-core-api/src/main/typescript/generated/proto/protoc-gen-ts/weaver/common/protos/common/verification_policy.ts
	./packages/cactus-core-api/src/main/typescript/generated/proto/protoc-gen-ts/weaver/common/protos/common/verification_policy_pb.d.ts


[tools/check-missing-node-deps.ts] ERROR - MISSING production dependency socket.io from ./packages/cactus-plugin-ledger-connector-besu/package.json. Found usage in 
	./packages/cactus-plugin-ledger-connector-besu/src/main/typescript/plugin-ledger-connector-besu.ts
	./packages/cactus-plugin-ledger-connector-besu/src/main/typescript/web-services/watch-blocks-v1-endpoint.ts

(add more description and relevant links)

Acceptance Criteria

  1. All the errors have been either addressed or marked as a false positive.
  2. The false positives have been documented as such so that we can fine-tune the new code of the CI that checks for missing dependencies
@petermetz petermetz added bug Something isn't working good-first-issue Good for newcomers dependencies Pull requests that update a dependency file Hacktoberfest Hacktoberfest participants are welcome to take a stab at issues marked with this label. good-first-issue-400-expert P1 Priority 1: Highest labels Jun 21, 2024
@petermetz petermetz added this to the v2.0.0 milestone Jun 21, 2024
@petermetz petermetz self-assigned this Jun 21, 2024
@petermetz petermetz changed the title build(deps): fix batch of missing production dependencies v2.0.0-rc.1 fix(deps): fix batch of missing production dependencies v2.0.0-rc.1 Jun 21, 2024
petermetz added a commit to petermetz/cacti that referenced this issue Jun 21, 2024
Huge diff, simple changes:
1. Where applicable, I added `import type {..}` instead of `import {...}`
so that we do not need the dependency in the production dependencies.
2. For imports where the code imported was actually used at runtime I
altered the package.json file so that the dependency is marked as a production
dependency and therefore won't crash anymore when being imported in isolation
(e.g. outside of the mono-repo dev build).

Fixes hyperledger-cacti#3344

Signed-off-by: Peter Somogyvari <peter.somogyvari@accenture.com>
petermetz added a commit to petermetz/cacti that referenced this issue Jun 24, 2024
Huge diff, simple changes:
1. Where applicable, I added `import type {..}` instead of `import {...}`
so that we do not need the dependency in the production dependencies.
2. For imports where the code imported was actually used at runtime I
altered the package.json file so that the dependency is marked as a production
dependency and therefore won't crash anymore when being imported in isolation
(e.g. outside of the mono-repo dev build).

Fixes hyperledger-cacti#3344

Signed-off-by: Peter Somogyvari <peter.somogyvari@accenture.com>
sandeepnRES pushed a commit to sandeepnRES/cacti that referenced this issue Jul 30, 2024
Huge diff, simple changes:
1. Where applicable, I added `import type {..}` instead of `import {...}`
so that we do not need the dependency in the production dependencies.
2. For imports where the code imported was actually used at runtime I
altered the package.json file so that the dependency is marked as a production
dependency and therefore won't crash anymore when being imported in isolation
(e.g. outside of the mono-repo dev build).

Fixes hyperledger-cacti#3344

Signed-off-by: Peter Somogyvari <peter.somogyvari@accenture.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working dependencies Pull requests that update a dependency file good-first-issue Good for newcomers good-first-issue-400-expert Hacktoberfest Hacktoberfest participants are welcome to take a stab at issues marked with this label. P1 Priority 1: Highest
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant