Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PAY-2860] SDK Services Config - Configure SDK using environment #8329

Merged
merged 16 commits into from
May 4, 2024

Conversation

rickyrombo
Copy link
Contributor

@rickyrombo rickyrombo commented May 3, 2024

Description

Fixes PAY-2860

Init SDK for a given environment using a simple environment key in config, eg:

const audiusSdk = sdk({ appName: 'ddex',  environment: 'staging' })
  • Moves "network/environment" related config into the generated config objects
  • Does away with the "bootstrap" idea, this is the config now
  • Revamps the service initialization to be much more clear
  • Changes defaultSomeServiceConfig constant to be getDefaultSomeServiceConfig() method that takes in the environment's service config

The SdkServiceConfig object has a key per network:

  • network is the Audius network, which contains all the nodes (eg. Discovery Nodes, Storage Nodes, Anti Abuse Oracle Nodes, and Identity Service) and the min service version
  • acdc is the key for the Audius chain, contains contracts and config used by Audius for ACDC
  • solana is the key for the Solana network, contains the programs and mints used by Audius on Solana
  • When we support eth contract interaction in SDK, that will go in an eth key.

Example SdkServicesConfig (dev) since it's hard to view in the diff:

/*
 * This file is autogenerated by ./scripts/generateServicesConfig.ts.
 * DO NOT EDIT MANUALLY!
 */
/* eslint-disable prettier/prettier */
import type { SdkServicesConfig } from './types'
export const developmentConfig: SdkServicesConfig = {
  "network": {
    "minVersion": "0.0.0",
    "discoveryNodes": [
      {
        "delegateOwnerWallet": "0xd09ba371c359f10f22ccda12fd26c598c7921bda3220c9942174562bc6a36fe8",
        "endpoint": "http://audius-protocol-discovery-provider-1",
        "ownerWallet": "0xd09ba371c359f10f22ccda12fd26c598c7921bda3220c9942174562bc6a36fe8"
      }
    ],
    "storageNodes": [
      {
        "delegateOwnerWallet": "0x0D38e653eC28bdea5A2296fD5940aaB2D0B8875c",
        "endpoint": "http://audius-protocol-creator-node-1"
      }
    ],
    "antiAbuseOracleNodes": {
      "endpoints": [
        "http://audius-protocol-anti-abuse-oracle-1:8000"
      ],
      "registeredAddresses": [
        "0xF0D5BC18421fa04D0a2A2ef540ba5A9f04014BE3"
      ]
    },
    "identityService": "https://audius-protocol-identity-service-1"
  },
  "acdc": {
    "entityManagerContractAddress": "0x254dffcd3277C0b1660F6d42EFbB754edaBAbC2B",
    "web3ProviderUrl": "http://audius-protocol-poa-ganache-1"
  },
  "solana": {
    "claimableTokensProgramAddress": "testHKV1B56fbvop4w6f2cTGEub9dRQ2Euta5VmqdX9",
    "rewardManagerProgramAddress": "testLsJKtyABc9UXJF8JWFKf1YH4LmqCWBC42c6akPb",
    "rewardManagerStateAddress": "DJPzVothq58SmkpRb1ATn5ddN2Rpv1j2TcGvM3XsHf1c",
    "rpcEndpoint": "http://audius-protocol-solana-test-validator-1",
    "usdcTokenMint": "26Q7gP8UfkDzi7GMFEQxTJaNJ8D2ybCUjex58M5MLu8y",
    "wAudioTokenMint": "37RCjhgV1qGV2Q54EHFScdxZ22ydRMdKMtVgod47fDP3"
  }
}

What getDefaultConfig looks like (DiscoveryNodeSelector):

import type { SdkServicesConfig } from '../../config/types'
import { Logger } from '../Logger'

import type { DiscoveryNodeSelectorServiceConfigInternal } from './types'

export const getDefaultDiscoveryNodeSelectorConfig = (
  config: SdkServicesConfig
): DiscoveryNodeSelectorServiceConfigInternal => ({
  initialSelectedNode: null,
  blocklist: null,
  allowlist: null,
  maxConcurrentRequests: 6,
  requestTimeout: 30000, // 30s
  unhealthyTTL: 3600000, // 1 hour
  backupsTTL: 120000, // 2 min
  healthCheckThresholds: {
    minVersion: config.network.minVersion,
    maxSlotDiffPlays: null,
    maxBlockDiff: 15
  },
  bootstrapServices: config.network.discoveryNodes,
  logger: new Logger()
})

Note that the service specific config (eg maxBlockDiff etc) is not in the root services config. It's not information about the environment.

The service constructor then looks like:

  constructor(config?: DiscoveryNodeSelectorServiceConfig) {
    this.config = mergeConfigWithDefaults(
      config,
      getDefaultDiscoveryNodeSelectorConfig(productionConfig)
    )
   // ...

And initialization in sdk.ts:

import { developmentConfig } from './config/development'
import { productionConfig } from './config/production'
import { stagingConfig } from './config/staging'

// ...

const initializeServices = (config: SdkConfig) => {
  const servicesConfig =
    config.environment === 'development'
      ? developmentConfig
      : config.environment === 'staging'
      ? stagingConfig
      : productionConfig

// ...

  const discoveryNodeSelector =
    config.services?.discoveryNodeSelector ??
    new DiscoveryNodeSelector({
      ...getDefaultDiscoveryNodeSelectorConfig(servicesConfig),
      logger
    })

//...

How Has This Been Tested?

TBD

Copy link

changeset-bot bot commented May 3, 2024

🦋 Changeset detected

Latest commit: f3d1dac

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@audius/sdk Major

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Copy link

gitguardian bot commented May 3, 2024

⚠️ GitGuardian has uncovered 14 secrets following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secrets in your pull request
GitGuardian id GitGuardian status Secret Commit Filename
1606950 Triggered Generic High Entropy Secret 87b840e packages/libs/src/sdk/scripts/generateServicesConfig.ts View secret
1606950 Triggered Generic High Entropy Secret 87b840e packages/libs/src/sdk/config/production.ts View secret
2416685 Triggered Generic High Entropy Secret 87b840e packages/libs/src/sdk/config/staging.ts View secret
2460750 Triggered Generic High Entropy Secret 87b840e packages/libs/src/sdk/config/production.ts View secret
4714545 Triggered Generic High Entropy Secret 87b840e packages/libs/src/sdk/config/development.ts View secret
4714545 Triggered Generic High Entropy Secret 87b840e packages/libs/src/sdk/scripts/generateServicesConfig.ts View secret
7150224 Triggered Generic High Entropy Secret 87b840e packages/libs/src/sdk/config/staging.ts View secret
7150224 Triggered Generic High Entropy Secret 87b840e packages/libs/src/sdk/scripts/generateServicesConfig.ts View secret
10622416 Triggered Generic High Entropy Secret 87b840e packages/libs/src/sdk/scripts/generateServicesConfig.ts View secret
10622416 Triggered Generic High Entropy Secret 87b840e packages/libs/src/sdk/scripts/generateServicesConfig.ts View secret
10622416 Triggered Generic High Entropy Secret 87b840e packages/libs/src/sdk/config/staging.ts View secret
10622416 Triggered Generic High Entropy Secret 87b840e packages/libs/src/sdk/config/production.ts View secret
10622417 Triggered Generic High Entropy Secret 87b840e packages/libs/src/sdk/scripts/generateServicesConfig.ts View secret
10622417 Triggered Generic High Entropy Secret 87b840e packages/libs/src/sdk/config/development.ts View secret
🛠 Guidelines to remediate hardcoded secrets
  1. Understand the implications of revoking this secret by investigating where it is used in your code.
  2. Replace and store your secrets safely. Learn here the best practices.
  3. Revoke and rotate these secrets.
  4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider


🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

@rickyrombo rickyrombo changed the title Mjp sdk config 2 Default configs by environment - One Config May 3, 2024
@rickyrombo rickyrombo changed the title Default configs by environment - One Config SDK Services Config - Configure SDK using environment May 3, 2024
@rickyrombo rickyrombo changed the title SDK Services Config - Configure SDK using environment [PAY-2860] SDK Services Config - Configure SDK using environment May 3, 2024
@rickyrombo rickyrombo marked this pull request as ready for review May 3, 2024 17:03
@rickyrombo rickyrombo requested review from sliptype and raymondjacobson and removed request for sliptype May 3, 2024 17:03
Copy link
Member

@raymondjacobson raymondjacobson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

structure looks great to me. @audius/erc stuff should be straightforward to add

Copy link
Contributor

@sliptype sliptype left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! We def want a changeset for this, but I believe it's just an additive change so minor version bump?

@rickyrombo
Copy link
Contributor Author

@sliptype i removed a barrel file for config but I can add it back to make it additive? want to talk about barrel files in sdk generally.. might be worth trying to eliminate those earlier rather than later

@sliptype
Copy link
Contributor

sliptype commented May 3, 2024

@sliptype i removed a barrel file for config but I can add it back to make it additive? want to talk about barrel files in sdk generally.. might be worth trying to eliminate those earlier rather than later

Oh right, well we already have an existing major bump that hasn't been released so we can mark this one as major too. It will only result in one version increase

And yeah, I'm all for removing barrel files

@audius-infra
Copy link
Collaborator

Preview this change https://demo.audius.co/mjp-sdk-config-2

@audius-infra
Copy link
Collaborator

Preview this change https://demo.audius.co/mjp-sdk-config-2

@audius-infra
Copy link
Collaborator

Preview this change https://demo.audius.co/mjp-sdk-config-2

@rickyrombo rickyrombo enabled auto-merge (squash) May 4, 2024 00:58
@rickyrombo rickyrombo merged commit f005b25 into main May 4, 2024
41 of 43 checks passed
@rickyrombo rickyrombo deleted the mjp-sdk-config-2 branch May 4, 2024 01:30
audius-infra pushed a commit that referenced this pull request May 4, 2024
[9f44712] Reapply "Convert swagger.json to OAS 3.0 before generating SDK types (#8274)" (#8324) (#8344) Marcus Pasell
[2ea74f3] Fix issues with text-link variant (#8343) Dylan Jeffers
[f005b25] [PAY-2860] SDK Services Config - Configure SDK using environment (#8329) Marcus Pasell
[11a0813] Disable add to playlist for prem tracks if not purchased (#8336) Reed
[33b1916] [C-4335] Migrate all user and collection cards (#8338) Dylan Jeffers
[dafaa21] [QA-1241] Fix hidden track add to playlist allowing non hidden items (#8340) JD Francis
[4c95031] [QA-956] Add deleted tile for tracks/albums/playlists (#8333) JD Francis
[b8f5990] Add error message to upload flow (#8334) JD Francis
[22894e8] [QA-957] Fix android overflowing text on access & sale modal (#8313) JD Francis
[86f6447] [PAY-2812] Change spend to earn challenge copy (#8332) Reed
[d4c4264] Close modal on done (#8331) Saliou Diallo
[ded4933] [PAY-2861] CollectionCard fixes (#8330) Dylan Jeffers
[36277cd] Revert "Convert swagger.json to OAS 3.0 before generating SDK types (#8274)" (#8324) Marcus Pasell
[146b2e0] [QA-1212] Fix chromecast (#8323) Raymond Jacobson
[0b33e68] [PAY-2738] Public album with prem track always shows buy button (#8321) Reed
[d897a98] [C-4333] Migrate follow-users to new sign-up flow (#8312) Dylan Jeffers
[3129a3a] [PAY-2799] Show preview button for collections (#8271) Reed
[8c58601] Modify /etc/hosts on CI machines before api test (#8319) Danny
[3586f41] [PAY-2859] Fix bugs in edit app flow (#8320) Raymond Jacobson
[1bf3e97] [PAY-2801] Purchased selectable filter in library albums (#8318) Reed
[6db35c6] [C-4285] Add claim all rewards button functionality (#8292) Saliou Diallo
[a20d85a] [PAY-2728] Implement edit app flow (#8256) Raymond Jacobson
[714a3d1] [C-4288] Migrate to new cards in profile, search, and library (#8300) Dylan Jeffers
[4dceb61] Upgrade dapp-store to 0.8.1 (#8311) Dylan Jeffers
[4d37233] [PAY-2794] Use UserLink for purchases/sales tables (#8306) Reed
[fc2aef3] [PAY-2772] Fix collection description text size (#8309) Reed
[c35bfe1] [PAY-2822] Convert LockedContentDetailsTile to Harmony (#8308) Reed
[f7a921e] [PAY-2781] Poll for track access on album purchase (#8296) Reed
[621c3ee] Convert swagger.json to OAS 3.0 before generating SDK types (#8274) Marcus Pasell
[09789c1] Retrigger logic for android notification permissions post-manifest fix (#8305) JD Francis
[b9c042a] [PAY-2815] Show unlocked icon if dm content is purchased (#8303) Andrew Mendelsohn
[722e933] [PAY-2779] Fix validation for albumTrackPrice (#8302) Andrew Mendelsohn
[963bec4] Fix EditCollection playwright test (#8304) Andrew Mendelsohn
[7e0aa7e] [PAY-2823] Edit Album access control fixes (#8298) Andrew Mendelsohn
[b42b6fd] [QA-971] Fix smart collections (#8299) Raymond Jacobson
[eceb876] [PAY-2791] Show only own albums in artist dashboard (#8297) Reed
[87ce027] [Web] [Harmony] Add accounts managing you modal [C-4295] [C-4292] (#8258) nicoback2
[7a3567c] [C-4238, C-4318] Add ssr small bundle flow for mobile track page (#8293) Kyle Shanks
[c7c3752] [PAY-2800] Fix mobile navigate to album on purchase success (#8295) Reed
[f113a91] [PAY-2802] Auto-favorite tracks in purchased album (#8294) Reed
[d32934f] [C-4284] Fix rewards in mobile web (#8291) Saliou Diallo
[45c4bc6] [PAY-2775] Remove suggested tracks for albums (#8288) Andrew Mendelsohn
[22280c6] [C-4080] Mobile cooldown and claim all (#8263) Isaac Solo
[80cca19] Jail broken test (#8289) Raymond Jacobson
[d7c8e34] [PAY-2766] Analytics events for premium albums (#8262) JD Francis
[c3ee4a8] [PAY-2817] Web collections show reposts, remove artist column (#8272) Reed
[6f8bab6] Fix lint items (#8285) Raymond Jacobson
[e8debc0] [PAY-2703] Redesign mobile collection and track screen tiles (#8255) Andrew Mendelsohn
[a3964a8] Bump android build number for release-client-v1.5.78 (#8286) Raymond Jacobson
[8481f8d] [C-4327] Fix track ordering within playlist/album (#8284) Raymond Jacobson
[5165172] Bump android version for manifest changes (#8283) JD Francis
[4e60f97] [C-4312] Add UserCard (#8276) Dylan Jeffers
[35ed4ec] [ONC-82] Fix android notif permissions (#8279) JD Francis
[82e1c67] [C-4323] Improve Avatar (#8273) Dylan Jeffers
[909db7c] [C-4316, C-4317] SSR work to render webplayer and track page for web (#8268) Kyle Shanks
[7417a40] Generate sdk after ursm code removal (#8275) Raymond Jacobson
[62b5a08] Update Add Funds copy and loading state (#8266) Saliou Diallo
[4503cc7] PROTO-1328: always default to discovery relay (#8247) alecsavvy
[f42c6cd] Fix text-input spacing (#8270) Dylan Jeffers
[75fd847] [C-4270] Fix text-link transition (#8260) Dylan Jeffers
[df0349a] [PAY-2761] Add managed users endpoints (#8238) Randy Schott
[5b4c998] [ONC-84] Add try/catch (#8264) Raymond Jacobson
[2b5f89c] Fix clipped text in TracksTable name column (#8265) Reed
[c3256fa] [C-4275] Add google-inspectiontool to crawler regex (#8261) Sebastian Klingler
[5750045] [C-4033] Prevent useTabs from redirecting on page load (#8259) Sebastian Klingler
[b17ff70] [PAY-2758] Allow gated tracks to be added to playlists (#8227) Reed
audius-infra pushed a commit that referenced this pull request May 6, 2024
[2eec73b] ddex: handle updates and deletes (#8327) Steve Perkins
[9f44712] Reapply "Convert swagger.json to OAS 3.0 before generating SDK types (#8274)" (#8324) (#8344) Marcus Pasell
[f005b25] [PAY-2860] SDK Services Config - Configure SDK using environment (#8329) Marcus Pasell
[f43790f] Add crm plugin (#8317) Raymond Jacobson
schottra added a commit that referenced this pull request May 6, 2024
…cher

* origin/main: (33 commits)
  Audius Protocol v0.6.96
  ddex: handle updates and deletes (#8327)
  Audius Client (Web and Mobile) v1.5.79
  Reapply "Convert swagger.json to OAS 3.0 before generating SDK types (#8274)" (#8324) (#8344)
  Fix issues with text-link variant (#8343)
  [PAY-2860] SDK Services Config - Configure SDK using environment (#8329)
  Disable add to playlist for prem tracks if not purchased (#8336)
  [C-4335] Migrate all user and collection cards (#8338)
  Fix crm CI (#8342)
  Add CI for crm plugin (#8335)
  [QA-1241] Fix hidden track add to playlist allowing non hidden items (#8340)
  [QA-956] Add deleted tile for tracks/albums/playlists (#8333)
  Add error message to upload flow (#8334)
  [PROTO-1706] fixups (#8337)
  Add crm plugin (#8317)
  [PROTO-1706] - Audius-d Node Documentation (#8325)
  [QA-957] Fix android overflowing text on access & sale modal (#8313)
  [PAY-2812] Change spend to earn challenge copy (#8332)
  Close modal on done (#8331)
  [PAY-2861] CollectionCard fixes (#8330)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants