Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(odsp-client): Add odsp-client as experimental package #18199

Merged

Conversation

sonalideshpandemsft
Copy link
Contributor

@sonalideshpandemsft sonalideshpandemsft commented Nov 7, 2023

Description

This PR introduces a new client that provides a set of common utility functions to be used whenever developers want to create and load Fluid containers backed by files stored on ODSP. The scope of the current package is marked fluid-experimental and the apis are marked with the alpha release tag.

Reviewer Guidance

Would appreciate feedback on:

  • API names
  • Coding patterns
  • Implementation details
  • Return values of functions

Other information or known dependencies

MSFT Internal Design Doc

AB#6224

@github-actions github-actions bot added area: dev experience Improving the experience of devs building on top of fluid dependencies Pull requests that update a dependency file public api change Changes to a public API base: main PRs targeted against main branch labels Nov 7, 2023
@github-actions github-actions bot removed the area: dev experience Improving the experience of devs building on top of fluid label Nov 7, 2023
@sonalideshpandemsft sonalideshpandemsft changed the title draft: odsp-client (odsp-client): Add odsp-client as experimental package Nov 8, 2023
OdspMember,
} from "./interfaces";
export { OdspClient } from "./OdspClient";
export { OdspAudience } from "./OdspAudience";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to export the class, or is just the interface sufficient? Do external users need to construct these?

*/
export interface OdspConnectionConfig {
/**
* Site url representing ODSP resource location
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there more concrete docs we can point to here so users know how to provide this?

import { ITokenProvider } from '@fluidframework/azure-client';
import { ServiceAudience } from '@fluidframework/fluid-static';

// @alpha (undocumented)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this is still a draft, but ideally any of the items currently marked as undocumented should be documented before we merge. More details on these items may also help reviewers evaluate the API surface.

@sonalideshpandemsft sonalideshpandemsft force-pushed the alpha-m365-client branch 2 times, most recently from 387fc76 to 02e5de7 Compare November 9, 2023 00:35
@sonalideshpandemsft sonalideshpandemsft force-pushed the alpha-m365-client branch 3 times, most recently from d5641a8 to c825326 Compare November 9, 2023 21:15
Copy link
Contributor

@ChumpChief ChumpChief left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good for an initial checkin I think. I'm sure there will be more iteration and feedback but definitely a great starting point.

layerInfo.json Outdated
"Runtime",
"Routerlicious-Driver",
"Server-Shared-Utils",
"AzureClient"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ultimately OdspClient probably shouldn't depend on AzureClient (maybe the token stuff moves somewhere else), but this is probably fine for now.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

adding to doc feedback section. will iterate on this in upcoming PRs

project: ["./tsconfig.json"],
},
rules: {
"@typescript-eslint/strict-boolean-expressions": "off",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be great to get this reenabled, but not prereq for initial checkin if it's too much work.

packages/service-client/odsp-client/README.md Outdated Show resolved Hide resolved
packages/service-client/odsp-client/README.md Outdated Show resolved Hide resolved
packages/service-client/odsp-client/README.md Outdated Show resolved Hide resolved
}

public async getContainer(
sharingUrl: string,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can start here, but would be nice if we can make this just an "id" rather than a full URL (this is what AzureClient does, and it's a lot friendlier to hide URL formats away from the customer).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've left comments in document in PR about it:

  1. I'm not sure it has to be sharing link (and types of sharing links we support). It has perf impact.
  2. I do not understand how it all works if URL points to different drive ID or different siteURI / path that are specified when OdspClient is created. Do we ignore them? What driveID is returned on different APIs - one that was supplied to OdspClient, or one that identifies a file?
  3. I think we are confusing users about exposing too many concepts, like URI, containerID, itemID. I think we should expose one, and provide static functions to get all the other data.
  4. I assume we will not allow users to provide their own URI resolvers, and leave them to do conversion of their app API -> our ID/path outside of these interactions (but it would be worth confirming and describing it somewhere)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regarding (2):
I do not understand how it all works if URL points to different drive ID or different siteURI / path that are specified when OdspClient is created. Do we ignore them? - I'll look into this more.

What driveID is returned on different APIs - one that was supplied to OdspClient, or one that identifies a file? - Does this mean there will be two separate drive ids? Why would there be a different driveId to identify a file?
You can only use one driveId for any given Fluid Container. So, if you need to use more, you can create additional odsp-clients. But the Drive ID will be different for each tenant.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ChumpChief @vladsud I checked the way azure client is doing this. It seems like that id is not enough there too. The azure endpoint is taken in constructor and then using this id we make some modifications to that endpoint and append this id to reach to the request url which is then used to load the container. I am not sure I like that a lot because that is kind of functionality provided by resolver and its utility apis. Similarly, ODSP also has endpoint encoding all this info but imo it is not a sound way to arrive at the request url.

I don't see any harm in taking the url when app can make use of the utility api to construct it and just pass it to this function. And it does not necessarily be a sharing link, getAbsoluteUrl in OdspDriverUrlResoover does not give back a asharing link, but it is still enough to load the container as the absolute url is resolvable by the same resolver.

packages/service-client/odsp-client/src/index.ts Outdated Show resolved Hide resolved
packages/service-client/odsp-client/src/interfaces.ts Outdated Show resolved Hide resolved
packages/service-client/odsp-client/src/odspClient.ts Outdated Show resolved Hide resolved
const getAttributes = async (): Promise<OdspServiceAttributes> => {
const resolvedUrl = container.resolvedUrl as IOdspResolvedUrl;
if (resolvedUrl === undefined) {
throw new Error("Resolved Url not available on attached container");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, we always expect these to be defined after attach. This approach is fine to start with, but it would probably make sense for us to instead await attachment, if the container is not yet attached (rather than reject the promise).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

adding this to doc feedback section

Copy link
Contributor

@scottn12 scottn12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work Sonali! Excited to see this progress 🙂

packages/service-client/odsp-client/README.md Outdated Show resolved Hide resolved
packages/service-client/odsp-client/README.md Outdated Show resolved Hide resolved
packages/service-client/odsp-client/README.md Outdated Show resolved Hide resolved
packages/service-client/odsp-client/README.md Outdated Show resolved Hide resolved
packages/service-client/odsp-client/.gitignore Outdated Show resolved Hide resolved
@@ -157,6 +157,10 @@
"Server-Shared-Utils"
]
},
"OdspClient": {
"dirs": ["packages/service-client/odsp-client"],
"deps": ["AzureClient"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does odsp-client need to depend on azure packages? Do we need a new Service-Clients layer for shared stuff between the service clients?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ITokenResponse is part of azure client. I have opened a work item to move this to fluid static or somewhere else. But that's only the shared piece as of now.

import { OdspClient } from "@fluid-experimental/odsp-client";

const odspClient = new OdspClient(props);
const { container, services } = await OdspClient.getContainer("_unique-id_", schema);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this example assuming that top-level await is available?


## Using initial objects

The most common way to use Fluid is through initial collaborative objects that are created when the Container is created. Distributed data structures and DataObjects are both supported types of collaborative objects.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we ready to talk about data objects? This would be the first time I think.

};

// Fetch back the container that had been created earlier with the same url and schema
const { container, services } = await OdspClient.getContainer("_unique-url_", schema);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a problem specific to odsp-client or a general design problem with the initial objects model?

}>;
}

// @alpha (undocumented)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: docs

packages/service-client/odsp-client/src/interfaces.ts Outdated Show resolved Hide resolved
}

private createLoader(containerSchema: ContainerSchema): Loader {
const runtimeFactory = new DOProviderContainerRuntimeFactory(containerSchema);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean "file content" defines schema?

permission: [],
scopes: [],
user: { id: "" },
mode: "write",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it starts in read mode how does the developer switch?

jenn-le and others added 5 commits November 15, 2023 18:31
…rosoft#18305)

adds a test for the concurrent restoration and edit of removed tree

---------

Co-authored-by: Noah Encke <Noah.Encke@microsoft.com>
…e/relayService (microsoft#18306)

## Description

Make improvements in calculating retry after time for calls to
storage/relayService. The logic at service side is not very sound to
calculate this value. So at least in short term, improving this logic at
client side to be intelligent in deciding this value.
Loop faced a bug related to this, so we need to provide a short-term
viable sol.

---------

Co-authored-by: Jatin Garg <jatingarg@Jatins-MacBook-Pro-2.local>
## Description

Two recent PRs conflicted to break the build. This fixes it.
## Description

Fix import to remove cycle.
@github-actions github-actions bot added area: dds Issues related to distributed data structures area: dds: tree area: loader Loader related issues area: runtime Runtime related issues area: server Server related issues (routerlicious) labels Nov 15, 2023
@github-actions github-actions bot removed area: server Server related issues (routerlicious) area: runtime Runtime related issues area: dds: tree area: dds Issues related to distributed data structures area: loader Loader related issues labels Nov 15, 2023
@sonalideshpandemsft
Copy link
Contributor Author

I'm merging this PR to move forward. I've documented the majority of the feedback as work items in ADO.

@sonalideshpandemsft sonalideshpandemsft merged commit 7dad164 into microsoft:main Nov 15, 2023
25 checks passed
@sonalideshpandemsft sonalideshpandemsft deleted the alpha-m365-client branch November 15, 2023 20:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
base: main PRs targeted against main branch dependencies Pull requests that update a dependency file public api change Changes to a public API
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

10 participants