Cannot use semantic search bindings in .NET 8

[paulyuk]

When I change my standard sample demo project to target .NET 8 in the csproj, the project builds clean, but then I get this runtime error when I test a post to IngestData (uses Embeddings binding):

This branch exactly repros it as you use test.http and post to the local IngestData function:.

Functions:

        IngestData: [POST] http://localhost:7071/api/IngestData

        PromptData: [POST] http://localhost:7071/api/PromptData

        WhoIs:  http://localhost:7071/api/whois/{name}

        OpenAI::ChatBotEntity: entityTrigger

For detailed output, run func with --verbose flag.
[2024-02-15T18:38:50.834Z] Host lock lease acquired by instance ID '000000000000000000000000731A3487'.
[2024-02-15T18:38:52.563Z] Executing 'IngestData' (Reason='This function was programmatically called via the host APIs.', Id=e43173fd-3292-44f4-b59c-c519224a4bd8)
[2024-02-15T18:38:52.586Z] Executed 'IngestData' (Failed, Id=e43173fd-3292-44f4-b59c-c519224a4bd8, Duration=39ms)
[2024-02-15T18:38:52.586Z] System.Private.CoreLib: Exception while executing function: IngestData. Microsoft.Azure.WebJobs.Host: One or more errors occurred. (Exception binding parameter 'req') (Exception binding parameter 'embeddings'). Exception binding parameter 'req'. System.Private.CoreLib: Could not load type 'System.Runtime.CompilerServices.NullableContextAttribute' from assembly 'System.Runtime, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.

It is throwing on this line of code that uses the EmbeddingsRequest:embeddings record object, and note I have two fields, with RawText being null here:

        public record EmbeddingsRequest(string RawText, string FilePath);

//...
            await output.AddAsync(new SearchableDocument(title, embeddings));

If i changed EmbeddingsRequest definition to match the sample having only a single field in the record, I go back to getting Build errors all the time (maybe this was bound to happen and I forced a rebuild?):

        public record EmbeddingsRequest(string FilePath);
        public record SemanticSearchRequest(string Prompt);

        [FunctionName("IngestData")]
        public static async Task<IActionResult> IngestData(
            [HttpTrigger(AuthorizationLevel.Function, "post")] EmbeddingsRequest req,
            [Embeddings("{FilePath}", inputType: InputType.FilePath, 
            Model = "%AZURE_OPENAI_EMBEDDINGS_DEPLOYMENT%")] EmbeddingsContext embeddings,
            [SemanticSearch("KustoConnectionString", "Documents")] IAsyncCollector<SearchableDocument> output)
        {
            string title = Path.GetFileNameWithoutExtension(req.FilePath);
            await output.AddAsync(new SearchableDocument(title, embeddings));
            return new OkObjectResult(new { status = "success", title, chunks = embeddings.Count });
        }

[2024-02-15T18:34:32.589Z] Found /Users/paulyuk/src/paulyuk/functions/ai/ai-bindings/ai-bindings.csproj. Using for user secrets file configuration.
[2024-02-15T18:34:33.488Z] No job functions found. Try making your job classes and methods public. If you're using binding extensions (e.g. Azure Storage, ServiceBus, Timers, etc.) make sure you've called the registration method for the extension(s) in your startup code (e.g. builder.AddAzureStorage(), builder.AddServiceBus(), builder.AddTimers(), etc.).
[2024-02-15T18:34:33.507Z] The 'IngestData' function is in error: The binding type(s) 'embeddings, semanticSearch' are not registered. Please ensure the type is correct and the binding extension is installed.
[2024-02-15T18:34:33.507Z] The 'PromptData' function is in error: The binding type(s) 'semanticSearch' are not registered. Please ensure the type is correct and the binding extension is installed.
[2024-02-15T18:34:33.507Z] The 'WhoIs' function is in error: The binding type(s) 'textCompletion' are not registered. Please ensure the type is correct and the binding extension is installed.
For detailed output, run func with --verbose flag.

The net of this (no pun intended :) ) is I cannot use .NET 8, or figure out a workaround.

[cgillum]

Your sample project targets .NET in-proc, so this is mostly expected. For the benefit of anyone else who lands here, per the .NET 8 announcement, Azure Functions doesn't yet support .NET 8 using the in-proc model.

Support for .NET 8 does not currently extend to applications using the in-process model. See the August 2023 roadmap update for more information regarding support timelines.

In the meantime, we need to create a .NET isolated version of these bindings so that we can support .NET 8 users on the .NET Isolated worker.

/cc @aishwaryabh

[paulyuk]

Oh thank you @cgillum - you're right I had not connected the dots. Is there any possibility of getting a better error message for this case, maybe pointing to the article/roadmap?

When Open.AI.Kusto is ready in isolated, I'll try .NET 8 again.

Once we have taken action on any doc clarifications or error messages please feel free to close this.

[cgillum]

As an extension vendor, I'm not sure if there's much we can do to control the error experience, but I wonder if @fabiocav or @mattchenderson have thoughts on how the "graceful error message" can be solved in general for customers that try to run in-process functions on .NET 8?

[manvkaur]

@paulyuk , @cgillum - We have added samples for dotnet-isolated using dotnet 8.
PR #50

Are we good to close this issue now?