Add VertexAI Sample to repo

[paulb777]

Migrated from the FirebaseVertexAI/Sample directory at firebase/firebase-ios-sdk@c7ceee4.

See individual commits for a summary of the follow up adaptations.

[gemini-code-assist]

Hello @paulb777, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

Summary of Changes

This pull request adds a new VertexAI sample application to the repository. The sample includes several features demonstrating the capabilities of the Vertex AI for Firebase SDK, including text summarization, multi-modal image reasoning, chat conversations, function calling, and image generation using Imagen. It also introduces UI components and view models to support these features.

Highlights

• New VertexAI Sample App: Adds a complete sample application showcasing various Vertex AI functionalities within a Firebase context.
• Feature Implementation: Includes implementations for text summarization, multi-modal image reasoning, chat conversations, function calling with external APIs, and image generation using Imagen.
• UI Components: Introduces reusable UI components in the GenerativeAIUIComponents package, such as InputField and MultimodalInputField, to facilitate user interaction.
• Workflow Integration: Sets up a new workflow (vertexai.yml) to automatically build and test the VertexAI sample on pull requests and scheduled runs.

Changelog

Click here to see the changelog

• .github/workflows/vertexai.yml
  • Added a new workflow to build and test the VertexAI sample on pull requests and a daily schedule.
  • Configured the workflow to run on macOS-15 with Xcode 16.1 and test on iOS, tvOS, and macOS.
• vertexai/ChatSample/Models/ChatMessage.swift
  • Defined ChatMessage model with id, message, participant, and pending properties.
  • Added sample messages for testing and previewing the chat functionality.
• vertexai/ChatSample/Screens/ConversationScreen.swift
  • Implemented the ConversationScreen view, which displays the chat interface.
  • Integrated InputField for user input and a List to display messages.
  • Added toolbar button for starting a new chat.
• vertexai/ChatSample/ViewModels/ConversationViewModel.swift
  • Implemented the ConversationViewModel to manage chat messages and interact with the Vertex AI model.
  • Added methods for sending messages, starting new chats, and handling streaming responses.
• vertexai/ChatSample/Views/BouncingDots.swift
  • Created a BouncingDots view to indicate loading or pending states in the UI.
• vertexai/ChatSample/Views/ErrorDetailsView.swift
  • Implemented ErrorDetailsView to display detailed information about errors encountered during Vertex AI operations, including safety ratings and block reasons.
• vertexai/ChatSample/Views/ErrorView.swift
  • Created ErrorView to display a generic error message with a button to show error details.
• vertexai/ChatSample/Views/MessageView.swift
  • Implemented MessageView to display individual chat messages with different styling for user and system participants.
  • Used MarkdownUI to render messages with Markdown formatting.
• vertexai/FunctionCallingSample/Screens/FunctionCallingScreen.swift
  • Implemented the FunctionCallingScreen view, which demonstrates function calling with Gemini.
  • Integrated InputField for user input and a List to display messages.
• vertexai/FunctionCallingSample/ViewModels/FunctionCallingViewModel.swift
  • Implemented the FunctionCallingViewModel to manage chat messages and interact with the Vertex AI model for function calling.
  • Added methods for sending messages, processing function calls, and handling streaming responses.
  • Defined a function to get exchange rates between currencies.
• vertexai/GenerativeAIMultimodalSample/Screens/PhotoReasoningScreen.swift
  • Implemented the PhotoReasoningScreen view, which allows users to upload images and ask questions about them.
  • Integrated MultimodalInputField for user input and image selection.
• vertexai/GenerativeAIMultimodalSample/ViewModels/PhotoReasoningViewModel.swift
  • Implemented the PhotoReasoningViewModel to manage user input, image selection, and interaction with the Vertex AI model for multi-modal reasoning.
  • Added methods for processing images and generating responses.
• vertexai/GenerativeAITextSample/Screens/SummarizeScreen.swift
  • Implemented the SummarizeScreen view, which allows users to enter text and summarize it.
  • Integrated TextField for user input and a List to display the summarized text.
• vertexai/GenerativeAITextSample/ViewModels/SummarizeViewModel.swift
  • Implemented the SummarizeViewModel to manage user input and interact with the Vertex AI model for text summarization.
  • Added methods for processing text and generating summaries.
• vertexai/GenerativeAIUIComponents/Sources/GenerativeAIUIComponents/InputField.swift
  • Created a reusable InputField component for text input with a submit button.
• vertexai/GenerativeAIUIComponents/Sources/GenerativeAIUIComponents/MultimodalInputField.swift
  • Created a reusable MultimodalInputField component for text input and image selection.
• vertexai/ImagenScreen/ImagenScreen.swift
  • Implemented the ImagenScreen view, which allows users to enter a prompt and generate an image.
  • Integrated TextField for user input and a button to generate the image.
• vertexai/ImagenScreen/ImagenViewModel.swift
  • Implemented the ImagenViewModel to manage user input and interact with the Vertex AI model for image generation.
  • Added methods for processing prompts and generating images.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

---

Trivia time!

What is the name of Google's large language model that powers Vertex AI?

Click here for the answer

Gemini

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request adds a VertexAI sample to the repository, migrating code from the Firebase iOS SDK. The changes include workflow configurations, UI components, view models, and assets for various generative AI functionalities like chat, function calling, multimodal interactions, text summarization, and image generation. Overall, the migration seems well-structured, but there are a few areas that could benefit from closer examination and potential improvements.

Summary of Findings

• Missing Error Handling in UI Components: Several UI components lack explicit error handling for Vertex AI API calls, which could lead to unhandled exceptions and a poor user experience. It's crucial to implement robust error handling to gracefully manage potential failures.
• Potential Performance Bottlenecks in Image Handling: The image resizing logic in PhotoReasoningViewModel might introduce performance bottlenecks when dealing with large images. Consider optimizing this process to ensure smooth performance, especially on lower-end devices.
• Inconsistent Use of Asynchronous Operations: There are inconsistencies in how asynchronous operations are handled across different view models. Standardizing the approach to asynchronous tasks can improve code maintainability and prevent potential race conditions.
• Hardcoded Model Names: The model names are hardcoded in the view models. It would be better to make them configurable to allow for easier experimentation with different models.

Merge Readiness

The pull request introduces significant new functionality to the repository. While the code appears to be well-structured, the identified issues regarding error handling, performance, and asynchronous operations should be addressed before merging. I am unable to approve this pull request, and recommend that others review and approve this code before merging. At a minimum, the high severity issues should be addressed before merging.

[andrewheard]

See individual commits for a summary of the follow up adaptations.
-- @paulb777

Thanks for structuring it this way. Definitely made it easier to review.

Out of scope for this PR which is moving repos for the sample