Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: explicitly call out that ImagenInlineImage#data is base64 encoded #6785

Closed
wants to merge 2 commits into from

Conversation

thatfiredev
Copy link
Member

@thatfiredev thatfiredev commented Mar 20, 2025

This is meant to match the JS ref docs.

See context in #6786

Copy link
Contributor

github-actions bot commented Mar 20, 2025

Javadoc Changes:
--- /home/runner/diff/original/firebase-kotlindoc/android/com/google/firebase/vertexai/type/ImagenInlineImage.html	2025-03-20 14:36:38.017667114 +0000
+++ /home/runner/diff/modified/firebase-kotlindoc/android/com/google/firebase/vertexai/type/ImagenInlineImage.html	2025-03-20 14:34:26.754044290 +0000
@@ -29,7 +29,7 @@
             <td><code>final @<a href="https://developer.android.com/reference/kotlin/androidx/annotation/NonNull.html">NonNull</a> byte[]</code></td>
             <td>
               <div><code><a href="/docs/reference/android/com/google/firebase/vertexai/type/ImagenInlineImage.html#data()">data</a></code></div>
-              <p>Contains the raw bytes of the image</p>
+              <p>Contains the base64-encoded image data.</p>
             </td>
           </tr>
           <tr>
@@ -69,7 +69,7 @@
       <div class="api-item"><a name="getData()"></a><a name="setData()"></a><a name="getData--"></a><a name="setData--"></a>
         <h3 class="api-name" id="data()">data</h3>
         <pre class="api-signature no-pretty-print">public&nbsp;final&nbsp;@<a href="https://developer.android.com/reference/kotlin/androidx/annotation/NonNull.html">NonNull</a> byte[]&nbsp;<a href="/docs/reference/android/com/google/firebase/vertexai/type/ImagenInlineImage.html#data()">data</a></pre>
-        <p>Contains the raw bytes of the image</p>
+        <p>Contains the base64-encoded image data.</p>
       </div>
       <div class="api-item"><a name="getMimeType()"></a><a name="setMimeType()"></a><a name="getMimeType--"></a><a name="setMimeType--"></a>
         <h3 class="api-name" id="mimeType()">mimeType</h3>
--- /home/runner/diff/original/firebase-kotlindoc/kotlin/com/google/firebase/vertexai/type/ImagenInlineImage.html	2025-03-20 14:36:38.006667132 +0000
+++ /home/runner/diff/modified/firebase-kotlindoc/kotlin/com/google/firebase/vertexai/type/ImagenInlineImage.html	2025-03-20 14:34:26.744044335 +0000
@@ -51,7 +51,7 @@
             <td><code><a href="https://kotlinlang.org/api/latest/jvm/stdlib/kotlin/-byte-array/index.html">ByteArray</a></code></td>
             <td>
               <div><code><a href="/docs/reference/kotlin/com/google/firebase/vertexai/type/ImagenInlineImage.html#data()">data</a></code></div>
-              <p>Contains the raw bytes of the image</p>
+              <p>Contains the base64-encoded image data.</p>
             </td>
           </tr>
           <tr>
@@ -77,7 +77,7 @@
       <div class="api-item"><a name="getData()"></a><a name="setData()"></a><a name="getData--"></a><a name="setData--"></a>
         <h3 class="api-name" id="data()">data</h3>
         <pre class="api-signature no-pretty-print">val&nbsp;<a href="/docs/reference/kotlin/com/google/firebase/vertexai/type/ImagenInlineImage.html#data()">data</a>:&nbsp;<a href="https://kotlinlang.org/api/latest/jvm/stdlib/kotlin/-byte-array/index.html">ByteArray</a></pre>
-        <p>Contains the raw bytes of the image</p>
+        <p>Contains the base64-encoded image data.</p>
       </div>
       <div class="api-item"><a name="getMimeType()"></a><a name="setMimeType()"></a><a name="getMimeType--"></a><a name="setMimeType--"></a>
         <h3 class="api-name" id="mimeType()">mimeType</h3>

@google-oss-bot
Copy link
Contributor

1 Warning
⚠️ Did you forget to add a changelog entry? (Add the 'no-changelog' label to the PR to silence this warning.)

Generated by 🚫 Danger

@google-oss-bot
Copy link
Contributor

google-oss-bot commented Mar 20, 2025

Co-authored-by: rachelsaunders <52258509+rachelsaunders@users.noreply.github.com>
Copy link
Contributor

Test Results

 18 files   -  1 014   18 suites   - 1 014   13s ⏱️ - 33m 56s
 85 tests  -  5 776   85 ✅  -  5 753  0 💤  - 22  0 ❌  - 1 
170 runs   - 11 615  170 ✅  - 11 570  0 💤  - 44  0 ❌  - 1 

Results for commit dbbc34f. ± Comparison against base commit 60a021d.

This pull request removes 5776 tests.
com.google.android.datatransport.cct.CctBackendFactoryTest ‑ create_returnCCTBackend_WhenBackendNameIsCCT
com.google.android.datatransport.cct.CctDestinationTest ‑ cctDestination_shouldOnlySupportProtoAndJson
com.google.android.datatransport.cct.CctDestinationTest ‑ cctDestination_shouldSupportProtoAndJson
com.google.android.datatransport.cct.CctTransportBackendTest ‑ decorate_whenOffline_shouldProperlyPopulateNetworkInfo
com.google.android.datatransport.cct.CctTransportBackendTest ‑ decorate_whenOnline_shouldProperlyPopulateNetworkInfo
com.google.android.datatransport.cct.CctTransportBackendTest ‑ schedule_shouldAddCookieOnPseudonymousIds
com.google.android.datatransport.cct.CctTransportBackendTest ‑ schedule_shouldDropCookieOnMixedPseudonymousIds
com.google.android.datatransport.cct.CctTransportBackendTest ‑ send_CompressedResponseIsUncompressed
com.google.android.datatransport.cct.CctTransportBackendTest ‑ send_whenBackendRedirectsMoreThan5Times_shouldOnlyRedirect4Times
com.google.android.datatransport.cct.CctTransportBackendTest ‑ send_whenBackendRedirects_shouldCorrectlyFollowTheRedirectViaPost
…

@google-oss-bot
Copy link
Contributor

@andrewheard
Copy link
Collaborator

Before we merge this, I think it would be ideal to determine if data should actually be base64-encoded. More details in firebase/firebase-ios-sdk#14603 (comment).

Going from the other direction (encoding), we don't ask devs to pass in base64-encoded data when using Gemini:

* @param inlineData the binary data as a [ByteArray]
* @param mimeType an IANA standard MIME type. For supported values, see the
* [Vertex AI documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/send-multimodal-prompts#media_requirements)
*/
public class InlineDataPart(public val inlineData: ByteArray, public val mimeType: String) : Part {

Internally we encode it as base64 for them:

is InlineDataPart ->
InlineDataPart.Internal(
InlineDataPart.Internal.InlineData(
mimeType,
android.util.Base64.encodeToString(inlineData, BASE_64_FLAGS)
)
)

If we are allowed to during Public Preview (cc: @rlazo for advice), IMHO, it would be ideal to decode the base64 for devs and to return a binary ByteArray.

@thatfiredev
Copy link
Member Author

We have discussed this internally and decided to return the decoded inline image - that was the intended behavior all along. (returning the encoded base64 image was an oversight at first launch)

Closing this PR in favor of #6800.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants