Input array dimensions mismatch

[smortezah]

Running the 'Getting started' code, I receive the following error:

ValueError: all the input array dimensions except for the concatenation axis must match exactly, but along dimension 1, the array at index 0 has size 1536 and the array at index 1 has size 3072

from typeagent import create_conversation
from typeagent.aitools.utils import load_dotenv
from typeagent.transcripts.transcript import TranscriptMessage, TranscriptMessageMeta

load_dotenv()


def read_messages(filename) -> list[TranscriptMessage]:
    messages: list[TranscriptMessage] = []
    with open(filename, "r") as f:
        for line in f:
            # Parse each line into a TranscriptMessage
            speaker, text_chunk = line.split(None, 1)
            message = TranscriptMessage(
                text_chunks=[text_chunk],
                metadata=TranscriptMessageMeta(speaker=speaker),
            )
            messages.append(message)
    return messages


async def main():
    conversation = await create_conversation("demo.db", TranscriptMessage)
    messages = read_messages("testdata.txt")
    print(f"Indexing {len(messages)} messages...")
    results = await conversation.add_messages_with_indexing(messages)
    print(f"Indexed {results.messages_added} messages.")
    print(f"Got {results.semrefs_added} semantic refs.")


if __name__ == "__main__":
    import asyncio

    asyncio.run(main())

My .env file includes:

AZURE_OPENAI_API_KEY=
AZURE_OPENAI_ENDPOINT=
AZURE_OPENAI_ENDPOINT_EMBEDDING=

Python version: 3.14

[gvanrossum]

@smortezah: Interesting. What's your embedding model named? I'm guessing it produces 3072-wide vectors.

There could well be a bug there -- I will do some digging once I have your embedding endpoint and/or embedding name. (Things around embeddings are confusing.)

[wirthual]

By default text-embedding-3-small has 1536 embedding length, text-embedding-3-large has 3072. Maybe the embedding model got changed in between runs?

[smortezah]

@smortezah: Interesting. What's your embedding model named? I'm guessing it produces 3072-wide vectors.

There could well be a bug there -- I will do some digging once I have your embedding endpoint and/or embedding name. (Things around embeddings are confusing.)

@gvanrossum The embedding model is text-embedding-3-large.

[smortezah]

By default text-embedding-3-small has 1536 embedding length, text-embedding-3-large has 3072. Maybe the embedding model got changed in between runs?

No. I didn't change it between runs :)

[gvanrossum]

The embedding model is text-embedding-3-large.

Workaround 1: Use text-embedding-3-small. (If that's available to you.) We use embeddings only to find related terms and a smaller embedding works fine.

Workaround 2. Make sure you have release 0.3.2 (there's a bug somewhere and the exact code to create the AsyncEmbeddingmodel instance will change in the next release). Use a ConversationSettings object, as follows:

from typeagent.aitools.embeddings import AsyncEmbeddingModel
from typeagent.knowpro.convsettings import ConversationSettings

async def main():
    model = AsyncEmbeddingModel(
        model_name="text-embedding-3-large",
        embedding_size=3072,
        endpoint_envvar="AZURE_OPENAI_ENDPOINT_EMBEDDING",
    )
    settings = ConversationSettings(model=model)
    settings.semantic_ref_index_settings.auto_extract_knowledge = True
    conversation = await create_conversation(
        "demo.db", TranscriptMessage, settings=settings
    )
    # ...original code...

[smortezah]

Workaround 1: Use text-embedding-3-small. (If that's available to you.) We use embeddings only to find related terms and a smaller embedding works fine.
...

Same error: ValueError: all the input array dimensions except for the concatenation axis must match exactly, but along dimension 1, the array at index 0 has size 1536 and the array at index 1 has size 3072

My code:

from typeagent import create_conversation
from typeagent.aitools.embeddings import AsyncEmbeddingModel
from typeagent.aitools.utils import load_dotenv
from typeagent.knowpro.convsettings import ConversationSettings
from typeagent.transcripts.transcript import TranscriptMessage, TranscriptMessageMeta

load_dotenv()


def read_messages(filename) -> list[TranscriptMessage]:
    messages: list[TranscriptMessage] = []
    with open(filename, "r") as f:
        for line in f:
            # Parse each line into a TranscriptMessage
            speaker, text_chunk = line.split(None, 1)
            message = TranscriptMessage(
                text_chunks=[text_chunk],
                metadata=TranscriptMessageMeta(speaker=speaker),
            )
            messages.append(message)
    return messages


async def main():
    model = AsyncEmbeddingModel(
        model_name="text-embedding-3-large",
        embedding_size=3072,
        endpoint_envvar="AZURE_OPENAI_ENDPOINT_EMBEDDING",
    )
    settings = ConversationSettings(model=model)
    settings.semantic_ref_index_settings.auto_extract_knowledge = True
    conversation = await create_conversation("demo.db", TranscriptMessage)
    messages = read_messages("testdata.txt")
    print(f"Indexing {len(messages)} messages...")
    results = await conversation.add_messages_with_indexing(messages)
    print(f"Indexed {results.messages_added} messages.")
    print(f"Got {results.semrefs_added} semantic refs.")


if __name__ == "__main__":
    import asyncio

    asyncio.run(main())

[gvanrossum]

Are you saying you get the same error with my "Workaround 2"?

In either case, can you post a full traceback here? (It's fine to censor your home directory out of the file paths, all I need is the paths from typeagent down.)

[smortezah]

Are you saying you get the same error with my "Workaround 2"?

In either case, can you post a full traceback here? (It's fine to censor your home directory out of the file paths, all I need is the paths from typeagent down.)

Yes, with workaround 2:

0.314s -- Using Azure OpenAI
0.008s -- Using Azure OpenAI
Indexing 3 messages...
Traceback (most recent call last):
  File "/ingest.py", line 43, in <module>
    asyncio.run(main())
    ~~~~~~~~~~~^^^^^^^^
  File "/Users/user/.local/share/uv/python/cpython-3.14.0-macos-aarch64-none/lib/python3.14/asyncio/runners.py", line 204, in run
    return runner.run(main)
           ~~~~~~~~~~^^^^^^
  File "/Users/user/.local/share/uv/python/cpython-3.14.0-macos-aarch64-none/lib/python3.14/asyncio/runners.py", line 127, in run
    return self._loop.run_until_complete(task)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/Users/user/.local/share/uv/python/cpython-3.14.0-macos-aarch64-none/lib/python3.14/asyncio/base_events.py", line 719, in run_until_complete
    return future.result()
           ~~~~~~~~~~~~~^^
  File "/ingest.py", line 35, in main
    results = await conversation.add_messages_with_indexing(messages)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.14/site-packages/typeagent/knowpro/conversation_base.py", line 155, in add_messages_with_indexing
    await self._update_secondary_indexes_incremental(start_points)
  File "/.venv/lib/python3.14/site-packages/typeagent/knowpro/conversation_base.py", line 232, in _update_secondary_indexes_incremental
    await self._update_related_terms_incremental(start_points.semref_count)
  File "/.venv/lib/python3.14/site-packages/typeagent/knowpro/conversation_base.py", line 290, in _update_related_terms_incremental
    await fuzzy_index.add_terms(list(new_terms))
  File "/.venv/lib/python3.14/site-packages/typeagent/storage/sqlite/reltermsindex.py", line 219, in add_terms
    await self._vector_base.add_key(text)
  File "/.venv/lib/python3.14/site-packages/typeagent/aitools/vectorbase.py", line 94, in add_key
    self._vectors = np.append(self._vectors, embeddings, axis=0)
                    ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.14/site-packages/numpy/lib/_function_base_impl.py", line 5728, in append
    return concatenate((arr, values), axis=axis)
ValueError: all the input array dimensions except for the concatenation axis must match exactly, but along dimension 1, the array at index 0 has size 1536 and the array at index 1 has size 3072

[gvanrossum]

I have overhauled the arguments to AsyncEmbeddingModel and their interpretation.

You can now specify embedding size, model name and env var name separately, and defaults will be provided as follows:

• If model name is known, take defaults from model name table
• If model name is unknown, use 1536 as emb.size and AZURE_OPENAI_ENDPOINT_EMBEDDING as default env.var.

Before I release this to PyPI, could you (@smortezah) help me test this? The code in your last example should work. To test, clone the repo, run make build and then pip install dist/*.whl (or something equivalent). If it doesn't work, post the traceback. I'd also like to see your full endpoint URL -- you can mail it to me at gvanrossum@microsoft.com.

[gvanrossum]

What I wrote to @smortezah (edited for clarity):

The env variable you set is "reserved" for text-embedding-ada-002 (the default). Use a different one (below).

As I said before, there are bugs in 0.3.2 that make it hard to work with text-embedding-3-large, but the latest main branch of the repo makes it easy.

Here's what you should do:

• Make sure you're using the typeagent package from the repo, not from PyPI.
• Delete the existing database (demo.db) before testing.
• Use model = AsyncEmbeddingModel(model_name="text-embedding-3-large")
• Use settings = ConversationSettings(model=model)
• There's no need any more to set auto_extract_knowledge = True (though it's harmless)
• Set and export the env var named AZURE_OPENAI_ENDPOINT_EMBEDDING_3_LARGE=(your endpoint)

Doing this in both demo/ingest.py and demo/query.py should work.

In particular, I'm taking demo/ingest.py and modifying main() as follows:

from typeagent.knowpro.convsettings import ConversationSettings
from typeagent.aitools.embeddings import AsyncEmbeddingModel
model = AsyncEmbeddingModel(model_name="text-embedding-3-large")
settings = ConversationSettings(model=model)
conversation = await create_conversation("demo.db", TranscriptMessage, settings=settings)

And this works for me.

[smortezah]

Fixed.

Thanks

[gvanrossum]

You're welcome. I suspect that the key fix was deleting the old db...