Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

error TypeError: Cannot read properties of undefined (reading 'text') when run npm run ingest #470

Open
Yenhi501 opened this issue Jun 11, 2024 · 2 comments
Labels
kind: bug Related to a bug, vulnerability, unexpected error with an existing feature

Comments

@Yenhi501
Copy link

Yenhi501 commented Jun 11, 2024

creating vector store...
error TypeError: Cannot read properties of undefined (reading 'text')
at (d:\demo\node_modules@pinecone-database\pinecone\dist\errors\utils.js:44:57)
at step (d:\demo\node_modules@pinecone-database\pinecone\dist\errors\utils.js:33:23)
at Object.next (d:\demo\node_modules@pinecone-database\pinecone\dist\errors\utils.js:14:53)
at (d:\demo\node_modules@pinecone-database\pinecone\dist\errors\utils.js:8:71)
at new Promise ()
at __awaiter (d:\demo\node_modules@pinecone-database\pinecone\dist\errors\utils.js:4:12)
at extractMessage (d:\demo\node_modules@pinecone-database\pinecone\dist\errors\utils.js:40:48)
at (d:\demo\node_modules@pinecone-database\pinecone\dist\errors\handling.js:66:70)
at step (d:\demo\node_modules@pinecone-database\pinecone\dist\errors\handling.js:33:23)
at Object.next (d:\demo\node_modules@pinecone-database\pinecone\dist\errors\handling.js:14:53)
d:\demo\scripts\ingest-data.ts:46
throw new Error('Failed to ingest your data');
^

Error: Failed to ingest your data
at run (d:\demo\scripts\ingest-data.ts:46:11)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at (d:\demo\scripts\ingest-data.ts:51:3)

Node.js v19.8.1

#My code :

import { RecursiveCharacterTextSplitter } from 'langchain/text_splitter';
import { OpenAIEmbeddings } from 'langchain/embeddings/openai';
import { PineconeStore } from 'langchain/vectorstores/pinecone';
import { pinecone } from '@/utils/pinecone-client';
import { PDFLoader } from 'langchain/document_loaders/fs/pdf';
import { PINECONE_INDEX_NAME, PINECONE_NAME_SPACE } from '@/config/pinecone';
import { DirectoryLoader } from 'langchain/document_loaders/fs/directory';

/* Name of directory to retrieve your files from 
   Make sure to add your PDF files inside the 'docs' folder
*/
const filePath = 'docs';

export const run = async () => {
  try {
    /*load raw docs from the all files in the directory */
    const directoryLoader = new DirectoryLoader(filePath, {
      '.pdf': (path) => new PDFLoader(path),
    });

    // const loader = new PDFLoader(filePath);
    const rawDocs = await directoryLoader.load();

    /* Split text into chunks */
    const textSplitter = new RecursiveCharacterTextSplitter({
      chunkSize: 1000,
      chunkOverlap: 200,
    });

    const docs = await textSplitter.splitDocuments(rawDocs);
    console.log('split docs', docs);

    console.log('creating vector store...');
    /*create and store the embeddings in the vectorStore*/
    const embeddings = new OpenAIEmbeddings();
    const index = pinecone.Index(PINECONE_INDEX_NAME); //change to your own index name

    //embed the PDF documents
    await PineconeStore.fromDocuments(docs, embeddings, {
      pineconeIndex: index,
      namespace: PINECONE_NAME_SPACE,
      textKey: 'text',
    });
  } catch (error) {
    console.log('error', error);
    throw new Error('Failed to ingest your data');
  }
};

(async () => {
  await run();
  console.log('ingestion complete');
})();


@Yenhi501
Copy link
Author

Yenhi501 commented Jun 11, 2024

Please help me, I have a report related to this

@dosubot dosubot bot added the kind: bug Related to a bug, vulnerability, unexpected error with an existing feature label Jun 11, 2024
@Yenhi501
Copy link
Author

Yenhi501 commented Jun 11, 2024

Hello, I have fixed it
I update
const index = pinecone.Index(PINECONE_INDEX_NAME);
to
const pc = new Pinecone({ apiKey: 'key' }); const index = pc.index(PINECONE_INDEX_NAME);

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind: bug Related to a bug, vulnerability, unexpected error with an existing feature
Projects
None yet
Development

No branches or pull requests

1 participant