Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow combining index sorting and nested mappings. #105738

Open
jpountz opened this issue Feb 22, 2024 · 2 comments
Open

Allow combining index sorting and nested mappings. #105738

jpountz opened this issue Feb 22, 2024 · 2 comments
Assignees
Labels
>enhancement :Search/Search Search-related issues that do not fall into other categories Team:Search Meta label for search team

Comments

@jpountz
Copy link
Contributor

jpountz commented Feb 22, 2024

Description

Until now, we disallowed configuring nested fields in mappings if an index sort was configured as the index sort could break the way that we organize doc IDs.

Lucene recently fixed this by enabling recording root documents via IndexWriterConfig#setParentField. When a parent field is configured, index sorting will make sure to preserve the block structure. We should now remove the Elasticsearch limitation by removing the check and configuring a parent field when index sorting is enabled. This would then allow nested mappings to be added.

@jpountz jpountz added >enhancement :Search/Search Search-related issues that do not fall into other categories labels Feb 22, 2024
@elasticsearchmachine elasticsearchmachine added the Team:Search Meta label for search team label Feb 22, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-search (Team:Search)

@itepifanio
Copy link

@jpountz does Elastic have any workaround to bypass this behavior?

I need to access nested information but I'm paginating a whole index and using the sort and search_after to do so.

def scrolling(...):
    result = elastic_client.search(
        index=self.index,
        size=self.batch_size,
        search_after=self._search_after,
        sort=self._sort
    ),

    hits = result.get("hits", {}).get("hits", [])
    if not hits:
        return
   
    self._search_after = hits[-1]["sort"]

@ioanatia ioanatia self-assigned this May 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>enhancement :Search/Search Search-related issues that do not fall into other categories Team:Search Meta label for search team
Projects
None yet
Development

No branches or pull requests

4 participants