How to add store:true in elasticsearch_dsl mapping

[Lisahtwy]

Hi,

I am new to elasticsearch and elasticsearch_dsl. I have created a new index with custom analyzer as below in kibana.

PUT /index_dev2/
{
   "settings": {
     "index": {
       "number_of_shards": 1,
       "number_of_replicas": 1
      },
      "analysis": {
         "analyzer": {
            "my_analyzer": {
               "type": "custom",
               "filter": [
                  "lowercase"
               ],
               "tokenizer": "whitespace"
            }
         }
      }
   },
   "mappings": {
     "dynamic": true,
      "properties": {
          "file": {
            "properties": {
              "filename": {
                "type": "keyword",
                "store": true
              }
            }
          },
        "content": {
          "type": "text",
          "analyzer": "my_analyzer"
        }
      }
    }
}

I am trying to create same index(with different name) using elasticsearch_dsl.

from elasticsearch import Elasticsearch
from elasticsearch_dsl import Index,Search,Document, analyzer, tokenizer,\
                           connections,Mapping, Nested, Text, Keyword, InnerDoc
from elasticsearch_dsl.field import Text

connections.create_connection(hosts=['localhost'])

my_analyzer = analyzer( \
    "my_analyzer", \
    type = "custom", \
    tokenizer=tokenizer("whitespace"),\
    filter=['lowercase'])

index1 = Index('myindex')
index1.analyzer(my_analyzer)
index1.create()

class File(InnerDoc):
    filename = Keyword()

class ResumeIndex(Document):
    content = Text(analyzer = my_analyzer)
    file = Nested(File)

    class Index:
        name = "myindex"

ResumeIndex.init()
response = ResumeIndex._index.analyze(body={'analyzer':'my_analyzer', 'text': 'c# developer'})
print(response)

When I reindex using FsCrawler, it is throwing a warning Mapping is incorrect and add store:true to file.filename

23:34:08,500 ^[[36mDEBUG^[[m [f.p.e.c.f.FsParserAbstract] fetching content from [/var/www/html/file-scanner/ESFiles],[test01741.doc]
23:34:08,500 ^[[36mDEBUG^[[m [f.p.e.c.f.f.FsCrawlerUtil] computeVirtualPathName(/var/www/html/file-scanner/ESFiles, /var/www/html/file-scanner/ESFiles/test01741.doc) = /test01741.doc
23:34:08,501 ^[[36mDEBUG^[[m [f.p.e.c.f.FsParserAbstract] Indexing myindex/ff9b8a1fc3d994d22f8d920f0b897?pipeline=null
23:34:08,501 ^[[36mDEBUG^[[m [f.p.e.c.f.FsParserAbstract] Looking for removed files in [/var/www/html/file-scanner/ESFiles]...
23:34:11,080 ^[[33mWARN ^[[m [f.p.e.c.f.FsParserAbstract] Can't find stored field name to check existing filenames in path [/var/www/html/file-scanner/ESFiles]. Please set store: true on field [file.filename]
23:34:11,080 ^[[33mWARN ^[[m [f.p.e.c.f.FsParserAbstract] Error while crawling /var/www/html/file-scanner/ESFiles: Mapping is incorrect: please set stored: true on field [file.filename].
23:34:11,080 ^[[33mWARN ^[[m [f.p.e.c.f.FsParserAbstract] Full stacktrace
java.lang.RuntimeException: Mapping is incorrect: please set stored: true on field [file.filename].
        at fr.pilato.elasticsearch.crawler.fs.FsParserAbstract.getFileDirectory(FsParserAbstract.java:374) ~[fscrawler-core-2.7-SNAPSHOT.jar:?]
        at fr.pilato.elasticsearch.crawler.fs.FsParserAbstract.addFilesRecursively(FsParserAbstract.java:309) ~[fscrawler-core-2.7-SNAPSHOT.jar:?]
        at fr.pilato.elasticsearch.crawler.fs.FsParserAbstract.run(FsParserAbstract.java:149) [fscrawler-core-2.7-SNAPSHOT.jar:?]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_232]
23:34:11,081 ^[[32mINFO ^[[m [f.p.e.c.f.FsParserAbstract] FS crawler is stopping after 1 run
23:34:11,162 ^[[36mDEBUG^[[m [f.p.e.c.f.FsCrawlerImpl] Closing FS crawler [myindex]
23:34:11,162 ^[[36mDEBUG^[[m [f.p.e.c.f.FsCrawlerImpl] FS crawler thread is now stopped

Also When I checked the "myindex" mappings in kibana, the file type is appearing as nested. will this create any problem?

  "file" : {
          "type" : "nested",
          "properties" : {
            "content_type" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              }
            },

Could you please tell me what changes should I make in my mapping?

-Lisa

[Lisahtwy]

Could anyone please answer this?

[cj564932957]

some_things = Text(fields={'keyword': Keyword()}, store=False)

[sethmlarson]

Using store=False on the field should accomplish what you want.