flair with the icelandic_ner dataset

[TatianaMoteuN]

No description provided.

[alanakbik]

Thanks for adding this! Unfortunately, this code does not compile for me. Some of the variables are undefined. Can you check and update?

[alanakbik]

The variable icelandic_ner is not defined

[TatianaMoteuN]

I have changed this morning please can you check it again?

[alanakbik]

The variable icelandic_ner_path is not defined

[TatianaMoteuN]

I have changed it this morning please can you check it again?

[alanakbik]

Why is this part necessary? Are extra offsets needed? Maybe you can use the files as they are?

[TatianaMoteuN]

I have changed it this morning please can you check it again?

[alanakbik]

There are indentation problems here, causing the program to break,

[TatianaMoteuN]

Hello @alanakbik,
please I would like you to have a look at this code because I'm confused about what is wrong

default dataset folder is the cache root

    if not base_path:
        base_path = Path(flair.cache_root) / "datasets"
    data_folder = base_path / dataset_name

    if not os.path.isfile(data_folder / 'icelandic_ner.txt'):
        # download zip
        icelandic_ner ="https://repository.clarin.is/repository/xmlui/handle/20.500.12537/42/allzip"
        icelandic_ner_path = cached_path(icelandic_ner, Path("datasets") / dataset_name)

        #unpacking the zip
        unpack_file(
              icelandic_ner_path,
              data_folder,
              mode="zip",
              keep=True
          )
    #merge the files in one as the zip is containing multiples files
    #entries = os.path.listfile(data_folder)

    with open("icelandic_ner.txt", "wb") as outfile:
        for files in os.walk(data_folder):
           # # print(files[2])
           #  files = glob.glob('*.txt')
            for filename in files[2]:
                if filename.endswith('.txt'):
                    with open(filename) as infile:
                        contents = infile.read()
                       # print(contents)
                        outfile.write(contents)

and the error

/home/aimsgh/home/aimsgh/SCIoI/flair/lib/python3.7/site-packages/torch/cuda/init.py:52: UserWarning: CUDA initialization: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx (Triggered internally at /pytorch/c10/cuda/CUDAFunctions.cpp:100.)
return torch._C._cuda_getDeviceCount() > 0
Traceback (most recent call last):
File "/home/aimsgh/SCIoI/flair/flair-1/load_dataset.py", line 3, in
icelandic_ner = ICELANDIC_NER()
File "/home/aimsgh/SCIoI/flair/flair-1/flair/datasets/sequence_labeling.py", line 654, in init
with open(filename) as infile:
FileNotFoundError: [Errno 2] No such file or directory: 'fbl.txt'

Process finished with exit code 1

[alanakbik]

You are not specifying the correct path to the file. You are only giving the open method the filename. You need to specify the full path to the file. Also for the outfile.

[TatianaMoteuN]

alright, I just add a commit.

…

On Fri, 5 Mar 2021 at 12:55, Alan Akbik ***@***.***> wrote: You are not specifying the correct path to the file. You are only giving the open method the filename. You need to specify the full path to the file. Also for the outfile. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#2114 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ANOMGJ6HS3U3X2R3HRALFVLTCDBCDANCNFSM4YG27IVA> .

-- DISCLAIMER: The contents of this email and any attachments are confidential. They are intended for the named recipient(s) only. If you have received this email by mistake, please notify the sender immediately and you are herewith notified that the contents are legally privileged and that you do not have permission to disclose the contents to anyone, make copies thereof, retain or distribute or act upon it by any means, electronically, digitally or in print. The views expressed in this communication may be of a personal nature and not be representative of AIMS-NEI and/or any of its Centres or Initiatives.

[alanakbik]

Looks good, but please remove the local file and add the tag_to_bioes parameter.

[alanakbik]

Please do not add local files to git!

[alanakbik]

can you also add the tag_to_bioes parameter here? i.e

        `tag_to_bioes=tag_to_bioes,`

[TatianaMoteuN]

Done!

[alanakbik]

@TatianaMoteuN thanks for adding this!