DisTEMIST corpus: detection and normalization of disease mentions in spanish clinical cases

DisTEMIST corpus: training set + MULTILINGUAL RESOURCES + CROSSMAPPINGS + test set + background set

Please cite if you use this dataset:

Miranda-Escalada, A., Gascó, L., Lima-López, S., Farré-Maduell, E., Estrada, D., Nentidis, A., Krithara, A., Katsimpras, G., Paliouras, G., & Krallinger, M. (2022). Overview of DisTEMIST at BioASQ: Automatic detection and normalization of diseases from clinical texts: results, methods, evaluation and multilingual resources. Working Notes of Conference and Labs of the Evaluation (CLEF) Forum. CEUR Workshop Proceedings

@article{miranda2022overview, title={Overview of DisTEMIST at BioASQ: Automatic detection and normalization of diseases from clinical texts: results, methods, evaluation and multilingual resources}, author={Miranda-Escalada, Antonio and Gascó, Luis and Lima-López, Salvador and Farré-Maduell, Eulàlia and Estrada, Darryl and Nentidis, Anastasios and Krithara, Anastasia and Katsimpras, Georgios and Paliouras, Georgios and Krallinger, Martin}, booktitle={Working Notes of Conference and Labs of the Evaluation (CLEF) Forum. CEUR Workshop Proceedings}, year={2022} }

Introduction

The DisTEMIST corpus is a collection of 1000 clinical cases with disease mention annotations manually mapped to Snomed-CT concepts. All documents are released in the context of the BioASQ DisTEMIST track for CLEF 2022. For more information about the track and its schedule, please visit the website.

File structure:

The DisTEMIST corpus has been randomly divided into a training set, containing 750 clinical cases, and a test set (584 in the case of subtrack2), consisting of 250 additional cases. Participants must train their systems using the train set and submit predictions for the test set, on which they will be evaluated.  The file structure of the corpus is as follows:

• train_set:
  • text_files: Folder with plain text files of the clinical cases
  • subtrack1_entities: It contains annotations in a tab-separated file (TSV) with the following columns:
    • filename: document name
    • mark: identifier mention id
    • label: mentions type (ENFERMEDAD)
    • off0: starting position of the mention in the document
    • off1: ending position of the mention in the document
    • span:  text span
  • subtrack2_linking: It contains annotations in a tab-separated file (TSV) with the following columns:
    • filename: document name
    • mark: identifier mention id
    • label: mentions type (ENFERMEDAD)
    • off0: starting position of the mention in the document
    • off1: ending position of the mention in the document
    • span:  text span
    • codes: List of Snomed-CT concept codes linked to the mention. If there is more than one code associated with a mention, they will be concatenated by the symbol "+".
    • semantic relation: the relationship between the assigned code and the mention. It can be EXACT, when the code corresponds exactly with the mention, or NARROW, when the mention corresponds to a narrower concept than the Snomed-CT code. For instance, the concept "Chorioretinal lacunae" does not exist in Snomed-CT. Then, it is normalized to the Snomed-CT ID 302893000 ("Chorioretinal disorder").
    • (Note: the training for entity linking were released in two parts, you must join them)

• test_annotated:
  • text_files: Folder with plain text files of the clinical cases
  • brat: Folder with text files and their annotations in brat's .ann format
  • subtrack1_entities: It contains annotations in a tab-separated file (TSV) with the same columns as the train set
  • subtrack2_linking: It contains annotations in a tab-separated file (TSV) with the same columns as the train set

• test_background_unannotated/text_files: 3000 clinical cases (test + background). In the the original task, participants had to make predictions for these 3000 files and they were evaluated on a subset of them.

• multilingual-resources: we have generated the annotated training and validation sets in 6 languages: English, Portuguese, Catalan, Italian, French and Romanian. The process was:
  1. The text files were translated with a neural machine translation system.
  2. The annotations were translated with the same neural machine translation system.
  3. The translated annotations were transferred to the translated text files using an annotation transfer technology.
  4. The text files are stored in the multilingual_resources/training-text-files subfolder.
  5. The annotated TSV files are stored in the multilingual_resources/lang subfolders.
  6. If you want to visualize the multilingual resources, check out this Brat server: https://temu.bsc.es/mDistemist/#/translations/
     For instance, you can see the parallel annotations in English vs in French, or in Spanish (the gold standard) vs in Catalan.

• cross-mappings. We include the same entities as in DISTEMIST-linking but mapped to Snomed-CT, MeSH, ICD-10, HPO, and OMIM. The original mappings are manual and to Snomed-CT. The mapping to the other terminologies was done through the UMLS Metathesaurus.

Resources 

• Task Web
• Evaluation Library
• DISTEMIST gazetteer
• DISTEMIST guidelines
• Citation: Miranda-Escalada, A., Gascó, L., Lima-López, S., Farré-Maduell, E., Estrada, D., Nentidis, A., Krithara, A., Katsimpras, G., Paliouras, G., & Krallinger, M. (2022). Overview of DisTEMIST at BioASQ: Automatic detection and normalization of diseases from clinical texts: results, methods, evaluation and multilingual resources. Working Notes of Conference and Labs of the Evaluation (CLEF) Forum. CEUR Workshop Proceedings
• Overview paper
• CLEF/BioASQ Workshop proceedings
• Youtube videos (overview & teams)
• DISTEMIST overview talk slides

License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Additional resources and corpora

If you are interested in DisTEMIST, you might want to check out these corpora and resources:

• SymTEMIST (Corpus of symptoms, signs and findings mentions and normalization to SNOMED CT, same document collection)
• MedProcNER (Corpus of clinical procedure mentions and normalization to SNOMED CT, same document collection)
• PharmaCoNER (Corpus of medications, drugs, chemical substances, genes, proteins and vaccine mentions and normalization, same document collection)
• MEDDOPROF (Corpus of mentions of professions, occupations and working status and normalization, different document collection with some overlapping documents)
• MEDDOPLACE (Corpus of mentions of place-related entity mentions, including departments, nationalities or patient movements etc.. and normalization, different document collection with some overlapping documents)
• MEDDOCAN (Corpus of mentions of Personal Health Identifiers (PHI), modified synthetic verions of the document collection)
• CANTEMIST (Corpus of cancer tumor morphology mentions and normalization, different document collection)
• CodiESp (Corpus of clinical case reportes with assigned clinical codes from ICD10, Spanish version, same document collection)
• LivingNER (Corpus of mentions of species, including human/family members, pathogens, food, etc.. and normalization to NCBI Taxonomy, different document collection with some overlapping documents)
• SPACCC-POS (Corpus of clinical case reports in Spanish annotated with POS-tags, same document collection)
• SPACCC-TOKEN (Corpus of clinical case reports in Spanish annotated with token-tags (word mention boundaries), same document collection)
• SPACCC-SPLIT (Corpus of clinical case reports in Spanish annotated with sentence boundary-tags, same document collection)
• MESINESP-2 (Corpus of manually indexed records with DeCS /MeSH terms comprising scientific literature abstracts, clinical trials, and patent abstracts, different document collection)

Contact

If you have any questions or suggestions, please contact us at:

- Martin Krallinger (<krallinger [dot] martin [at] gmail [dot] com>)