This repository contains data related to a research supervised project within the Master Intégration de Compétences (ICo) at Montpellier University.
The project consisted to explore the question: "can LLMs compute the full set of concepts of a formal context — a task known to be highly combinatorial?"
We explored this question using two approaches:
- the first ("direct computation") consisted of asking the LLMs directly for the list of concepts.
- the second ("code generation") involved asking them to produce code that performs the computation, which we then tested.
We selected a benchmark of formal contexts of various sizes, several LLM models, and multiple prompting strategies.
The project has been selected for presentation at Consoft @ CONCEPTS 2025
- Directory "Synthetics" contains a set of randomly generated formal contexts of increasing size (from 2x3 to 50x50), with incidence probability p=0.5
- Real formal contexts have been selected in UCI Machine Learning Repository (https://archive.ics.uci.edu/), then binarized by A. Gutierrez for FCA4J evaluation and are available at https://gite.lirmm.fr/gutierre/fca4j-benchmark
- Code directory: contains the best Java codes generated by LLMs. Its subdirectory "Utils"contains the parsers used to extract the intent/extent pairs from the fca4j-generated dot files. It also contains the context generator: a simple program the populates a csv file with either 1s or 0s each with incidence probability p=0.5
- DirectComputation directory: contains the used synthetics contexts with the expected result (Sub-directory "Contexts") and the LLM conversations (Sub-directory "tests")
- Synthetics directory: has been described before
- Prompts directory: contains the prompts for code generation
- M1_TER_rapport_FCA.pdf: is the report on the project
- 2025_CONSOFT_Slides: The presentation slides for the ConSoft workshop
- Valentino Cocks
- Arame Diop
- Oscar Jimenez Flores
- Yazmín Mendoza