jupyter/mineral_db.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Mineral Search Using the RRUFF and WikiData Databases\n",
    "\n",
    "The RUFF database contains about 5,700 minerals (many redundant with disparite names).  Many of these come with formula's that can be parsed into distinct compositions.\n",
    "\n",
    "This workbook will create a SQLite database from the mineral compositions and then search this database."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "using NeXLCore\n",
    "using DataFrames\n",
    "using SQLite"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This code downloads the database from its source (on the Internet) and constructs a DataFrame containing the data.\n",
    "\n",
    "It also attempts to parse the `IMA Chemistry (plain)` column to convert the mineral to mass-fraction representation.  This isn't always possible as many \"minerals\" are actually ambiguously defined."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ENV[\"DATADEPS_ALWAYS_ACCEPT\"]=true\n",
    "mdb = loadmineraldata(true)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load the minerals with parseable compositions into a in-memory SQLite database.  (You could write it to disk to but...)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "db = SQLite.DB()\n",
    "NeXLCore.buildMaterialTables(db)\n",
    "for mat in filter(!isnothing, mdb[:,:Material])\n",
    "    NeXLCore.write(db, Material, mat)\n",
    "end"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Search the database on an elemental basis.\n",
    "\n",
    "In this case, the search looks for palladium between 0.6062 and 0.6066 mass-fraction and lead between 0.3930 and 0.3940.\n",
    "\n",
    "This search style is very flexible but a little tedious."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "NeXLCore.findall(db, Material, Dict(n\"Pd\"=>( 0.6062, 0.6066), n\"Pb\"=>(0.3930, 0.3940)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alternatively, you can search by composition.  It is easy to create compositions by mass-fractions using this syntax. \n",
    "\n",
    "The final number is a tolerance that is applied to each element. So this is equivalent to Pd between 0.596 and 0.616 and Pb between 0.383 and 0.403. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "NeXLCore.findall(db, mat\"0.606*Pd+0.393*Pb\", 0.01)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or like this.."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "findall(db, mat\"NaAlSi3O8\", 0.001)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's try again with the WikiData database which contains 3711 minerals."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "wdm = wikidata_minerals()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "db_wd = SQLite.DB()\n",
    "NeXLCore.buildMaterialTables(db_wd)\n",
    "for mat in values(wdm)\n",
    "    NeXLCore.write(db_wd, Material, mat)\n",
    "end"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's perform the same searches as before..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "NeXLCore.findall(db_wd, Material, Dict(n\"Pd\"=>( 0.6062, 0.6066), n\"Pb\"=>(0.3930, 0.3940)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "NeXLCore.findall(db_wd, mat\"0.606*Pd+0.393*Pb\", 0.01)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "findall(db_wd, mat\"NaAlSi3O8\", 0.001)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Of course, these are very simple searches and must more sophisticated search algorithms can readily be imagined and implemented.\n",
    "\n",
    "See [materialdb.jl](https://github.com/usnistgov/NeXLCore.jl/blob/master/src/materialdb.jl) to see how the database is organized and how to search it."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Julia 1.10.0",
   "language": "julia",
   "name": "julia-1.10"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "1.10.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}