From 4fda6f38325fe3a7cfbe1660fb7212eb8d858025 Mon Sep 17 00:00:00 2001
From: Charlotte Silvester <charlotte.silvester@nag.co.uk>
Date: Mon, 12 Aug 2024 10:44:11 +0100
Subject: [PATCH 1/7] add elastic net notebook and data
---
local_optimization/NLDF/elastic_net.ipynb | 535 ++++++++++++++++++++++
local_optimization/NLDF/possum.csv | 105 +++++
2 files changed, 640 insertions(+)
create mode 100644 local_optimization/NLDF/elastic_net.ipynb
create mode 100644 local_optimization/NLDF/possum.csv
diff --git a/local_optimization/NLDF/elastic_net.ipynb b/local_optimization/NLDF/elastic_net.ipynb
new file mode 100644
index 0000000..a57e94e
--- /dev/null
+++ b/local_optimization/NLDF/elastic_net.ipynb
@@ -0,0 +1,535 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "0edec5c6-4a7c-41d2-8241-03deed2b6acd",
+ "metadata": {},
+ "source": [
+ "# Using elastic net in a linear regression to predict a possum's length\n",
+ "\n",
+ "The routine **[handle_solve_nldf](https://support.nag.com/numeric/nl/nagdoc_30.1/flhtml/e04/e04gnf.html)** is a general nonlinear data-fitting solver in the [NAG® Library](https://nag.com/nag-library/) that supports a variety of different loss functions and regularization options - including elastic net.\n",
+ "\n",
+ "**Elastic net** regularization is a combination of L1 (lasso) and L2 (ridge) regularization that is ideally suited for high-dimensional and noisy data. In these cases, it can be used for **feature selection** by setting the coefficients of irrelevant features to zero. Further, it is particularly useful when dealing with **multicollinear** features - where two or more features are highly correlated. It achieves this by shrinking the coefficients of correlated features towards each other. It also helps to **reduce overfitting** by penalizing large coefficients, which can lead to better generalization performance.\n",
+ "\n",
+ "To demonstrate the use of elastic net regularization, we will build a linear regression model to predict a possum's total length based upon several features, such as, capture site, age, and head length.\n",
+ "\n",
+ "\n",
+ "\n",
+ "**Reference:** \\\n",
+ "Lindenmayer, D. B., Viggers, K. L., Cunningham, R. B., and Donnelly, C. F. 1995. Morphological variation among columns of the mountain brushtail possum, Trichosurus caninus Ogilby (Phalangeridae: Marsupiala). Australian Journal of Zoology 43: 449-458. \\\n",
+ "Dataset source: https://www.kaggle.com/datasets/abrambeyer/openintro-possum"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "id": "fcd8009c-2402-4374-a125-87f7caf0ff01",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Import packages\n",
+ "import pandas as pd\n",
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "from naginterfaces.library import opt\n",
+ "from naginterfaces.base import utils\n",
+ "\n",
+ "# Set a random seed\n",
+ "np.random.seed(0)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "5411c4c6-2c14-40d1-86d0-c5e713f6a0ff",
+ "metadata": {},
+ "source": [
+ "## 1. Load and preprocess the data\n",
+ "This dataset has 13 features and 101 observations."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "id": "5497ff22-6a92-4cd5-a112-ed4a3250435b",
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "<div>\n",
+ "<style scoped>\n",
+ " .dataframe tbody tr th:only-of-type {\n",
+ " vertical-align: middle;\n",
+ " }\n",
+ "\n",
+ " .dataframe tbody tr th {\n",
+ " vertical-align: top;\n",
+ " }\n",
+ "\n",
+ " .dataframe thead th {\n",
+ " text-align: right;\n",
+ " }\n",
+ "</style>\n",
+ "<table border=\"1\" class=\"dataframe\">\n",
+ " <thead>\n",
+ " <tr style=\"text-align: right;\">\n",
+ " <th></th>\n",
+ " <th>site</th>\n",
+ " <th>Pop</th>\n",
+ " <th>sex</th>\n",
+ " <th>age</th>\n",
+ " <th>hdlngth</th>\n",
+ " <th>skullw</th>\n",
+ " <th>totlngth</th>\n",
+ " <th>taill</th>\n",
+ " <th>footlgth</th>\n",
+ " <th>earconch</th>\n",
+ " <th>eye</th>\n",
+ " <th>chest</th>\n",
+ " <th>belly</th>\n",
+ " </tr>\n",
+ " </thead>\n",
+ " <tbody>\n",
+ " <tr>\n",
+ " <th>0</th>\n",
+ " <td>1</td>\n",
+ " <td>Vic</td>\n",
+ " <td>m</td>\n",
+ " <td>8.0</td>\n",
+ " <td>94.1</td>\n",
+ " <td>60.4</td>\n",
+ " <td>89.0</td>\n",
+ " <td>36.0</td>\n",
+ " <td>74.5</td>\n",
+ " <td>54.5</td>\n",
+ " <td>15.2</td>\n",
+ " <td>28.0</td>\n",
+ " <td>36.0</td>\n",
+ " </tr>\n",
+ " <tr>\n",
+ " <th>1</th>\n",
+ " <td>1</td>\n",
+ " <td>Vic</td>\n",
+ " <td>f</td>\n",
+ " <td>6.0</td>\n",
+ " <td>92.5</td>\n",
+ " <td>57.6</td>\n",
+ " <td>91.5</td>\n",
+ " <td>36.5</td>\n",
+ " <td>72.5</td>\n",
+ " <td>51.2</td>\n",
+ " <td>16.0</td>\n",
+ " <td>28.5</td>\n",
+ " <td>33.0</td>\n",
+ " </tr>\n",
+ " <tr>\n",
+ " <th>2</th>\n",
+ " <td>1</td>\n",
+ " <td>Vic</td>\n",
+ " <td>f</td>\n",
+ " <td>6.0</td>\n",
+ " <td>94.0</td>\n",
+ " <td>60.0</td>\n",
+ " <td>95.5</td>\n",
+ " <td>39.0</td>\n",
+ " <td>75.4</td>\n",
+ " <td>51.9</td>\n",
+ " <td>15.5</td>\n",
+ " <td>30.0</td>\n",
+ " <td>34.0</td>\n",
+ " </tr>\n",
+ " <tr>\n",
+ " <th>3</th>\n",
+ " <td>1</td>\n",
+ " <td>Vic</td>\n",
+ " <td>f</td>\n",
+ " <td>6.0</td>\n",
+ " <td>93.2</td>\n",
+ " <td>57.1</td>\n",
+ " <td>92.0</td>\n",
+ " <td>38.0</td>\n",
+ " <td>76.1</td>\n",
+ " <td>52.2</td>\n",
+ " <td>15.2</td>\n",
+ " <td>28.0</td>\n",
+ " <td>34.0</td>\n",
+ " </tr>\n",
+ " <tr>\n",
+ " <th>4</th>\n",
+ " <td>1</td>\n",
+ " <td>Vic</td>\n",
+ " <td>f</td>\n",
+ " <td>2.0</td>\n",
+ " <td>91.5</td>\n",
+ " <td>56.3</td>\n",
+ " <td>85.5</td>\n",
+ " <td>36.0</td>\n",
+ " <td>71.0</td>\n",
+ " <td>53.2</td>\n",
+ " <td>15.1</td>\n",
+ " <td>28.5</td>\n",
+ " <td>33.0</td>\n",
+ " </tr>\n",
+ " </tbody>\n",
+ "</table>\n",
+ "</div>"
+ ],
+ "text/plain": [
+ " site Pop sex age hdlngth skullw totlngth taill footlgth earconch \\\n",
+ "0 1 Vic m 8.0 94.1 60.4 89.0 36.0 74.5 54.5 \n",
+ "1 1 Vic f 6.0 92.5 57.6 91.5 36.5 72.5 51.2 \n",
+ "2 1 Vic f 6.0 94.0 60.0 95.5 39.0 75.4 51.9 \n",
+ "3 1 Vic f 6.0 93.2 57.1 92.0 38.0 76.1 52.2 \n",
+ "4 1 Vic f 2.0 91.5 56.3 85.5 36.0 71.0 53.2 \n",
+ "\n",
+ " eye chest belly \n",
+ "0 15.2 28.0 36.0 \n",
+ "1 16.0 28.5 33.0 \n",
+ "2 15.5 30.0 34.0 \n",
+ "3 15.2 28.0 34.0 \n",
+ "4 15.1 28.5 33.0 "
+ ]
+ },
+ "execution_count": 2,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "df = pd.read_csv('possum.csv', usecols=range(1,14))\n",
+ "df.head()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "id": "06841ec7-443e-4831-b013-3aa79cc5333a",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Remove NAN data\n",
+ "df.dropna(axis=0,inplace=True)\n",
+ "\n",
+ "# Categorical features (site, population, sex) need to be one-hot encoded\n",
+ "df_encoded = pd.get_dummies(df, columns=['site','Pop','sex'], dtype=float, drop_first=True)\n",
+ "\n",
+ "# Extract total length (y), which is the variable to be predicted\n",
+ "y = df_encoded[[\"totlngth\"]].values\n",
+ "X = df_encoded.drop(columns=[\"totlngth\"]).values"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "b654fd99-5544-42d4-8467-b959b109f3cd",
+ "metadata": {},
+ "source": [
+ "## 2. Split the data into training and testing sets"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "id": "5c6e103d-4e2c-4640-b8d5-9f690c444192",
+ "metadata": {
+ "jupyter": {
+ "source_hidden": true
+ }
+ },
+ "outputs": [],
+ "source": [
+ "def train_test(X, y, test_size=0.2):\n",
+ " \"\"\"\n",
+ " Split dataset into training and testing sets.\n",
+ "\n",
+ " Parameters:\n",
+ " X (numpy array): Features\n",
+ " y (numpy array): Observations\n",
+ " test_size (float, optional): Proportion of data to use for testing\n",
+ "\n",
+ " Returns:\n",
+ " X_train, y_train, X_test, y_test\n",
+ " \"\"\"\n",
+ " # Get total number of samples\n",
+ " num_samples = X.shape[0]\n",
+ "\n",
+ " # Calculate number of test samples\n",
+ " num_test_samples = int(num_samples * test_size)\n",
+ "\n",
+ " # Generate random indices for training set\n",
+ " train_indices = np.random.choice(num_samples, num_samples - num_test_samples, replace=False)\n",
+ "\n",
+ " # Create training sets\n",
+ " X_train = X[train_indices]\n",
+ " y_train = y[train_indices]\n",
+ "\n",
+ " # Create testing sets\n",
+ " test_indices = np.setdiff1d(np.arange(num_samples), train_indices)\n",
+ " X_test = X[test_indices]\n",
+ " y_test = y[test_indices]\n",
+ "\n",
+ " return X_train, y_train, X_test, y_test"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "id": "86657422-9e33-45ff-899f-7f7840150914",
+ "metadata": {
+ "jupyter": {
+ "source_hidden": true
+ }
+ },
+ "outputs": [],
+ "source": [
+ "def scale_data(X_train, y_train, X_test, y_test):\n",
+ " \"\"\"\n",
+ " Scale the training and testing datasets.\n",
+ "\n",
+ " Returns:\n",
+ " X_train, y_train, X_test, y_test\n",
+ " \"\"\"\n",
+ " mu = X_train.mean(0)\n",
+ " sigma = X_train.std(0)\n",
+ " for j in range(X_train.shape[-1]):\n",
+ " xs = X_train[:,j]\n",
+ " is_categorical = np.logical_or(np.isclose(xs, 1.), np.isclose(xs, 0.)).all()\n",
+ " if not is_categorical:\n",
+ " X_train[:,j] = (X_train[:,j] - mu[j]) / sigma[j]\n",
+ " X_test[:,j] = (X_test[:,j] - mu[j]) / sigma[j]\n",
+ " \n",
+ " y_test = (y_test - y_train.mean()) / y_train.std()\n",
+ " y_train = (y_train - y_train.mean()) / y_train.std()\n",
+ "\n",
+ " return X_train, y_train, X_test, y_test\n",
+ " "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "id": "487c2e0f-f494-4ac3-8c34-1db320178693",
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Training set shapes: (81, 17) (81, 1)\n",
+ "Testing set shapes: (20, 17) (20, 1)\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Split data into training and testing sets\n",
+ "X_train, y_train, X_test, y_test = train_test(X, y)\n",
+ "\n",
+ "# Scale the data\n",
+ "X_train, y_train, X_test, y_test = scale_data(X_train, y_train, X_test, y_test)\n",
+ "\n",
+ "print(\"Training set shapes:\", X_train.shape, y_train.shape)\n",
+ "print(\"Testing set shapes:\", X_test.shape, y_test.shape)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7acd46ae-881f-4d5d-b174-0ee5a66703b7",
+ "metadata": {},
+ "source": [
+ "## 3. Fit a linear regression with least squares loss and elastic net regularization"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "id": "ad4796a5-724c-4a55-b485-4b18ce603328",
+ "metadata": {
+ "scrolled": true
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ " E04GN, Nonlinear Data-Fitting\n",
+ " Status: converged, an optimal solution found\n",
+ " Final objective value 1.652821E+01\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Number of variables = number of features + bias term\n",
+ "nvar = X_train.shape[1] + 1\n",
+ "\n",
+ "# Create a handle for the model\n",
+ "handle = opt.handle_init(nvar=nvar)\n",
+ "\n",
+ "# Register residual structure\n",
+ "nres = X_train.shape[0]\n",
+ "opt.handle_set_nlnls(handle, nres)\n",
+ "\n",
+ "# Create the data structure to be passed to the solver\n",
+ "data = {}\n",
+ "data[\"X_train\"] = X_train\n",
+ "data[\"y_train\"] = y_train\n",
+ "\n",
+ "# Define the residual callback function and its gradient\n",
+ "def lsqfun(x, nres, inform, data):\n",
+ " rx = np.zeros(nres, dtype=float)\n",
+ " X_train = data[\"X_train\"]\n",
+ " y_train = data[\"y_train\"].squeeze()\n",
+ " \n",
+ " # Fit a linear regression to the data\n",
+ " r_full = y_train - (x[0] + X_train @ x[1:]) \n",
+ " for i in range(nres):\n",
+ " rx[i] = r_full[i]\n",
+ " \n",
+ " return rx, inform\n",
+ " \n",
+ "def lsqgrd(x, nres, rdx, inform, data):\n",
+ " X_train = data[\"X_train\"]\n",
+ "\n",
+ " for i in range(nres):\n",
+ " for j in range(nvar):\n",
+ " if j==0:\n",
+ " rdx[i*nvar] = -1 \n",
+ " else:\n",
+ " rdx[i*nvar + j] = -X_train[i, j-1]\n",
+ " \n",
+ " return inform\n",
+ "\n",
+ "# Set loss function to l2-norm, elastic net regularization, and printing options\n",
+ "for option in [\n",
+ " 'NLDF Loss Function Type = L2',\n",
+ " 'Print Level = 1',\n",
+ " 'Print Options = No',\n",
+ " 'Reg Term Type = Elastic Net',\n",
+ " ]:\n",
+ " opt.handle_opt_set (handle, option)\n",
+ "\n",
+ "# Use an explicit I/O manager for abbreviated iteration output\n",
+ "iom = utils.FileObjManager(locus_in_output=False)\n",
+ "\n",
+ "# Set initial guess and solve\n",
+ "x = np.array([np.random.rand() for _ in range(nvar)])\n",
+ "\n",
+ "sol_en = opt.handle_solve_nldf(handle, lsqfun, lsqgrd, x, nres, data=data, io_manager=iom)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "id": "b84636fe-1a3c-4188-8dec-7449dbc059b2",
+ "metadata": {
+ "scrolled": true
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ " E04GN, Nonlinear Data-Fitting\n",
+ " Status: converged, an optimal solution found\n",
+ " Final objective value 1.362383E+01\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Resolve the problem with no regularization\n",
+ "opt.handle_opt_set(handle, 'Reg Term Type = Off')\n",
+ "sol_noreg = opt.handle_solve_nldf(handle, lsqfun, lsqgrd, x, nres, data=data, io_manager=iom)\n",
+ "\n",
+ "# Destroy the handle\n",
+ "opt.handle_free(handle)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "ada90e01-6845-41b8-9f91-9944497a94ed",
+ "metadata": {},
+ "source": [
+ "## 4. Compute root mean square error (RMSE)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 9,
+ "id": "126cca3a-2b1e-461b-af09-df70c8c636b8",
+ "metadata": {
+ "jupyter": {
+ "source_hidden": true
+ }
+ },
+ "outputs": [],
+ "source": [
+ "def calculate_rmse(y_actual, y_pred):\n",
+ " \"\"\"\n",
+ " Calculate the Root Mean Square Error (RMSE) between two lists of numbers\n",
+ "\n",
+ " Args:\n",
+ " y_actual (list): The actual values\n",
+ " y_pred (list): The predicted values\n",
+ "\n",
+ " Returns:\n",
+ " float: The Root Mean Squared Error\n",
+ " \"\"\"\n",
+ " return np.sqrt(np.square(y_actual - y_pred).mean())"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 10,
+ "id": "207f4ff1-b1c3-4c54-81ff-4789fa3bbad0",
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Using elastic net regularization decreased the RMSE by 0.034 compared to using no regularization.\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Calculate predicted values for y with elastic net regularization and find RMSE\n",
+ "y_pred_en = sol_en.x[0] + X_test @ sol_en.x[1:]\n",
+ "rmse_elastic_net = calculate_rmse(y_test, y_pred_en)\n",
+ "\n",
+ "# Calculate predicted values for y with no regularization and find RMSE\n",
+ "y_pred_noreg = sol_noreg.x[0] + X_test @ sol_noreg.x[1:]\n",
+ "rmse_noreg = calculate_rmse(y_test, y_pred_noreg)\n",
+ "\n",
+ "# Report the difference in RMSE\n",
+ "print(f\"Using elastic net regularization decreased the RMSE by {round(rmse_noreg - rmse_elastic_net, 4)} compared to using no regularization.\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "4dfdf65a-c992-4c48-ba6a-0a8438328218",
+ "metadata": {},
+ "source": [
+ "For more information on the NAG® Library and our [Optimization Modelling Suite](https://nag.com/mathematical-optimization/) or to try it for yourself, visit [‘Getting Started with the NAG Library’](https://support.nag.com/content/getting-started-nag-library?_gl=1*xmlppm*_gcl_au*MTEwNDczODM2NS4xNzIyMDAyNzkz*_ga*MjA2NzgxMjY0NS4xNzIyMDAyNzk0*_ga_6MCQDQP46G*MTcyMzEzNDUxNi41LjAuMTcyMzEzNDUzNy4zOS4wLjA.), select your configuration and language, download the software, request a trial key and experiment for yourself."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.10.8"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/local_optimization/NLDF/possum.csv b/local_optimization/NLDF/possum.csv
new file mode 100644
index 0000000..0f7cc0f
--- /dev/null
+++ b/local_optimization/NLDF/possum.csv
@@ -0,0 +1,105 @@
+case,site,Pop,sex,age,hdlngth,skullw,totlngth,taill,footlgth,earconch,eye,chest,belly
+1,1,Vic,m,8,94.1,60.4,89,36,74.5,54.5,15.2,28,36
+2,1,Vic,f,6,92.5,57.6,91.5,36.5,72.5,51.2,16,28.5,33
+3,1,Vic,f,6,94,60,95.5,39,75.4,51.9,15.5,30,34
+4,1,Vic,f,6,93.2,57.1,92,38,76.1,52.2,15.2,28,34
+5,1,Vic,f,2,91.5,56.3,85.5,36,71,53.2,15.1,28.5,33
+6,1,Vic,f,1,93.1,54.8,90.5,35.5,73.2,53.6,14.2,30,32
+7,1,Vic,m,2,95.3,58.2,89.5,36,71.5,52,14.2,30,34.5
+8,1,Vic,f,6,94.8,57.6,91,37,72.7,53.9,14.5,29,34
+9,1,Vic,f,9,93.4,56.3,91.5,37,72.4,52.9,15.5,28,33
+10,1,Vic,f,6,91.8,58,89.5,37.5,70.9,53.4,14.4,27.5,32
+11,1,Vic,f,9,93.3,57.2,89.5,39,77.2,51.3,14.9,31,34
+12,1,Vic,f,5,94.9,55.6,92,35.5,71.7,51,15.3,28,33
+13,1,Vic,m,5,95.1,59.9,89.5,36,71,49.8,15.8,27,32
+14,1,Vic,m,3,95.4,57.6,91.5,36,74.3,53.7,15.1,28,31.5
+15,1,Vic,m,5,92.9,57.6,85.5,34,69.7,51.8,15.7,28,35
+16,1,Vic,m,4,91.6,56,86,34.5,73,51.4,14.4,28,32
+17,1,Vic,f,1,94.7,67.7,89.5,36.5,73.2,53.2,14.7,29,31
+18,1,Vic,m,2,93.5,55.7,90,36,73.7,55.4,15.3,28,32
+19,1,Vic,f,5,94.4,55.4,90.5,35,73.4,53.9,15.2,28,32
+20,1,Vic,f,4,94.8,56.3,89,38,73.8,52.4,15.5,27,36
+21,1,Vic,f,3,95.9,58.1,96.5,39.5,77.9,52.9,14.2,30,40
+22,1,Vic,m,3,96.3,58.5,91,39.5,73.5,52.1,16.2,28,36
+23,1,Vic,f,4,92.5,56.1,89,36,72.8,53.3,15.4,28,35
+24,1,Vic,m,2,94.4,54.9,84,34,75,53.5,16.2,27,32
+25,1,Vic,m,3,95.8,58.5,91.5,35.5,72.3,51.6,14.9,31,35
+26,1,Vic,m,7,96,59,90,36,73.6,56.2,15,29,38
+27,1,Vic,f,2,90.5,54.5,85,35,70.3,50.8,14.2,23,28
+28,1,Vic,m,4,93.8,56.8,87,34.5,73.2,53,15.3,27,30
+29,1,Vic,f,3,92.8,56,88,35,74.9,51.8,14,24,32
+30,1,Vic,f,2,92.1,54.4,84,33.5,70.6,50.8,14.5,24.5,33
+31,1,Vic,m,3,92.8,54.1,93,37,68,52.5,14.5,27,31
+32,1,Vic,f,4,94.3,56.7,94,39,74.8,52,14.9,28,34
+33,1,Vic,m,3,91.4,54.6,89,37,70.8,51.8,14.8,24,30
+34,2,Vic,m,2,90.6,55.7,85.5,36.5,73.1,53.1,14.4,26,28.5
+35,2,Vic,m,4,94.4,57.9,85,35.5,71.2,55.5,16.4,28,35.5
+36,2,Vic,m,7,93.3,59.3,88,35,74.3,52,14.9,25.5,36
+37,2,Vic,f,2,89.3,54.8,82.5,35,71.2,52,13.6,28,31.5
+38,2,Vic,m,7,92.4,56,80.5,35.5,68.4,49.5,15.9,27,30
+39,2,Vic,f,1,84.7,51.5,75,34,68.7,53.4,13,25,25
+40,2,Vic,f,3,91,55,84.5,36,72.8,51.4,13.6,27,30
+41,2,Vic,f,5,88.4,57,83,36.5,NA,40.3,15.9,27,30.5
+42,2,Vic,m,3,85.3,54.1,77,32,62.7,51.2,13.8,25.5,33
+43,2,Vic,f,2,90,55.5,81,32,72,49.4,13.4,29,31
+44,2,Vic,m,NA,85.1,51.5,76,35.5,70.3,52.6,14.4,23,27
+45,2,Vic,m,3,90.7,55.9,81,34,71.5,54,14.6,27,31.5
+46,2,Vic,m,NA,91.4,54.4,84,35,72.8,51.2,14.4,24.5,35
+47,3,other,m,2,90.1,54.8,89,37.5,66,45.5,15,25,33
+48,3,other,m,5,98.6,63.2,85,34,66.9,44.9,17,28,35
+49,3,other,m,4,95.4,59.2,85,37,69,45,15.9,29.5,35.5
+50,3,other,f,5,91.6,56.4,88,38,65,47.2,14.9,28,36
+51,3,other,f,5,95.6,59.6,85,36,64,43.9,17.4,28,38.5
+52,3,other,m,6,97.6,61,93.5,40,67.9,44.3,15.8,28.5,32.5
+53,3,other,f,3,93.1,58.1,91,38,67.4,46,16.5,26,33.5
+54,4,other,m,7,96.9,63,91.5,43,71.3,46,17.5,30,36.5
+55,4,other,m,2,103.1,63.2,92.5,38,72.5,44.9,16.4,30.5,36
+56,4,other,m,3,99.9,61.5,93.7,38,68.7,46.8,16.4,27.5,31.5
+57,4,other,f,4,95.1,59.4,93,41,67.2,45.3,14.5,31,39
+58,4,other,m,3,94.5,64.2,91,39,66.5,46.4,14.4,30.5,33
+59,4,other,m,2,102.5,62.8,96,40,73.2,44.5,14.7,32,36
+60,4,other,f,2,91.3,57.7,88,39,63.1,47,14.4,26,30
+61,5,other,m,7,95.7,59,86,38,63.1,44.9,15,26.5,31
+62,5,other,f,3,91.3,58,90.5,39,65.5,41.3,16,27,32
+63,5,other,f,6,92,56.4,88.5,38,64.1,46.3,15.2,25.5,28.5
+64,5,other,f,3,96.9,56.5,89.5,38.5,63,45.1,17.1,25.5,33
+65,5,other,f,5,93.5,57.4,88.5,38,68.2,41.7,14,29,38.5
+66,5,other,f,3,90.4,55.8,86,36.5,63.2,44.2,15.7,26.5,34
+67,5,other,m,4,93.3,57.6,85,36.5,64.7,44.1,16.5,27.5,29.5
+68,5,other,m,5,94.1,56,88.5,38,65.9,43.1,17.4,27,30
+69,5,other,m,5,98,55.6,88,37.5,65,45.6,15,28.5,34
+70,5,other,f,7,91.9,56.4,87,38,65.4,44.1,13,27,34
+71,5,other,m,6,92.8,57.6,90,40,65.7,42.8,15,27.5,34
+72,5,other,m,1,85.9,52.4,80.5,35,62,42.4,14.1,25.5,30
+73,5,other,m,1,82.5,52.3,82,36.5,65.7,44.7,16,23.5,28
+74,6,other,f,4,88.7,52,83,38,61.5,45.9,14.7,26,34
+75,6,other,m,6,93.8,58.1,89,38,66.2,45.6,16.9,26,33.5
+76,6,other,m,5,92.4,56.8,89,41,64.5,46.4,17.8,26,33
+77,6,other,m,6,93.6,56.2,84,36,62.8,42.9,16.2,25,35
+78,6,other,m,1,86.5,51,81,36.5,63,44.3,13.2,23,28
+79,6,other,m,1,85.8,50,81,36.5,62.8,43,14.8,22,28.5
+80,6,other,m,1,86.7,52.6,84,38,62.3,44.8,15,23.5,30.5
+81,6,other,m,3,90.6,56,85.5,38,65.6,41.7,17,27.5,35
+82,6,other,f,4,86,54,82,36.5,60.7,42.9,15.4,26,32
+83,6,other,f,3,90,53.8,81.5,36,62,43.3,14,25,29
+84,6,other,m,3,88.4,54.6,80.5,36,62.6,43.6,16.3,25,28.5
+85,6,other,m,3,89.5,56.2,92,40.5,65.6,43.5,14.5,27,31.5
+86,6,other,f,3,88.2,53.2,86.5,38.5,60.3,43.7,13.6,26,31
+87,7,other,m,2,98.5,60.7,93,41.5,71.7,46.8,15,26,36
+88,7,other,f,2,89.6,58,87.5,38,66.7,43.5,16,25.5,31.5
+89,7,other,m,6,97.7,58.4,84.5,35,64.4,46.2,14.4,29,30.5
+90,7,other,m,3,92.6,54.6,85,38.5,69.8,44.8,14.5,25.5,32.5
+91,7,other,m,3,97.8,59.6,89,38,65.5,48,15,26,32
+92,7,other,m,2,90.7,56.3,85,37,67.6,46.8,14.5,25.5,31
+93,7,other,m,3,89.2,54,82,38,63.8,44.9,12.8,24,31
+94,7,other,m,7,91.8,57.6,84,35.5,64.2,45.1,14.4,29,35
+95,7,other,m,4,91.6,56.6,88.5,37.5,64.5,45.4,14.9,27,31
+96,7,other,m,4,94.8,55.7,83,38,66.5,47.7,14,25,33
+97,7,other,m,3,91,53.1,86,38,63.8,46,14.5,25,31.5
+98,7,other,m,5,93.2,68.6,84,35,65.6,44.3,14.5,28.5,32
+99,7,other,f,3,93.3,56.2,86.5,38.5,64.8,43.8,14,28,35
+100,7,other,m,1,89.5,56,81.5,36.5,66,46.8,14.8,23,27
+101,7,other,m,1,88.6,54.7,82.5,39,64.4,48,14,25,33
+102,7,other,f,6,92.4,55,89,38,63.5,45.4,13,25,30
+103,7,other,m,4,91.5,55.2,82.5,36.5,62.9,45.9,15.4,25,29
+104,7,other,f,3,93.6,59.9,89,40,67.6,46,14.8,28.5,33.5
From b34a78a44470bf7f27a6e5c4e8ab3b496c8785ea Mon Sep 17 00:00:00 2001
From: Charlotte Silvester <charlottelsilvester@gmail.com>
Date: Mon, 12 Aug 2024 16:13:30 +0100
Subject: [PATCH 2/7] Change url in elastic net notebook
---
local_optimization/NLDF/elastic_net.ipynb | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/local_optimization/NLDF/elastic_net.ipynb b/local_optimization/NLDF/elastic_net.ipynb
index a57e94e..6d46334 100644
--- a/local_optimization/NLDF/elastic_net.ipynb
+++ b/local_optimization/NLDF/elastic_net.ipynb
@@ -7,7 +7,7 @@
"source": [
"# Using elastic net in a linear regression to predict a possum's length\n",
"\n",
- "The routine **[handle_solve_nldf](https://support.nag.com/numeric/nl/nagdoc_30.1/flhtml/e04/e04gnf.html)** is a general nonlinear data-fitting solver in the [NAG® Library](https://nag.com/nag-library/) that supports a variety of different loss functions and regularization options - including elastic net.\n",
+ "The routine **[handle_solve_nldf](https://support.nag.com/numeric/py/nagdoc_latest/naginterfaces.library.opt.handle_solve_nldf.html)** is a general nonlinear data-fitting solver in the [NAG® Library](https://nag.com/nag-library/) that supports a variety of different loss functions and regularization options - including elastic net.\n",
"\n",
"**Elastic net** regularization is a combination of L1 (lasso) and L2 (ridge) regularization that is ideally suited for high-dimensional and noisy data. In these cases, it can be used for **feature selection** by setting the coefficients of irrelevant features to zero. Further, it is particularly useful when dealing with **multicollinear** features - where two or more features are highly correlated. It achieves this by shrinking the coefficients of correlated features towards each other. It also helps to **reduce overfitting** by penalizing large coefficients, which can lead to better generalization performance.\n",
"\n",
From 493e198d18f858ae9c4a02803f2d52155fa67178 Mon Sep 17 00:00:00 2001
From: Charlotte Silvester <charlottelsilvester@gmail.com>
Date: Mon, 12 Aug 2024 16:47:17 +0100
Subject: [PATCH 3/7] Add e04gn info to elastic net notebook
---
local_optimization/NLDF/elastic_net.ipynb | 1 +
1 file changed, 1 insertion(+)
diff --git a/local_optimization/NLDF/elastic_net.ipynb b/local_optimization/NLDF/elastic_net.ipynb
index 6d46334..9ec0155 100644
--- a/local_optimization/NLDF/elastic_net.ipynb
+++ b/local_optimization/NLDF/elastic_net.ipynb
@@ -13,6 +13,7 @@
"\n",
"To demonstrate the use of elastic net regularization, we will build a linear regression model to predict a possum's total length based upon several features, such as, capture site, age, and head length.\n",
"\n",
+ "Note, the purpose of this notebook is to illustrate the use of e04gn which is a general data-fitting framework that utilises nonlinear programming algorithms, such as sequential quadratic programming and interior point method. Therefore, it may not be as performant as one of our dedicated linear regression solvers."
"\n",
"\n",
"**Reference:** \\\n",
From dbf531e1b8b3bc4e9e851bf30b78bbb09137538d Mon Sep 17 00:00:00 2001
From: Charlotte Silvester <charlottelsilvester@gmail.com>
Date: Mon, 12 Aug 2024 16:47:52 +0100
Subject: [PATCH 4/7] Fix typo
---
local_optimization/NLDF/elastic_net.ipynb | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/local_optimization/NLDF/elastic_net.ipynb b/local_optimization/NLDF/elastic_net.ipynb
index 9ec0155..cc85b7d 100644
--- a/local_optimization/NLDF/elastic_net.ipynb
+++ b/local_optimization/NLDF/elastic_net.ipynb
@@ -13,7 +13,7 @@
"\n",
"To demonstrate the use of elastic net regularization, we will build a linear regression model to predict a possum's total length based upon several features, such as, capture site, age, and head length.\n",
"\n",
- "Note, the purpose of this notebook is to illustrate the use of e04gn which is a general data-fitting framework that utilises nonlinear programming algorithms, such as sequential quadratic programming and interior point method. Therefore, it may not be as performant as one of our dedicated linear regression solvers."
+ "Note, the purpose of this notebook is to illustrate the use of e04gn which is a general data-fitting framework that utilises nonlinear programming algorithms, such as sequential quadratic programming and interior point method. Therefore, it may not be as performant as one of our dedicated linear regression solvers.\n",
"\n",
"\n",
"**Reference:** \\\n",
From 081bc8fb5b9879aef3f9c861e0cca1b66b7f20f2 Mon Sep 17 00:00:00 2001
From: Charlotte Silvester <charlottelsilvester@gmail.com>
Date: Mon, 12 Aug 2024 17:38:36 +0100
Subject: [PATCH 5/7] fix name
---
local_optimization/NLDF/elastic_net.ipynb | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/local_optimization/NLDF/elastic_net.ipynb b/local_optimization/NLDF/elastic_net.ipynb
index cc85b7d..d52cbb3 100644
--- a/local_optimization/NLDF/elastic_net.ipynb
+++ b/local_optimization/NLDF/elastic_net.ipynb
@@ -13,7 +13,7 @@
"\n",
"To demonstrate the use of elastic net regularization, we will build a linear regression model to predict a possum's total length based upon several features, such as, capture site, age, and head length.\n",
"\n",
- "Note, the purpose of this notebook is to illustrate the use of e04gn which is a general data-fitting framework that utilises nonlinear programming algorithms, such as sequential quadratic programming and interior point method. Therefore, it may not be as performant as one of our dedicated linear regression solvers.\n",
+ "Note, the purpose of this notebook is to illustrate the use of handle_solve_nldf which is a general data-fitting framework that utilises nonlinear programming algorithms, such as sequential quadratic programming and interior point method. Therefore, it may not be as performant as one of our dedicated linear regression solvers.\n",
"\n",
"\n",
"**Reference:** \\\n",
From 9f0003ec5c9cc2279d9e4496b75a41eb6d346611 Mon Sep 17 00:00:00 2001
From: Markus Towara <markus.towara@nag.co.uk>
Date: Mon, 2 Sep 2024 13:24:32 +0200
Subject: [PATCH 6/7] Update README.md
Fix broken links
---
README.md | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/README.md b/README.md
index 6449762..168e1e3 100644
--- a/README.md
+++ b/README.md
@@ -10,7 +10,7 @@
# Examples using the NAG Library for Python <a name=examples></a>
-This repository contains examples and demonstrations using the [NAG Library for Python](https://www.nag.com/nag-library-python). The NAG Library for Python contains 1900+ functions spanning many areas of numerical computing and data science.
+This repository contains examples and demonstrations using the [NAG Library for Python](https://nag.com/nag-library/). The NAG Library for Python contains 1900+ functions spanning many areas of numerical computing and data science.
Designed to work alongside the open source Python packages, [Numpy](http://www.numpy.org/) and [Scipy](https://www.scipy.org/), The NAG Library for Python can augment your computational workflow in many areas.
@@ -106,7 +106,7 @@ The **two** important bits are the
**Note** that the **product code** and **KUSARI ID** can be different from the previous example.
- With these, you are set to [contact NAG and request a trial licence](https://www.nag.com/content/software-trials?product=NAG%20Library).
+ With these, you are set to [contact NAG and request a trial licence](https://nag.com/contact-us/).
The trial licence is a plain text chunk similar to
```
@@ -268,7 +268,6 @@ The following links take you to the relevant section in the official documentati
# Useful links<a name=links></a>
* [NAG Library for Python Documentation](https://www.nag.com/numeric/py/nagdoc_latest/naginterfaces.library.opt.html)
-* [Request a trial licence](https://www.nag.com/content/software-trials?product=NAG%20Library)
* [Kusari licence module Documentation](https://www.nag.com/numeric/py/nagdoc_latest/naginterfaces.kusari.html#kusari)
From e150706309cf07a34b09e778fd31cb01d52066cf Mon Sep 17 00:00:00 2001
From: Caleb Hamilton <caleb.hamilton@nag.com>
Date: Thu, 14 Nov 2024 09:40:25 -0600
Subject: [PATCH 7/7] Update README.md
brand updates
---
README.md | 40 ++++++++++++++++++++--------------------
1 file changed, 20 insertions(+), 20 deletions(-)
diff --git a/README.md b/README.md
index 168e1e3..8411961 100644
--- a/README.md
+++ b/README.md
@@ -1,18 +1,18 @@
-<a name=top></a>
+<a name=top></a>
# Content<a name=content></a>
-* [Examples using the NAG Library for Python](#examples)
-* [How to install the NAG Library for Python](#install)
+* [Examples using the *n*AG Library for Python](#examples)
+* [How to install the *n*AG Library for Python](#install)
* [How to run the Jupyter notebook examples](#jupyter)
-* [List of Chapters in the NAG Library for Python](#chapters)
+* [List of Chapters in the *n*AG Library for Python](#chapters)
* [Useful links](#links)
-# Examples using the NAG Library for Python <a name=examples></a>
+# Examples using the *n*AG Library for Python <a name=examples></a>
-This repository contains examples and demonstrations using the [NAG Library for Python](https://nag.com/nag-library/). The NAG Library for Python contains 1900+ functions spanning many areas of numerical computing and data science.
+This repository contains examples and demonstrations using the [*n*AG Library for Python](https://nag.com/nag-library/). The *n*AG Library for Python contains 1900+ functions spanning many areas of numerical computing and data science.
-Designed to work alongside the open source Python packages, [Numpy](http://www.numpy.org/) and [Scipy](https://www.scipy.org/), The NAG Library for Python can augment your computational workflow in many areas.
+Designed to work alongside the open source Python packages, [Numpy](http://www.numpy.org/) and [Scipy](https://www.scipy.org/), The *n*AG Library for Python can augment your computational workflow in many areas.
## Directory of GitHub examples
@@ -31,26 +31,26 @@ Designed to work alongside the open source Python packages, [Numpy](http://www.n
## Examples that ship with the product
-In addition to those presented here, The NAG Library for Python ships with a set of usage examples. To see them all, run the following command
+In addition to those presented here, The *n*AG Library for Python ships with a set of usage examples. To see them all, run the following command
```
python -m naginterfaces.library.examples --locate
```
-# How to install the NAG Library for Python<a name=install></a>
+# How to install the *n*AG Library for Python<a name=install></a>
-In this section we illustrate how to install the NAG Library for Python, request a Trial Licence and make sure the Library is working. Details and further information regarding the installation can be found [here](https://www.nag.com/numeric/py/nagdoc_latest/readme.html#installation).
+In this section we illustrate how to install the *n*AG Library for Python, request a Trial Licence and make sure the Library is working. Details and further information regarding the installation can be found [here](https://www.nag.com/numeric/py/nagdoc_latest/readme.html#installation).
**Note** Before starting make sure you have access to a host that has Python 3 (3.4 or more recent).
### Step 1. Downloading and installing
-Installing the NAG Library is done using the `pip` package manager, fire-up a terminal and create a Python 3 virtual environment where to install and test the NAG Library
+Installing the *n*AG Library is done using the `pip` package manager, fire-up a terminal and create a Python 3 virtual environment where to install and test the *n*AG Library
```{bash}
guest@nag-37:~$ python3 -m venv nag3
guest@nag-37:~$ . nag3/bin/activate
(nag3) guest@nag-37:~$
```
-Now use `pip` to install the NAG Library for Python
+Now use `pip` to install the *n*AG Library for Python
```{bash}
(nag3) guest@nag-37:~$ python -m pip install --extra-index-url https://www.nag.com/downloads/py/naginterfaces_nag naginterfaces
```
@@ -79,7 +79,7 @@ The next step is to get the licensing info (**product code** and **KUSARI ID**)
```
The output should be similar to
```
-The NAG Library for Python on this platform uses
+The *n*AG Library for Python on this platform uses
underlying Library NLL6I271VL.
This Library has been installed as part of the package
and it requires a valid licence key.
@@ -106,7 +106,7 @@ The **two** important bits are the
**Note** that the **product code** and **KUSARI ID** can be different from the previous example.
- With these, you are set to [contact NAG and request a trial licence](https://nag.com/contact-us/).
+ With these, you are set to [contact *n*AG and request a trial licence](https://nag.com/contact-us/).
The trial licence is a plain text chunk similar to
```
@@ -116,8 +116,8 @@ The **two** important bits are the
The final step is to make sure the licence is valid and the library is working as expected.
-### Step 3. Testing the NAG Library
-The last step is to make sure the licence was correctly stored and that the NAG Library is working correctly. From the same virtual terminal re-run the Kusari licence module
+### Step 3. Testing the *n*AG Library
+The last step is to make sure the licence was correctly stored and that the *n*AG Library is working correctly. From the same virtual terminal re-run the Kusari licence module
```{bash}
(nag3) guest@nag-37:~$ python -m naginterfaces.kusari
```
@@ -165,7 +165,7 @@ Run `python -m naginterfaces.library.examples --help` to see any additional usag
# How to run the Jupyter notebook examples<a name=jupyter></a>
This section briefly illustrates how to setup a host in order to open and run the [Jupyter notebooks](https://jupyter.org/) provided in this repository.
-Before running the notebooks make sure the [NAG Library is installed and working](#install). Before starting, it is advised to read [Jupyter's installation page](https://jupyter.org/install.html).
+Before running the notebooks make sure the [*n*AG Library is installed and working](#install). Before starting, it is advised to read [Jupyter's installation page](https://jupyter.org/install.html).
<!-- You can [view a static render of the notebooks using Jupyter's nbviewer here](https://nbviewer.jupyter.org/github/numericalalgorithmsgroup/NAGPythonExamples/tree/master/local_optimization/)
[](https://nbviewer.jupyter.org/github/numericalalgorithmsgroup/NAGPythonExamples/tree/master/local_optimization/)
@@ -174,7 +174,7 @@ or alternatively use [Binder](https://mybinder.org/) to [view them here](https:/
### Installing Jupyter notebook
-To install Jupyter, launch a terminal and activate the virtual environment used to install the NAG Library for Python
+To install Jupyter, launch a terminal and activate the virtual environment used to install the *n*AG Library for Python
```{bash}
guest@nag-37:~$ . nag3/bin/activate
(nag3) guest@nag-37:~$ pip install notebook matplotlib
@@ -211,7 +211,7 @@ This command will fire-up your web browser and open the `rosenbrock2d.ipynb` not
-# List of Chapters in the NAG Library for Python<a name=chapters></a>
+# List of Chapters in the *n*AG Library for Python<a name=chapters></a>
The following links take you to the relevant section in the official documentation
@@ -267,7 +267,7 @@ The following links take you to the relevant section in the official documentati
# Useful links<a name=links></a>
-* [NAG Library for Python Documentation](https://www.nag.com/numeric/py/nagdoc_latest/naginterfaces.library.opt.html)
+* [*n*AG Library for Python Documentation](https://www.nag.com/numeric/py/nagdoc_latest/naginterfaces.library.opt.html)
* [Kusari licence module Documentation](https://www.nag.com/numeric/py/nagdoc_latest/naginterfaces.kusari.html#kusari)