From 1d87e895e4dfcd2c8092704a66b850935487d06e Mon Sep 17 00:00:00 2001 From: sspencerwire Date: Mon, 5 Aug 2024 10:09:12 -0500 Subject: [PATCH] Reorganize web services book * correct some content that was incorrect * mostly this PR is focused on reorganization, more editing will be required --- docs/books/web_services/01-files-servers.md | 7 +- .../web_services/02-web-servers-intro.md | 92 +++ ...b-servers.md => 021-web-servers-apache.md} | 88 +-- .../web_services/022-web-servers-nginx.md | 10 + .../web_services/03-application-servers.md | 12 +- .../books/web_services/04-database-servers.md | 9 + ...ers.md => 041-database-servers-mariadb.md} | 558 +----------------- .../042-database-servers-mysql.md | 133 +++++ .../043-database-servers-replication.md | 427 ++++++++++++++ 9 files changed, 692 insertions(+), 644 deletions(-) create mode 100644 docs/books/web_services/02-web-servers-intro.md rename docs/books/web_services/{02-web-servers.md => 021-web-servers-apache.md} (85%) create mode 100644 docs/books/web_services/022-web-servers-nginx.md create mode 100644 docs/books/web_services/04-database-servers.md rename docs/books/web_services/{04-databases-servers.md => 041-database-servers-mariadb.md} (65%) create mode 100644 docs/books/web_services/042-database-servers-mysql.md create mode 100644 docs/books/web_services/043-database-servers-replication.md diff --git a/docs/books/web_services/01-files-servers.md b/docs/books/web_services/01-files-servers.md index 2155ceb88c..7e9415d1be 100644 --- a/docs/books/web_services/01-files-servers.md +++ b/docs/books/web_services/01-files-servers.md @@ -6,6 +6,11 @@ tags: - ftp - sftp --- + +!!! info + + The content for this page has yet to be written. + \ No newline at end of file +--> diff --git a/docs/books/web_services/02-web-servers-intro.md b/docs/books/web_services/02-web-servers-intro.md new file mode 100644 index 0000000000..e7ae6a950b --- /dev/null +++ b/docs/books/web_services/02-web-servers-intro.md @@ -0,0 +1,92 @@ +--- +author: Antoine Le Morvan +contributors: Steven Spencer, Ganna Zhyrnova +title: Part 2. Web Servers Introduction +--- + +## Introduction + +### HTTP protocol + +**HTTP** (**H**yper**T**ext **T**ransfer **P**rotocol) has been the most widely used protocol on the Internet since 1990. + +This protocol enables the transfer of files (mainly in HTML format, but also in CSS, JS, AVI...) localized by a character string called **URL** between a browser (the client) and a Web server (called `httpd` on UNIX machines). + +HTTP is a "request-response" protocol operating on top of **TCP** (**T**ransmission **C**ontrol **P**rotocol). + +1. The client opens a TCP connection to the server and sends a request. +2. The server analyzes the request and responds according to its configuration. + +The HTTP protocol is "**STATELESS**": it does not retain any information about the client's state from one request to the next. Dynamic languages such as php, python, or java store client session information in memory (as on an e-commerce site, for example). + +The current HTTP protocols are version 1.1, used widely, and versions 2 and 3 which are gaining adoption. + +An HTTP response is a set of lines sent to the browser by the server. It includes: + +* A **status line**: this specifies the protocol version used and the processing status of the request, using a code and explanatory text. The line comprises three elements separated with a space: + * The protocol version used + * The status code + * The meaning of the code + +* **Response header fields**: these are a set of optional lines providing additional information about the response and/or the server. Each of these lines consists of a name qualifying the header type, followed by a colon (:) and the header value. + +* **The response body**: this contains the requested document. + +Here is an example of an HTTP response: + +```bash +$ curl --head --location https://docs.rockylinux.org +HTTP/2 200 +accept-ranges: bytes +access-control-allow-origin: * +age: 109725 +cache-control: public, max-age=0, must-revalidate +content-disposition: inline +content-type: text/html; charset=utf-8 +date: Fri, 21 Jun 2024 12:05:24 GMT +etag: "cba6b533f892339d3818dc59c3a5a69a" +server: Vercel +strict-transport-security: max-age=63072000 +x-vercel-cache: HIT +x-vercel-id: cdg1::pdqbh-1718971524213-4892bf82d7b2 +content-length: 154696 +``` + +!!! NOTE + + Learning the `curl` command usage will be very helpfull for you to troubleshoot your servers in the future. + +The role of the web server is to translate a URL into a local resource. Consulting the page is like sending an HTTP request to this machine. The DNS service plays an essential role. + +### URLs + +A **URL** (**U**niform **R**esource **L**ocator) is an ASCII character string used to designate resources on the Internet. It is informally referred to as a web address. + +A URL has three parts: + +```text +://:/ +``` + +* **Protocol name**: this is the language used to communicate over the network, for example HTTP, HTTPS, FTP, and so on. The most widely used protocols are HTTP (HyperText TransferProtocol) and its secure version HTTPS, the protocol used to exchange Web pages in HTML format. + +* **Login** and **password**: allows you to specify access parameters to a secure server. This option is not recommended, as the password is visible in the URL (for security purposes). + +* **Host**: This is the name of the computer hosting the requested resource. Note that it is possible to use the server's IP address, which makes the URL less readable. + +* **Port number**: this is a number associated with a service, enabling the server to know the requested resource type. The default port associated with the HTTP protocol is port number 80 and 443 with HTTPS. So, when the protocol in use is HTTP or HTTPS +, the port number is optional. + +* Resource path: This part lets the server know the location of the resource. Generally, the location (directory) and name of the requested file. If nothing in the address specifies a location, it indicates the first page of the host. Otherwise it indicates the path to the page to display. + +### Ports + +An HTTP request will arrive on port 80 (default port for http) of the server running on the host. However, the administrator is free to choose the server's listening port. + +The http protocol is available in a secure version: the https protocol (port 443). Implement this encrypted protocol with the `mod_ssl` module. + +Using other ports is also possible, such as port `8080` (Java EE application servers). + +## Apache and Nginx + +The two most common web servers for Linux are Apache and Nginx. These will be discussed in the following chapters. diff --git a/docs/books/web_services/02-web-servers.md b/docs/books/web_services/021-web-servers-apache.md similarity index 85% rename from docs/books/web_services/02-web-servers.md rename to docs/books/web_services/021-web-servers-apache.md index bf14c19642..f9becb49c7 100644 --- a/docs/books/web_services/02-web-servers.md +++ b/docs/books/web_services/021-web-servers-apache.md @@ -1,91 +1,9 @@ --- author: Antoine Le Morvan contributors: Steven Spencer, Ganna Zhyrnova -title: Part 2. Web Servers +title: Part 2.1 Web Servers Apache --- -## Introduction - -### HTTP Protocol - -**HTTP** (**H**yper**T**ext **T**ransfer **P**rotocol) has been the most widely used protocol on the Internet since 1990. - -This protocol enables the transfer of files (mainly in HTML format, but also in CSS, JS, AVI...) localized by a character string called **URL** between a browser (the client) and a Web server (called `httpd` on UNIX machines). - -HTTP is a "request-response" protocol operating on top of **TCP** (**T**ransmission **C**ontrol **P**rotocol). - -1. The client opens a TCP connection to the server and sends a request. -2. The server analyzes the request and responds according to its configuration. - -The HTTP protocol is "**STATELESS**": it does not retain any information about the client's state from one request to the next. Dynamic languages such as php, python, or java store client session information in memory (as on an e-commerce site, for example). - -The HTTP protocol is version 1.1. Version 2 is still under development. - -An HTTP response is a set of lines sent to the browser by the server. It includes: - -* A **status line**: this specifies the protocol version used and the processing status of the request, using a code and explanatory text. The line comprises three elements separated with a space: - * The protocol version used - * The status code - * The meaning of the code - -* **Response header fields**: these are a set of optional lines providing additional information about the response and/or the server. Each of these lines consists of a name qualifying the header type, followed by a colon (:) and the header value. - -* **The response body**: this contains the requested document. - -Here is an example of an HTTP response: - -```bash -$ curl --head --location https://docs.rockylinux.org -HTTP/2 200 -accept-ranges: bytes -access-control-allow-origin: * -age: 109725 -cache-control: public, max-age=0, must-revalidate -content-disposition: inline -content-type: text/html; charset=utf-8 -date: Fri, 21 Jun 2024 12:05:24 GMT -etag: "cba6b533f892339d3818dc59c3a5a69a" -server: Vercel -strict-transport-security: max-age=63072000 -x-vercel-cache: HIT -x-vercel-id: cdg1::pdqbh-1718971524213-4892bf82d7b2 -content-length: 154696 -``` - -!!! NOTE - - Learning the `curl` command usages will be very helpfull for you to troubleshoot your servers in the future. - -The role of the web server is to translate a URL into a local resource. Consulting the page is like sending an HTTP request to this machine. The DNS service therefore plays an essential role. - -### URLs - -A **URL** (**U**niform **R**esource **L**ocator) is an ASCII character string used to designate resources on the Internet. It is informally referred to as a web address. - -A URL has three parts: - -```text -://:/ -``` - -* **Protocol name**: this is the language used to communicate over the network, for example HTTP, HTTPS, FTP, and so on. The most widely used protocols are HTTP (HyperText TransferProtocol) and its secure version HTTPS, the protocol used to exchange Web pages in HTML format. - -* **Login** and **password**: allows you to specify access parameters to a secure server. This option is not recommended, as the password is visible in the URL (for security purposes). - -* **Host**: This is the name of the computer hosting the requested resource. Note that it is possible to use the server's IP address, which makes the URL less readable. - -* **Port number**: this is a number associated with a service, enabling the server to know the requested resource type. The default port associated with the HTTP protocol is port number 80 and 443 with HTTPS. So, when the protocol in use is HTTP or HTTPS, the port number is optional. - -* Resource path: This part lets the server know the location of the resource. Generally, the location (directory) and name of the requested file. If nothing in the address specifies a location, it indicates the first page of the host. Otherwise it indicates the path to the page to display. - -### Ports - -An HTTP request will arrive on port 80 (default port for http) of the server running on the host. However, the administrator is free to choose the server's listening port. - -The http protocol is available in a secure version: the https protocol (port 443). Implement this encrypted protocol with the `mod_ssl` module. - -Using other ports is also possible, such as port `8080` (Java EE application servers). - ## Apache In this chapter, you will learn about Apache, the web server. @@ -98,8 +16,8 @@ In this chapter, you will learn about Apache, the web server. :checkered_flag: **apache**, **http**, **httpd** -**Knowledge**: :star: :star: -**Complexity**: :star: :star: +**Knowledge**: :star: :star: +**Complexity**: :star: :star: **Reading time**: 30 minutes diff --git a/docs/books/web_services/022-web-servers-nginx.md b/docs/books/web_services/022-web-servers-nginx.md new file mode 100644 index 0000000000..8d2ab49b2f --- /dev/null +++ b/docs/books/web_services/022-web-servers-nginx.md @@ -0,0 +1,10 @@ +--- +author: +contributors: +title: Part 2.2 Web Servers Nginx +--- + +!!! info + + This content is not written yet. + diff --git a/docs/books/web_services/03-application-servers.md b/docs/books/web_services/03-application-servers.md index 842e53a22e..a67ffde97c 100644 --- a/docs/books/web_services/03-application-servers.md +++ b/docs/books/web_services/03-application-servers.md @@ -8,7 +8,7 @@ tags: - php-fpm - application server - dynamic language ---- +--- ## PHP and PHP-FPM @@ -22,14 +22,14 @@ In this chapter, you will learn about PHP and PHP-FPM. **Objectives**: In this chapter, you will learn how to: -:heavy_check_mark: install a PHP application server -:heavy_check_mark: configure PHP-FPM pool -:heavy_check_mark: optimize a PHP-FPM application server +:heavy_check_mark: install a PHP application server +:heavy_check_mark: configure PHP-FPM pool +:heavy_check_mark: optimize a PHP-FPM application server :checkered_flag: **PHP**, **PHP-FPM**, **Application server** -**Knowledge**: :star: :star: :star: -**Complexity**: :star: :star: :star: +**Knowledge**: :star: :star: :star: +**Complexity**: :star: :star: :star: **Reading time**: 30 minutes diff --git a/docs/books/web_services/04-database-servers.md b/docs/books/web_services/04-database-servers.md new file mode 100644 index 0000000000..eaa6878349 --- /dev/null +++ b/docs/books/web_services/04-database-servers.md @@ -0,0 +1,9 @@ +--- +author: +contributors: +title: Part 4. Database Servers +--- + +!!! info + + An introduction to relational database servers needs to be written. diff --git a/docs/books/web_services/04-databases-servers.md b/docs/books/web_services/041-database-servers-mariadb.md similarity index 65% rename from docs/books/web_services/04-databases-servers.md rename to docs/books/web_services/041-database-servers-mariadb.md index 44785cfabc..3d6a5c7062 100644 --- a/docs/books/web_services/04-databases-servers.md +++ b/docs/books/web_services/041-database-servers-mariadb.md @@ -6,7 +6,7 @@ tags: - mysql - database - rdbms -title: Part 4. Databases servers +title: Part 4.1 Database servers MariaDB --- MySQL, MariaDB and PostgreSQL are open-source RDBMS (Relational DataBase Management System). @@ -19,13 +19,13 @@ In this chapter, you will learn about the RDBMS MariaDB and MySQL. **Objectives**: In this chapter, you will learn how to: -:heavy_check_mark: install, configure, and secure MariaDB server and MySQL server; -:heavy_check_mark: perform some administrative actions on databases and users. +:heavy_check_mark: install, configure, and secure MariaDB server and MySQL server; +:heavy_check_mark: perform some administrative actions on databases and users. :checkered_flag: **RDBMS**, **database**, **MariaDB**, **MySQL** -**Knowledge**: :star: :star: :star: -**Complexity**: :star: :star: :star: +**Knowledge**: :star: :star: :star: +**Complexity**: :star: :star: :star: **Reading time**: 30 minutes @@ -261,7 +261,7 @@ MariaDB [(none)]> show databases; #### The `mariadb-admin` command The `mariadb-admin` command is a client for administering a MariaDB server. - + ```bash mariadb-admin -u user -p command ``` @@ -881,549 +881,3 @@ In this chapter, you have installed and secured a MariaDB database server, creat These skills are a prerequisite for the administration of your databases. In the next section, you will see how to install the MySQL database instead of the MariaDB fork. - -## Mysql - -In this chapter, you will learn how to install MySQL server. - -Only notable differences between the MariaDB and MySQL versions are included. - -**** - -**Objectives**: In this chapter, you will learn how to: - -:heavy_check_mark: install, configure and secure MariaDB server and MySQL server; - -:checkered_flag: **RDBMS**, **database**, **MariaDB**, **MySQL** - -**Knowledge**: :star: :star: :star: -**Complexity**: :star: :star: :star: - -**Reading time**: 10 minutes - -**** - -### Installation of MySQL - -By default, the installed version of MySQL is version 8.0. - -This time, you have to install the `mysql-server` package: - -```bash -sudo dnf install mysql-server -``` - -and start the `mysqld` service: - -```bash -sudo systemctl enable mysqld.service --now -``` - -You can now follow the previous chapter replacing the following commands: - -* `mariadb` => `mysql` -* `mariadb-admin` => `mysql_admin` -* `mariadb-dump` => `mysql_dump` -* `mariadb-secure-installation` => `mysql_secure_installation` - -To install the latest version of mysql-server, you will have to install a different repository. - -Visit this page: https://dev.mysql.com/downloads/repo/yum/ and copy the repository URL. - -For example: - -```bash -sudo dnf install -y https://dev.mysql.com/get/mysql84-community-release-el9-1.noarch.rpm -``` - -When completed, you can perform the `dnf update`: - -```bash -$ dnf update -Error: This command has to be run with superuser privileges (under the root user on most systems). -[antoine@localhost ~]$ sudo dnf update -MySQL 8.4 LTS Community Server 377 kB/s | 226 kB 00:00 -MySQL Connectors Community 110 kB/s | 53 kB 00:00 -MySQL Tools 8.4 LTS Community 170 kB/s | 97 kB 00:00 -Dependencies resolved. -============================================================================================================================================= Package Architecture Version Repository Size -=============================================================================================================================================Installing: - mysql-community-client x86_64 8.4.0-1.el9 mysql-8.4-lts-community 3.1 M - replacing mysql.x86_64 8.0.36-1.el9_3 - mysql-community-server x86_64 8.4.0-1.el9 mysql-8.4-lts-community 50 M - replacing mariadb-connector-c-config.noarch 3.2.6-1.el9_0 - replacing mysql-server.x86_64 8.0.36-1.el9_3 -Installing dependencies: - ... - -Transaction Summary -=============================================================================================================================================Install 7 Packages - -Total download size: 59 M -Is this ok [y/N]: y -Downloading Packages: -(1/7): mysql-community-client-plugins-8.4.0-1.el9.x86_64.rpm 3.4 MB/s | 1.4 MB 00:00 -(2/7): mysql-community-common-8.4.0-1.el9.x86_64.rpm 1.3 MB/s | 576 kB 00:00 -(3/7): mysql-community-icu-data-files-8.4.0-1.el9.x86_64.rpm 30 MB/s | 2.3 MB 00:00 -(4/7): mysql-community-client-8.4.0-1.el9.x86_64.rpm 5.8 MB/s | 3.1 MB 00:00 -(5/7): mysql-community-libs-8.4.0-1.el9.x86_64.rpm 6.8 MB/s | 1.5 MB 00:00 -(6/7): net-tools-2.0-0.62.20160912git.el9.x86_64.rpm 1.1 MB/s | 292 kB 00:00 -(7/7): mysql-community-server-8.4.0-1.el9.x86_64.rpm 48 MB/s | 50 MB 00:01 ----------------------------------------------------------------------------------------------------------------------------------------------Total 30 MB/s | 59 MB 00:01 -MySQL 8.4 LTS Community Server 3.0 MB/s | 3.1 kB 00:00 -Importing GPG key 0xA8D3785C: - Userid : "MySQL Release Engineering " - Fingerprint: BCA4 3417 C3B4 85DD 128E C6D4 B7B3 B788 A8D3 785C - From : /etc/pki/rpm-gpg/RPM-GPG-KEY-mysql-2023 -Is this ok [y/N]: y -Key imported successfully -Running transaction check -Transaction check succeeded. -Running transaction test -Transaction test succeeded. -Running transaction - Preparing : - ... -Installed: - mysql-community-server-8.4.0-1.el9.x86_64 - ... - -Complete! -``` - -Do not forget to re-enable and restart your server: - -```bash -sudo systemctl enable mysqld.service --now -``` - -### Check your Knowledge MySQL - -:heavy_check_mark: Which MySQL database version is installed by default? - -* [ ] MySQL 5.5 -* [ ] MariaDB 10.5 -* [ ] MariaDB 11.11 -* [ ] Mysql 8 - -## Secondary server with MariaDB - -In this chapter, you will learn how to configure a Primary/Secondary system servers with MariaDB. - -**** - -**Objectives**: In this chapter, you will learn how to: - -:heavy_check_mark: activate the binlogs in your servers; -:heavy_check_mark: setup a secondary server to replicate data from primary server. - -:checkered_flag: **MariaDB**, **Replication**, **Primary**, **Secondary** - -**Knowledge**: :star: :star: -**Complexity**: :star: :star: :star: - -**Reading time**: 10 minutes - -**** - -### Generalities secondary server with MariaDB - -As soon as you start using your database more intensively, you will need to replicate your data on several servers. - -This can be done in several ways: - -* Distribute write requests to the primary server and read requests to the secondary server. -* Perform database backups on the secondary server, which avoids blocking writes to the primary server for the duration of the backups. - -If your usage becomes even more demanding, you may consider switching to a primary/primary system: replications are then made crosswise, but beware of the risk of blocking the uniqueness of primary keys. Otherwise, you will need to switch to a more advanced clustering system. - -### Configuration secondary server with MariaDB - -#### How to activate the binlogs - -Perform this action on the primary and secondary servers: - -Add the following options to your `/etc/my.cnf.d/mariadb-server.cnf` file, under the `[mariadb]` key: - -```file -[mariadb] -log-bin -server_id=1 -log-basename=server1 -binlog-format=mixed -``` - -for the primary server, and for the secondary server: - -```file -[mariadb] -log-bin -server_id=2 -log-basename=server2 -binlog-format=mixed -``` - -The `server_id` option must be unique on each server in the cluster, while the `log-basename` option allows you to specify a prefix to the binlog files. If you do not do this, you will not be able to rename your server in the future. - -You can now restart the mariadb service on both servers: - -```bash -sudo systemctl restart mariadb -``` - -You can check that binlogs files are well created: - -```bash -$ ll /var/lib/mysql/ -total 123332 -... --rw-rw----. 1 mysql mysql 0 Jun 21 11:07 multi-master.info -drwx------. 2 mysql mysql 4096 Jun 21 11:07 mysql -srwxrwxrwx. 1 mysql mysql 0 Jun 21 11:16 mysql.sock --rw-rw----. 1 mysql mysql 330 Jun 21 11:16 server1-bin.000001 --rw-rw----. 1 mysql mysql 21 Jun 21 11:16 server1-bin.index -... -``` - -#### How to configure the replication - -First of all, on the primary, you will need to create users authorized to replicate data (be careful to restrict the IPs authorized): - -```bash -$ sudo mariadb - -MariaDB [(none)]> CREATE USER 'replication'@'%' IDENTIFIED BY 'PASSWORD'; -Query OK, 0 rows affected (0.002 sec) - -MariaDB [(none)]> GRANT REPLICATION SLAVE ON *.* TO 'replication'@'%'; -Query OK, 0 rows affected (0.002 sec) -``` - -or better for security (change '192.168.1.101' with your own secondary IP): - -```bash -$ sudo mariadb - -MariaDB [(none)]> CREATE USER 'replication'@'192.168.1.101' IDENTIFIED BY 'PASSWORD'; -Query OK, 0 rows affected (0.002 sec) - -MariaDB [(none)]> GRANT REPLICATION SLAVE ON *.* TO 'replication'@'192.168.1.101'; -Query OK, 0 rows affected (0.002 sec) -``` - -If your primary server already contains data, you will need to lock new transactions while the exporting or importing of data occurs to the secondary server(s), and tell the secondary servers when to start replication. If your server does not yet contain any data, the procedure is greatly simplified. - -Prevent any changes to the data while you view the binary log position: - -```bash -$ sudo mariadb - -MariaDB [(none)]> FLUSH TABLES WITH READ LOCK; -Query OK, 0 rows affected (0.021 sec) - -MariaDB [(none)]> SHOW MASTER STATUS; -+--------------------+----------+--------------+------------------+ -| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | -+--------------------+----------+--------------+------------------+ -| server1-bin.000001 | 1009 | | | -+--------------------+----------+--------------+------------------+ -1 row in set (0.000 sec) - -``` - -Do not quit your session to keep the lock. - -Record the File and Position details. - -If your server contains data, it is time to create a backup and import it onto your secondary server(s). Keep the lock for the duration of the backup, and release it as soon as the backup is complete. This reduces downtime (the time it takes to copy and import the data on the secondary servers). - -You can remove the lock now: - -```bash -$ sudo mariadb - -MariaDB [(none)]> UNLOCK TABLES; -Query OK, 0 rows affected (0.000 sec) -``` - -On the secondary server, you can now ready to setup the primary server to replicate with: - -```bash -MariaDB [(none)]> CHANGE MASTER TO - MASTER_HOST='192.168.1.100', - MASTER_USER='replication', - MASTER_PASSWORD='PASSWORD', - MASTER_PORT=3306, - MASTER_LOG_FILE='server1-bin.000001', - MASTER_LOG_POS=1009, - MASTER_CONNECT_RETRY=10; -Query OK, 0 rows affected, 1 warning (0.021 sec) - -MariaDB [(none)]> START SLAVE; -Query OK, 0 rows affected (0.001 sec) -``` - -Replace the primary server IP with yours and the `MASTER_LOG_FILE` and `MASTER_LOG_POS` values with those you previously registered. - -Check if the replication is ok: - -```bash -MariaDB [(none)]> SHOW SLAVE STATUS \G -*************************** 1. row *************************** - Slave_IO_State: Waiting for master to send event - Master_Host: 192.168.1.100 - Master_User: replication - Master_Log_File: server1-bin.000001 - Read_Master_Log_Pos: 1009 -... - Seconds_Behind_Master: 0 - Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates -... -1 row in set (0.001 sec) -``` - -The `Seconds_Behind_Master` is an interesting value to monitor as it can help you see if there is a replication issue. - -### Workshop secondary server using MariaDB - -For this workshop, you will need two servers with MariaDB services installed, configured and secured as described in the previous chapters. - -You will configure replication on the secondary server, then create a new database, insert data into it and check that the data is accessible on the secondary server. - -Our two servers have the following IP addresses: - -* server1: 192.168.1.100 -* server2: 192.168.1.101 - -Remember to replace these values with your own. - -#### Task 1: Create a dedicated replication user - -On the primary server: - -```bash -$ sudo mariadb - -MariaDB [(none)]> CREATE USER 'replication'@'192.168.1.101' IDENTIFIED BY 'PASSWORD'; -Query OK, 0 rows affected (0.002 sec) - -MariaDB [(none)]> GRANT REPLICATION SLAVE ON *.* TO 'replication'@'192.168.1.101'; -Query OK, 0 rows affected (0.002 sec) -``` - -#### Task 2: Record the primary server values - -```bash -$ sudo mariadb - -MariaDB [(none)]> FLUSH TABLES WITH READ LOCK; -Query OK, 0 rows affected (0.021 sec) - -MariaDB [(none)]> SHOW MASTER STATUS; -+--------------------+----------+--------------+------------------+ -| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | -+--------------------+----------+--------------+------------------+ -| server1-bin.000001 | 1009 | | | -+--------------------+----------+--------------+------------------+ -1 row in set (0.000 sec) - -MariaDB [(none)]> UNLOCK TABLES; -Query OK, 0 rows affected (0.000 sec) -``` - -#### Task 3: Activate the replication - -On the secondary server: - -```bash -MariaDB [(none)]> CHANGE MASTER TO - MASTER_HOST='192.168.1.100', - MASTER_USER='replication', - MASTER_PASSWORD='PASSWORD', - MASTER_PORT=3306, - MASTER_LOG_FILE='server1-bin.000001', - MASTER_LOG_POS=1009, - MASTER_CONNECT_RETRY=10; -Query OK, 0 rows affected, 1 warning (0.021 sec) - -MariaDB [(none)]> START SLAVE; -Query OK, 0 rows affected (0.001 sec) -``` - -Check if the replication is ok: - -```bash -MariaDB [(none)]> SHOW SLAVE STATUS \G -*************************** 1. row *************************** - Slave_IO_State: Waiting for master to send event - Master_Host: 192.168.1.100 - Master_User: replication - Master_Log_File: server1-bin.000001 - Read_Master_Log_Pos: 1009 -... - Seconds_Behind_Master: 0 - Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates -... -1 row in set (0.001 sec) -``` - -#### Task 4: Create a new database and a user - -On the primary: - -```bash -MariaDB [(none)]> create database NEW_DATABASE_NAME; -Query OK, 1 row affected (0.002 sec) - -MariaDB [(none)]> grant all privileges on NEW_DATABASE_NAME.* TO 'NEW_USER_NAME'@'localhost' identified by 'PASSWORD'; -Query OK, 0 rows affected (0.004 sec) -``` - -On the secondary, check for creation of the database: - -```bash -MariaDB [(none)]> show databases; -+--------------------+ -| Database | -+--------------------+ -| NEW_DATABASE_NAME | -| information_schema | -| mysql | -| performance_schema | -| sys | -+--------------------+ -``` - -Magic ! - -On the secondary, try connecting the new user created on the primary: - -```bash -$ mariadb -u NEW_USER_NAME -p -Enter password: -Welcome to the MariaDB monitor. Commands end with ; or \g. - -MariaDB [(none)]> show databases; -+--------------------+ -| Database | -+--------------------+ -| NEW_DATABASE_NAME | -| information_schema | -+--------------------+ -2 rows in set (0.000 sec) -``` - -#### Task 5: Insert new data - -Insert new data on the primary server: - -```bash -MariaDB [(none)]> use NEW_DATABASE_NAME -Database changed - -MariaDB [(none)]> CREATE TABLE users( - -> id INT NOT NULL AUTO_INCREMENT, - -> first_name VARCHAR(30) NOT NULL, - -> last_name VARCHAR(30) NOT NULL, - -> age INT DEFAULT NULL, - -> PRIMARY KEY (id)); - -MariaDB [NEW_DATABASE_NAME]> INSERT INTO users (first_name, last_name, age) VALUES ("Antoine", "Le Morvan", 44); -Query OK, 1 row affected (0.004 sec) - -``` - -On the secondary, check that data are replicated: - -```bash -MariaDB [(none)]> use NEW_DATABASE_NAME -Database changed - -MariaDB [NEW_DATABASE_NAME]> show tables; -+-----------------------------+ -| Tables_in_NEW_DATABASE_NAME | -+-----------------------------+ -| users | -+-----------------------------+ -1 row in set (0.000 sec) - -MariaDB [NEW_DATABASE_NAME]> SELECT * FROM users; -+----+------------+-----------+------+ -| id | first_name | last_name | age | -+----+------------+-----------+------+ -| 1 | Antoine | Le Morvan | 44 | -+----+------------+-----------+------+ -1 row in set (0.000 sec) -``` - -### Check your Knowledge secondary server with MariaDB - -:heavy_check_mark: Each server must have the same id within a cluster? - -* [ ] True -* [ ] False - -:heavy_check_mark: Binary logs must be enabled before replication is activated.? - -* [ ] True -* [ ] False -* [ ] It depends - -### Conclusion secondary server with MariaDB - -As you can see, creating one or more secondary servers is a relatively easy action, but it does require service interruption on the main server. - -It does, however, offer many advantages: high data availability, load balancing, and simplified backup. - -It goes without saying that, in the event of a main server crash, promotion of one of the secondary servers to main server can occur. - - diff --git a/docs/books/web_services/042-database-servers-mysql.md b/docs/books/web_services/042-database-servers-mysql.md new file mode 100644 index 0000000000..e07f821ca7 --- /dev/null +++ b/docs/books/web_services/042-database-servers-mysql.md @@ -0,0 +1,133 @@ +--- +author: Antoine Le Morvan +contributors: Steven Spencer +title: Part 4.2 Database Servers MySQL +--- + +## MySQL + +In this chapter, you will learn how to install MySQL server. + +!!! NOTE + + Only notable differences between the MariaDB and MySQL versions are included. + +--- + +**Objectives**: In this chapter, you will learn how to: + +:heavy_check_mark: install, configure and secure MariaDB server and MySQL server; + +:checkered_flag: **RDBMS**, **database**, **MariaDB**, **MySQL** + +**Knowledge**: :star: :star: :star: +**Complexity**: :star: :star: :star: + +**Reading time**: 10 minutes + +--- + +### Installation of MySQL + +By default, the installed version of MySQL is version 8.0. + +This time, you have to install the `mysql-server` package: + +```bash +sudo dnf install mysql-server +``` + +and start the `mysqld` service: + +```bash +sudo systemctl enable mysqld.service --now +``` + +You can now follow the previous chapter replacing the following commands: + +* `mariadb` => `mysql` +* `mariadb-admin` => `mysql_admin` +* `mariadb-dump` => `mysql_dump` +* `mariadb-secure-installation` => `mysql_secure_installation` + +To install the latest version of mysql-server, you will have to install a different repository. + +Visit this page: https://dev.mysql.com/downloads/repo/yum/ and copy the repository URL. + +For example: + +```bash +sudo dnf install -y https://dev.mysql.com/get/mysql84-community-release-el9-1.noarch.rpm +``` + +When completed, you can perform the `dnf update`: + +```bash +$ dnf update +Error: This command has to be run with superuser privileges (under the root user on most systems). +[antoine@localhost ~]$ sudo dnf update +MySQL 8.4 LTS Community Server 377 kB/s | 226 kB 00:00 +MySQL Connectors Community 110 kB/s | 53 kB 00:00 +MySQL Tools 8.4 LTS Community 170 kB/s | 97 kB 00:00 +Dependencies resolved. +============================================================================================================================================= Package Architecture Version Repository + Size +=============================================================================================================================================Installing: + mysql-community-client x86_64 8.4.0-1.el9 mysql-8.4-lts-community 3.1 M + replacing mysql.x86_64 8.0.36-1.el9_3 + mysql-community-server x86_64 8.4.0-1.el9 mysql-8.4-lts-community 50 M + replacing mariadb-connector-c-config.noarch 3.2.6-1.el9_0 + replacing mysql-server.x86_64 8.0.36-1.el9_3 +Installing dependencies: + ... + +Transaction Summary +=============================================================================================================================================Install 7 Packages + +Total download size: 59 M +Is this ok [y/N]: y +Downloading Packages: +(1/7): mysql-community-client-plugins-8.4.0-1.el9.x86_64.rpm 3.4 MB/s | 1.4 MB 00:00 +(2/7): mysql-community-common-8.4.0-1.el9.x86_64.rpm 1.3 MB/s | 576 kB 00:00 +(3/7): mysql-community-icu-data-files-8.4.0-1.el9.x86_64.rpm 30 MB/s | 2.3 MB 00:00 +(4/7): mysql-community-client-8.4.0-1.el9.x86_64.rpm 5.8 MB/s | 3.1 MB 00:00 +(5/7): mysql-community-libs-8.4.0-1.el9.x86_64.rpm 6.8 MB/s | 1.5 MB 00:00 +(6/7): net-tools-2.0-0.62.20160912git.el9.x86_64.rpm 1.1 MB/s | 292 kB 00:00 +(7/7): mysql-community-server-8.4.0-1.el9.x86_64.rpm 48 MB/s | 50 MB 00:01 +---------------------------------------------------------------------------------------------------------------------------------------------Total 30 +MB/s | 59 MB 00:01 +MySQL 8.4 LTS Community Server 3.0 MB/s | 3.1 kB 00:00 +Importing GPG key 0xA8D3785C: + Userid : "MySQL Release Engineering " + Fingerprint: BCA4 3417 C3B4 85DD 128E C6D4 B7B3 B788 A8D3 785C + From : /etc/pki/rpm-gpg/RPM-GPG-KEY-mysql-2023 +Is this ok [y/N]: y +Key imported successfully +Running transaction check +Transaction check succeeded. +Running transaction test +Transaction test succeeded. +Running transaction + Preparing : + ... +Installed: + mysql-community-server-8.4.0-1.el9.x86_64 + ... + +Complete! +``` + +Do not forget to re-enable and restart your server: + +```bash +sudo systemctl enable mysqld.service --now +``` + +### Check your Knowledge MySQL + +:heavy_check_mark: Which MySQL database version is installed by default? + +* [ ] MySQL 5.5 +* [ ] MariaDB 10.5 +* [ ] MariaDB 11.11 +* [ ] Mysql 8 diff --git a/docs/books/web_services/043-database-servers-replication.md b/docs/books/web_services/043-database-servers-replication.md new file mode 100644 index 0000000000..982fd42b9b --- /dev/null +++ b/docs/books/web_services/043-database-servers-replication.md @@ -0,0 +1,427 @@ +--- +author: Antoine Le Morvan +contributors: Steven Spencer +title: Part 4.3 MariaDB database replication +--- + +## Secondary server with MariaDB + +In this chapter, you will learn how to configure a Primary/Secondary system servers with MariaDB. + +**** + +**Objectives**: In this chapter, you will learn how to: + +:heavy_check_mark: activate the binlogs in your servers; +:heavy_check_mark: setup a secondary server to replicate data from primary server. + +:checkered_flag: **MariaDB**, **Replication**, **Primary**, **Secondary** + +**Knowledge**: :star: :star: +**Complexity**: :star: :star: :star: + +**Reading time**: 10 minutes + +**** + +### Generalities secondary server with MariaDB + +As soon as you start using your database more intensively, you will need to replicate your data on several servers. + +This can be done in several ways: + +* Distribute write requests to the primary server and read requests to the secondary server. +* Perform database backups on the secondary server, which avoids blocking writes to the primary server for the duration of the backups. + +If your usage becomes even more demanding, you may consider switching to a primary/primary system: replications are then made crosswise, but beware of the risk of blocking the uniqueness of primary keys. Otherwise, you will need to switch to a more advanced clustering system. + +### Configuration secondary server with MariaDB + +#### How to activate the binlogs + +Perform this action on the primary and secondary servers: + +Add the following options to your `/etc/my.cnf.d/mariadb-server.cnf` file, under the `[mariadb]` key: + +```file +[mariadb] +log-bin +server_id=1 +log-basename=server1 +binlog-format=mixed +``` + +for the primary server, and for the secondary server: + +```file +[mariadb] +log-bin +server_id=2 +log-basename=server2 +binlog-format=mixed +``` + +The `server_id` option must be unique on each server in the cluster, while the `log-basename` option allows you to specify a prefix to the binlog files. If you do not do this, you will not be able to rename your server in the future. + +You can now restart the mariadb service on both servers: + +```bash +sudo systemctl restart mariadb +``` + +You can check that binlogs files are well created: + +```bash +$ ll /var/lib/mysql/ +total 123332 +... +-rw-rw----. 1 mysql mysql 0 Jun 21 11:07 multi-master.info +drwx------. 2 mysql mysql 4096 Jun 21 11:07 mysql +srwxrwxrwx. 1 mysql mysql 0 Jun 21 11:16 mysql.sock +-rw-rw----. 1 mysql mysql 330 Jun 21 11:16 server1-bin.000001 +-rw-rw----. 1 mysql mysql 21 Jun 21 11:16 server1-bin.index +... +``` + +#### How to configure the replication + +First of all, on the primary, you will need to create users authorized to replicate data (be careful to restrict the IPs authorized): + +```bash +$ sudo mariadb + +MariaDB [(none)]> CREATE USER 'replication'@'%' IDENTIFIED BY 'PASSWORD'; +Query OK, 0 rows affected (0.002 sec) + +MariaDB [(none)]> GRANT REPLICATION SLAVE ON *.* TO 'replication'@'%'; +Query OK, 0 rows affected (0.002 sec) +``` + +or better for security (change '192.168.1.101' with your own secondary IP): + +```bash +$ sudo mariadb + +MariaDB [(none)]> CREATE USER 'replication'@'192.168.1.101' IDENTIFIED BY 'PASSWORD'; +Query OK, 0 rows affected (0.002 sec) + +MariaDB [(none)]> GRANT REPLICATION SLAVE ON *.* TO 'replication'@'192.168.1.101'; +Query OK, 0 rows affected (0.002 sec) +``` + +If your primary server already contains data, you will need to lock new transactions while the exporting or importing of data occurs to the secondary server(s), and tell the secondary servers when to start replication. If your server does not yet contain any data, the procedure is greatly simplified. + +Prevent any changes to the data while you view the binary log position: + +```bash +$ sudo mariadb + +MariaDB [(none)]> FLUSH TABLES WITH READ LOCK; +Query OK, 0 rows affected (0.021 sec) + +MariaDB [(none)]> SHOW MASTER STATUS; ++--------------------+----------+--------------+------------------+ +| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | ++--------------------+----------+--------------+------------------+ +| server1-bin.000001 | 1009 | | | ++--------------------+----------+--------------+------------------+ +1 row in set (0.000 sec) + +``` + +Do not quit your session to keep the lock. + +Record the File and Position details. + +If your server contains data, it is time to create a backup and import it onto your secondary server(s). Keep the lock for the duration of the backup, and release it as soon as the backup is complete. This reduces downtime (the time it takes to copy and import the data on the secondary servers). + +You can remove the lock now: + +```bash +$ sudo mariadb + +MariaDB [(none)]> UNLOCK TABLES; +Query OK, 0 rows affected (0.000 sec) +``` + +On the secondary server, you can now ready to setup the primary server to replicate with: + +```bash +MariaDB [(none)]> CHANGE MASTER TO + MASTER_HOST='192.168.1.100', + MASTER_USER='replication', + MASTER_PASSWORD='PASSWORD', + MASTER_PORT=3306, + MASTER_LOG_FILE='server1-bin.000001', + MASTER_LOG_POS=1009, + MASTER_CONNECT_RETRY=10; +Query OK, 0 rows affected, 1 warning (0.021 sec) + +MariaDB [(none)]> START SLAVE; +Query OK, 0 rows affected (0.001 sec) +``` + +Replace the primary server IP with yours and the `MASTER_LOG_FILE` and `MASTER_LOG_POS` values with those you previously registered. + +Check if the replication is ok: + +```bash +MariaDB [(none)]> SHOW SLAVE STATUS \G +*************************** 1. row *************************** + Slave_IO_State: Waiting for master to send event + Master_Host: 192.168.1.100 + Master_User: replication + Master_Log_File: server1-bin.000001 + Read_Master_Log_Pos: 1009 +... + Seconds_Behind_Master: 0 + Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates +... +1 row in set (0.001 sec) +``` + +The `Seconds_Behind_Master` is an interesting value to monitor as it can help you see if there is a replication issue. + +### Workshop secondary server using MariaDB + +For this workshop, you will need two servers with MariaDB services installed, configured and secured as described in the previous chapters. + +You will configure replication on the secondary server, then create a new database, insert data into it and check that the data is accessible on the secondary server. + +Our two servers have the following IP addresses: + +* server1: 192.168.1.100 +* server2: 192.168.1.101 + +Remember to replace these values with your own. + +#### Task 1: Create a dedicated replication user + +On the primary server: + +```bash +$ sudo mariadb + +MariaDB [(none)]> CREATE USER 'replication'@'192.168.1.101' IDENTIFIED BY 'PASSWORD'; +Query OK, 0 rows affected (0.002 sec) + +MariaDB [(none)]> GRANT REPLICATION SLAVE ON *.* TO 'replication'@'192.168.1.101'; +Query OK, 0 rows affected (0.002 sec) +``` + +#### Task 2: Record the primary server values + +```bash +$ sudo mariadb + +MariaDB [(none)]> FLUSH TABLES WITH READ LOCK; +Query OK, 0 rows affected (0.021 sec) + +MariaDB [(none)]> SHOW MASTER STATUS; ++--------------------+----------+--------------+------------------+ +| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | ++--------------------+----------+--------------+------------------+ +| server1-bin.000001 | 1009 | | | ++--------------------+----------+--------------+------------------+ +1 row in set (0.000 sec) + +MariaDB [(none)]> UNLOCK TABLES; +Query OK, 0 rows affected (0.000 sec) +``` + +#### Task 3: Activate the replication + +On the secondary server: + +```bash +MariaDB [(none)]> CHANGE MASTER TO + MASTER_HOST='192.168.1.100', + MASTER_USER='replication', + MASTER_PASSWORD='PASSWORD', + MASTER_PORT=3306, + MASTER_LOG_FILE='server1-bin.000001', + MASTER_LOG_POS=1009, + MASTER_CONNECT_RETRY=10; +Query OK, 0 rows affected, 1 warning (0.021 sec) + +MariaDB [(none)]> START SLAVE; +Query OK, 0 rows affected (0.001 sec) +``` + +Check if the replication is ok: + +```bash +MariaDB [(none)]> SHOW SLAVE STATUS \G +*************************** 1. row *************************** + Slave_IO_State: Waiting for master to send event + Master_Host: 192.168.1.100 + Master_User: replication + Master_Log_File: server1-bin.000001 + Read_Master_Log_Pos: 1009 +... + Seconds_Behind_Master: 0 + Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates +... +1 row in set (0.001 sec) +``` + +#### Task 4: Create a new database and a user + +On the primary: + +```bash +MariaDB [(none)]> create database NEW_DATABASE_NAME; +Query OK, 1 row affected (0.002 sec) + +MariaDB [(none)]> grant all privileges on NEW_DATABASE_NAME.* TO 'NEW_USER_NAME'@'localhost' identified by 'PASSWORD'; +Query OK, 0 rows affected (0.004 sec) +``` + +On the secondary, check for creation of the database: + +```bash +MariaDB [(none)]> show databases; ++--------------------+ +| Database | ++--------------------+ +| NEW_DATABASE_NAME | +| information_schema | +| mysql | +| performance_schema | +| sys | ++--------------------+ +``` + +Magic ! + +On the secondary, try connecting the new user created on the primary: + +```bash +$ mariadb -u NEW_USER_NAME -p +Enter password: +Welcome to the MariaDB monitor. Commands end with ; or \g. + +MariaDB [(none)]> show databases; ++--------------------+ +| Database | ++--------------------+ +| NEW_DATABASE_NAME | +| information_schema | ++--------------------+ +2 rows in set (0.000 sec) +``` + +#### Task 5: Insert new data + +Insert new data on the primary server: + +```bash +MariaDB [(none)]> use NEW_DATABASE_NAME +Database changed + +MariaDB [(none)]> CREATE TABLE users( + -> id INT NOT NULL AUTO_INCREMENT, + -> first_name VARCHAR(30) NOT NULL, + -> last_name VARCHAR(30) NOT NULL, + -> age INT DEFAULT NULL, + -> PRIMARY KEY (id)); + +MariaDB [NEW_DATABASE_NAME]> INSERT INTO users (first_name, last_name, age) VALUES ("Antoine", "Le Morvan", 44); +Query OK, 1 row affected (0.004 sec) + +``` + +On the secondary, check that data are replicated: + +```bash +MariaDB [(none)]> use NEW_DATABASE_NAME +Database changed + +MariaDB [NEW_DATABASE_NAME]> show tables; ++-----------------------------+ +| Tables_in_NEW_DATABASE_NAME | ++-----------------------------+ +| users | ++-----------------------------+ +1 row in set (0.000 sec) + +MariaDB [NEW_DATABASE_NAME]> SELECT * FROM users; ++----+------------+-----------+------+ +| id | first_name | last_name | age | ++----+------------+-----------+------+ +| 1 | Antoine | Le Morvan | 44 | ++----+------------+-----------+------+ +1 row in set (0.000 sec) +``` + +### Check your Knowledge secondary server with MariaDB + +:heavy_check_mark: Each server must have the same id within a cluster? + +* [ ] True +* [ ] False + +:heavy_check_mark: Binary logs must be enabled before replication is activated.? + +* [ ] True +* [ ] False +* [ ] It depends + +### Conclusion secondary server with MariaDB + +As you can see, creating one or more secondary servers is a relatively easy action, but it does require service interruption on the main server. + +It does, however, offer many advantages: high data availability, load balancing, and simplified backup. + +It goes without saying that, in the event of a main server crash, promotion of one of the secondary servers to main server can occur. + +