MySQL 5.7 and administration

MySQL 5.7 improved as compare to previous releases in terms of transnational capabilities, performance with high load, high Availability, Security and it’s defaults.

Check my blog post : MySQL 5.7 features

This blog post will describe End to End implementation of MySQL on Linux distributions Which will cover MySQL Installation, configuration and administration in production environment with proper configuration. So you can start using your application by implementing following setup and in future you can change it if requires.

Database Installation:

There are number of ways to install MySQL,

  • Source code
  • Binaries : Compressed tar file binary distribution
  • Packages :  RPM-based Linux distributions
  • MySQL Installer MSI and ZIP Archive
  • Yum repository

MySQL installation using packages is one the easy way to install MySQL and you don’t have to worry about installation configuration part.Another option is installing MySQL using compressed tar file, with this installation method user has to perform installation and most of the configuration part.

On Windows : MySQL Installer MSI will take care of everything including installation of supporting monitoring tool and utilities, MySQL configuration settings and user management

Let’s Install MySQL RPM packages.For standard installation, we will install mysql-community-servermysql-community-clientmysql-community-libsmysql-community-common, and mysql-community-libs-compat packages.

Always use new version of MySQL GA release for new installations.

MySQL Installation steps:

1. Download RPM package from for your OS architecture : http://dev.mysql.com/downloads/mysql

2. Run following command to install MySQL.

sudo yum install mysql-community-{server,client,common,libs}-*

The installation also creates a user named mysql and a group named mysql on the system.

3. Default configuration file location /etc/my.cnf file.

4. Standard directory structure for MySQL:

A standard installation of MySQL using the RPM packages result in files and resources created under the system directories, shown in the following table.

mysql-rpm

MySQL basedir will have default distributed dir structure, except datadir. It is recommended to keep datadir in a different location for numerous reasons.For, this you just need to update ‘datadir’  variable value with new datadir location

5. MySQL configuration file: /etc/my.cnf

By, default my.cnf will get created by MySQL rpm installation with default configuration in it. We need to add few more configuration variables in order to make MySQL DB server ready for production use. Following are the standard configuration settings for a production database. These variable values may vary as per the application scope and data workload.

MySQL configuration file sections – MySQL configuration file have many sessions, such as [mysqld], [mysql], [client], [mysqld_safe] , [mysqldump] and so on.

  • Add/update following variables to appropriate configuration section of my.cnf.
[mysqld]

#GENERAL
user = mysql
port = 3306
server_id=152
skip-name-resolve
default_storage_engine = InnoDB
socket = /var/lib/mysql/mysql.sock
pid_file = /var/run/mysql/mysqld.pid

# DATA STORAGE
datadir =/var/lib/mysql/data

#INNODB
innodb_file_per_table=1
innodb_buffer_pool_size = 4000M (60-70 % of RAM memory)
innodb_data_file_path= ibdata1:1G;ibdata2:500M:autoextend
innodb-log-files-in-group=3
innodb_flush_method = O_DIRECT

#Logging
log_error = /var/log/mysql/mysqld.log
master_info_repository = TABLE
relay_log_info_repository = TABLE
log-bin=mysql-bin152
relay-log=relay-bin152
relay_log_recovery=on
log-slave-updates= 1
expire_logs_days = 7
gtid-mode=on
enforce-gtid-consistency=1
binlog_format=row

[mysql]
socket = /var/lib/mysql/mysql.sock

[client]
socket = /var/lib/mysql/mysql.sock
port = 3306

6. Start MySQL service

sudo service mysqld start

7. At the initial start-up of the server, the following happens, when MySQL data directory is empty:

  • The server is initialized.
  • An SSL certificate and key files are generated in the data directory.
  • Thevalidate_password plugin is installed and enabled.
  • A superuser account’root’@’localhost’ is created. A password for the superuser is set and stored in the error log file.
  • Look for root password in error log file.
sudo grep 'temporary password' /var/log/mysqld.log

8. After login first time into MySQL , we can not proceed further without resetting root password.

shell> mysql –uroot -p
mysql> ALTER USER 'root'@'localhost' IDENTIFIED BY 'r00t$PWD';

For any startup issue  check  /var/log/mysqld.log 

MySQL Variables and Configuration:

MySQL Variables
User mysql service user
Server-id Value : 1 default

Any number in DB group

Port Value: 3306 default
Skip-name-resolve Do not resolve host names when checking client connections. Use only IP addresses.
bind_address MySQL bind_address for network interfaces.

IPv4 : 0.0.0.0

IPv4 and IPv6 : *

Socket Unix socket file for listening local connections
Pid-file The path name of the process ID file.
default_storage_engine Default storage engine for MySQL

Value: Innodb

Datadir Main directory where database,system tablespace and log files will get store.
innodb_file_per_table Seperate tablespace for each table.Good for performance and reclaiming free space.

Value : on

Innodb_buffer_pool_size Value should be 60-70 percent of RAM memory of server
innodb_log_file_size Redo and undo logs ,useful for innodb recovery.

Value should be greater if you are using BLOB datatype in your database.

Value: innodb_log_file_size=150M

innodb_log_files_in_group Number for innodb_log_file

Value : 3

innodb_data_file_path= System tablespace configuration

Value:ibdata1:1G;ibdata2:1G:autoextend (vary)

innodb_flush_method Method used to flush data to the InnoDB data files and log files.

value : O_DIRECT

innodb_tmpdir tmp directory for ONLINE ALTER operations.
log_error mysql server log
log-bin Binary log file name

Value : mysql-bin152

binlog_format binary log formate for data

Value:row

master_info_repository

relay_log_info_repository

crash-safe replication settings, storing log info in table instead of file.

Value:  TABLE

relay-log relay log name

relay-log=relay-bin152

relay_log_recovery relay_log_recovery= on
log-slave-updates log-slave-updates=1
expire_logs_days Auto delete binary logs after mentioned days

expire_logs_days= 60

gtid-mode Enable GTID for transactions

Value : on

enforce-gtid-consistency Value : on

MySQL User Management

MySQL Database Users

CREATE USER :

CREATE USER [IF NOT EXISTS] 'local_user1'@'localhost' IDENTIFIED BY 'usr#PWD';

(Remote connection restricted for this user)

If you specify only the username part of the account name, a host name part of ‘%’ is used.

CREATE USER [IF NOT EXISTS] 'remote_user1'@'%' IDENTIFIED BY 'usr#PWD';

(Remote connection enabled for this user)

-User details getting stored under mysql.user table.

RENAME USER:

RENAME USER 'abc'@'localhost' TO 'xyz'@'%';

User password management:

  1. Change/Update user password. [IF EXISTS] -Optional
ALETR USER [IF EXISTS] 'remote_user1'@'%' IDENTIFIED BY 'usr#PWD';
  1. Password expire user account
ALETR USER[IF EXISTS] 'remote_user1'@'%' PASSWORD EXPIRE;
  1. Locked User account
ALTER USER [IF EXISTS] 'remote_user1'@'%' ACCOUNT LOCK;

DROP USER:

DROP USER 'remote_user1'@'%’;

MySQL Database Users Access Restrictions using privileges

Grant privileges to user:

Privileges can be granted on database and table only.

Examples.

Case1:  Grant all privileges on ‘db1’ database to user ‘remote_user1’@’%’

GRANT ALL ON db1.* TO 'remote_user1'@'%';

Case2: Grant selected privileges on ‘db1’ database to user ‘remote_user1’@’%’

GRANT SELECT, INSERT, UPDATE, DELETE ON db1.* TO 'remote_user1'@'%';

Case3. Grant SELECT privilege single table access to user ‘remote_user1’@’%’

GRANT SELECT ON db1.table1 TO 'remote_user1'@'%';

Revoking privileges from user:

Example:

REVOKE SELECT, INSERT, UPDATE, DELETE ON db1.* FROM 'remote_user1'@'%';

Check User Privileges using SHOW GRANTS command:

Example:

SHOW GRANTS FOR 'root'@'localhost';
SHOW GRANTS; (It will display the privileges granted to the current account)
SHOW GRANTS FOR 'remote_user1'@'%';

MySQL DB Backup and Restore:

Logical backup

The mysqldump client utility performs logical backups, which create a pain file with sql statements in it.

System database backup not required, so while taking backup only take backup of non-system database i.e. application databases

Require privileges for mysqldump :  SELECT, RELOAD, SHOW DATABASES, LOCK TABLES, SHOW VIEW, EVENT, TRIGGER

Full Database backup:

mysqldump -u root -h db_host -p --single-transaction --databases db1 --routines --events > db1_fullbkp.sql

 OR

mysqldump -u root -h db_host -p --single-transaction --databases db1 --routines --events | gzip > db1_fullbkp.sql.gz

Single table backup:

mysqldump -u db_username -h db_host -p --single-transaction db_name table_name > db1_full.sql

Restore:

To reload a dump file, you must have the privileges required to execute the statements that it contains.

mysql -u username -p db_name < db1_fullbkp.sql

OR

gunzip < db1_fullbkp.sql.gz | mysql -u username -p db_name

Note:  –single-transaction option for consistent non-blocking backup.

For InnoDB tables, it is possible to perform an online backup that takes no locks on tables using the –single-transaction option to mysqldump.

  • Create separate backup user with require backup privileges.
  • Schedule the backup script in crontab of mysql UNIX account.

Physical/Binary backup:

MySQL Enterprise Backup

Percona XtraBackup

Note: Binary backup mostly use for backing up large volume database.For small volume database use mysqldump.

MySQL DB Monitoring:

1. MySQL enterprise monitor

2.  MySQL workbench GUI tool.

3. Script automation : Write scripts with required monitoring command and automate this scripts using cronjobs. Email notification also can be added in this script for critical alerts.

– Create a separate user for monitoring with required privileges.

CREATE USER [IF NOT EXISTS] 'monitor'@'localhost' IDENTIFIED BY 'mon$pwd';
GRANT SELECT,PROCESS ON *.* TO 'monitor'@'localhost';

–  Create monitoring schedule it with cronjob for automatic runs.

Sample script To check server status:

#!/bin/sh

EMAIL_IDS=user1@abc.com,user2@abc.com

# Connection details

MYSQL_USER=monitor

MYSQL_PASS=mon$pwd

MYSQL_HOST=localhost

MYSQL_CONN="-u${MYSQL_USER} -p${MYSQL_PASS} -h${MYSQL_HOST}"

SERVER_HOST=$( hostname )

# MySQL status

mysqladmin ${MYSQL_CONN} ping 2>/dev/null 1>/dev/null

if [ $? -ne 0 ]; then

echo "MySQL Down" | mail -s " MySQL not running on $SERVER_HOST" "$EMAIL_IDS"

fi

NOTE: There are many other mysql monitoring command you can add in this script.

Replication :

We can setup relication using binlog and binlog position and other one is GTID based replication.Will setup GTID Replication which is new and more relable.

  1. Enable binary log and GTID on both master and slave.
  2. Create a replication user on MASTER as follow:
CREATE USER 'rpluser'@'%' IDENTIFIED BY 'rpluser1234';
GRANT REPLICATION SLAVE,REPLICATION CLIENT ON *.* TO 'rpluser'@'%';
FLUSH PRIVILEGES;

3. Setup replication on slave using CHANGE MASTER TO cmd as follow:

CHANGE MASTER TO MASTER_HOST='',MASTER_USER='rpluser',MASTER_PASSWORD='rpluser1234',MASTER_PORT=3306,MASTER_AUTO_POSITION=1;

4.  Start replication and check slave status.

START SLAVE;
SHOW SLAVE STATUS;

5 .  Slave_IO_Running and Slave_SQL_Running column value should be ‘YES’.

Check My Blog post : GTID Replication and troubleshooting

All set !!

Advertisements