Author: tadmin

Upgrading MySQL 5.6 to 5.7 on CentOs

~ tadmin

First of all I would recommend you to make backups for all MySQL databases and configuration files you have.

The second recommendation is to use official Yum repositories located on http://dev.mysql.com/downloads/repo/yum/
It is the best possible option for CentOs and other Red Hat based Linux ditstibutions.

You should download repository depending on your distribution version.
For example on CentOs 6 you should use Red Hat Enterprise Linux 6 / Oracle Linux 6 (Architecture Independent), RPM Package – http://dev.mysql.com/downloads/file/?id=465605

Also, read the following page: https://dev.mysql.com/doc/mysql-repo-excerpt/5.7/en/updating-yum-repo.html before starting updates to understand how does it work better.

If you have previously installed mysql from different repository run command:

# yum repolist all | grep mysql

to find out all Yum repositories list which contains mysql.

Example output of such command on my server:

# yum repolist all | grep mysql
mysql-connectors-community MySQL Connectors Community disabled
mysql-connectors-community-source MySQL Connectors Community – S disabled
mysql-tools-community MySQL Tools Community enabled: 40
mysql-tools-community-source MySQL Tools Community – Source disabled
mysql-tools-preview MySQL Tools Preview disabled
mysql-tools-preview-source MySQL Tools Preview – Source disabled
mysql55-community MySQL 5.5 Community Server disabled
mysql55-community-source MySQL 5.5 Community Server – S disabled
mysql56-community MySQL 5.6 Community Server disabled
mysql56-community-source MySQL 5.6 Community Server – S disabled
mysql57-community MySQL 5.7 Community Server enabled: 146
mysql57-community-source MySQL 5.7 Community Server – S disabled
#

To select disable version 5.6 and select version 5.7 you can use commands such commands:

# sudo yum-config-manager --disable mysql56-community
# sudo yum-config-manager --enable mysql57-community

And then make all server packages update including mysql:

# sudo yum-update -y

Once all server updates are completed you should run:

# sudo mysql_upgrade --force

Some interesting task related to recursion and arrays in PHP

~ tadmin

Sometimes it is difficult to find best solution for some problem quickly.
For example, if you need to walk some nested array. You don’t know how deep it could be nested and extract values from it to make it flattern you can use amazing function array_walk_recursive together with anonymous function to solve such task like it is shown in example below:

$inputArray = [9, [7, 5, [11, [3]]], 14];
array_walk_recursive($inputArray, function($value, $key) use (&$resultArray) {
    $resultArray[] = $value;
});

This amazing example of magic is very simple: it calls our anonymous function for each $inputArray element, storing nested element values in $resultArray. Note, usage of & in use (&$resultArray) it tells we shoud pass variable by reference.

MySQL GROUP_CONCAT default result length & possible problems with big results

~ tadmin

If you use such function like GROUP_CONCAT you should be aware of default result size is 1024 bytes. It could lead possible problems.

For example, check the SQL query below:

set @user_ids = (SELECT GROUP_CONCAT(user_id ORDER BY user_id SEPARATOR ',') FROM user WHERE enabled = 1);

If you expect result to be list of IDs separated by comma and you have data result bigger than 1024 characters it will be truncated to default length. If you use such result in another SQL queries such as:

SELECT some_stuff FROM user_stuff WHERE user_id IN (@user_ids);

It could lead to possible logical problems when some IDs will be cropped and wrong data will be used.
To solve such problem you can change session variable using SQL query:

SET SESSION group_concat_max_len = 1000000;

Where 1000000 is maximum possible GROUP_CONCAT result size.
Also, you can set this variable globally (it is preffered way if you work with big data in most cases. Then you should add such configuration to your /etc/my.cnf

group_concat_max_len = 1000000;

and restart MySQL server to apply new default configuration.

OpenSSL generate CSR using secure SHA256 instead of SHA1

OpenSSL
~ tadmin

To generate RSA private key using 2048 long modulus use command:

# openssl genrsa -out www.shkodenko.com.key 2048

If you want to password protect your key add -des3 parameter. You can also increase key modulus to 4096 to make it more secure.

To generate CSR using secure SHA256 algorithm the following command can be used:

# openssl req -sha256 -new -key www.shkodenko.com.key -out www.shkodenko.com-sha256.csr

There is also genkey utility. By default it will generate CSR with SHA1 but you can change it in configuration file /etc/pki/tls/openssl.cnf:

# diff -Nau /etc/pki/tls/openssl.cnf-2015-06-26.bak /etc/pki/tls/openssl.cnf

— /etc/pki/tls/openssl.cnf-2015-06-26.bak 2015-01-13 13:34:32.000000000 +0000
+++ /etc/pki/tls/openssl.cnf 2015-06-26 16:13:28.000000000 +0000
@@ -104,7 +104,7 @@
####################################################################
[ req ]
default_bits = 2048
-default_md = sha1
+default_md = sha256
default_keyfile = privkey.pem
distinguished_name = req_distinguished_name
attributes = req_attributes

Linux malware detect aka maldet installation procedure

~ tadmin

I’m using the following procedure to install and configure Linux Malware Detect (LMD):

1. Download it:

# wget http://www.rfxn.com/downloads/maldetect-current.tar.gz

–2013-12-06 16:10:52– http://www.rfxn.com/downloads/maldetect-current.tar.gz
Resolving www.rfxn.com… 129.121.132.46
Connecting to www.rfxn.com|129.121.132.46|:80… connected.
HTTP request sent, awaiting response… 200 OK
Length: 811434 (792K) [application/x-gzip]
Saving to: `maldetect-current.tar.gz’

100%[====================================================================================================>] 811,434 –.-K/s in 0.1s

2013-12-06 16:10:53 (7.51 MB/s) – `maldetect-current.tar.gz’ saved [811434/811434]

2. Verify downloaded package:

# ls -alh maldetect-current.tar.gz

-rw-r–r– 1 root root 793K Apr 13 2013 maldetect-current.tar.gz

# sha1sum -b maldetect-current.tar.gz
fe2b0629ca473997afc717d56dc8e2bc2ec868a8 *maldetect-current.tar.gz

#

3. Unpack archive:

# tar xzvf maldetect-current.tar.gz

maldetect-1.4.2/
maldetect-1.4.2/CHANGELOG
maldetect-1.4.2/cron.daily
maldetect-1.4.2/README
maldetect-1.4.2/files/
maldetect-1.4.2/files/ignore_file_ext
maldetect-1.4.2/files/sigs/
maldetect-1.4.2/files/sigs/rfxn.ndb
maldetect-1.4.2/files/sigs/md5.dat
maldetect-1.4.2/files/sigs/hex.dat
maldetect-1.4.2/files/sigs/maldet.sigs.ver
maldetect-1.4.2/files/sigs/rfxn.hdb
maldetect-1.4.2/files/tmp/
maldetect-1.4.2/files/VERSION.hash
maldetect-1.4.2/files/ignore_inotify
maldetect-1.4.2/files/hexstring.pl
maldetect-1.4.2/files/maldet
maldetect-1.4.2/files/conf.maldet
maldetect-1.4.2/files/clean/
maldetect-1.4.2/files/clean/gzbase64.inject.unclassed
maldetect-1.4.2/files/clean/base64.inject.unclassed
maldetect-1.4.2/files/sess/
maldetect-1.4.2/files/pub/
maldetect-1.4.2/files/internals.conf
maldetect-1.4.2/files/quarantine/
maldetect-1.4.2/files/inotify/
maldetect-1.4.2/files/inotify/tlog
maldetect-1.4.2/files/inotify/inotifywait
maldetect-1.4.2/files/inotify/libinotifytools.so.0
maldetect-1.4.2/files/hexfifo.pl
maldetect-1.4.2/files/ignore_paths
maldetect-1.4.2/files/ignore_sigs
maldetect-1.4.2/files/modsec.sh
maldetect-1.4.2/cron.d.pub
maldetect-1.4.2/COPYING.GPL
maldetect-1.4.2/.ca.def
maldetect-1.4.2/install.sh
#

4. Install it:

# cd maldetect-1.4.2/
# ./install.sh

Linux Malware Detect v1.4.1
(C) 2002-2013, R-fx Networks (C) 2013, Ryan MacDonald
inotifywait (C) 2007, Rohan McGovern
This program may be freely redistributed under the terms of the GNU GPL

installation completed to /usr/local/maldetect
config file: /usr/local/maldetect/conf.maldet
exec file: /usr/local/maldetect/maldet
exec link: /usr/local/sbin/maldet
exec link: /usr/local/sbin/lmd
cron.daily: /etc/cron.daily/maldet

maldet(12981): {sigup} performing signature update check…
maldet(12981): {sigup} local signature set is version 25
maldet(12981): {sigup} new signature set (260) available
maldet(12981): {sigup} downloaded http://www.rfxn.com/downloads/md5.dat
maldet(12981): {sigup} downloaded http://www.rfxn.com/downloads/hex.dat
maldet(12981): {sigup} downloaded http://www.rfxn.com/downloads/rfxn.ndb
maldet(12981): {sigup} downloaded http://www.rfxn.com/downloads/rfxn.hdb
maldet(12981): {sigup} downloaded http://www.rfxn.com/downloads/maldet-clean.tgz
maldet(12981): {sigup} signature set update completed
maldet(12981): {sigup} 11362 signatures (9490 MD5 / 1872 HEX)
#

5. Edit configuration file to send e-mail notifications to administrator:

# conf /usr/local/maldetect/conf.maldet

# The destination addresses for email alerts
# [ values are comma (,) spaced ]
email_addr=”taras@shkodenko.com”

# ls -alh /usr/local/maldetect/conf.malde*

-rw-r–r– 1 root root 4.3K Dec 6 16:12 /usr/local/maldetect/conf.maldet
-rw-r–r– 1 root root 4.3K Apr 9 2013 /usr/local/maldetect/conf.maldet-2013-12-06.bak

# sha1sum -b /usr/local/maldetect/conf.malde*

8f28ec69d1007120d4b96f99cb14eb219091134d */usr/local/maldetect/conf.maldet
586016e43d31454e683320c18fe79c7e9e3273ce */usr/local/maldetect/conf.maldet-2013-12-06.bak
#

6. View cron job shell script which was added to run malware check on daily basis:
# more /etc/cron.daily/maldet

#!/bin/bash

# clear quarantine/session/tmp data every 14 days
/usr/sbin/tmpwatch 336 /usr/local/maldetect/tmp >> /dev/null 2>&1
/usr/sbin/tmpwatch 336 /usr/local/maldetect/sess >> /dev/null 2>&1
/usr/sbin/tmpwatch 336 /usr/local/maldetect/quarantine >> /dev/null 2>&1
/usr/sbin/tmpwatch 336 /usr/local/maldetect/pub/*/ >> /dev/null 2>&1

# check for new release version
/usr/local/maldetect/maldet -d >> /dev/null 2>&1

# check for new definition set
/usr/local/maldetect/maldet -u >> /dev/null 2>&1

# if were running inotify monitoring, send daily hit summary
if [ "$(ps -A --user root -o "comm" | grep inotifywait)" ]; then
        /usr/local/maldetect/maldet --alert-daily >> /dev/null 2>&1
else
        # scan the last 2 days of file changes
        if [ -d "/home/virtual" ] && [ -d "/usr/lib/opcenter" ]; then
                # ensim
                /usr/local/maldetect/maldet -b -r /home/virtual/?/fst/var/www/html 2 >> /dev/null 2>&1
                /usr/local/maldetect/maldet -b -r /home/virtual/?/fst/home/?/public_html 2 >> /dev/null 2>&1
        elif [ -d "/etc/psa" ] && [ -d "/var/lib/psa" ]; then
                # psa
                /usr/local/maldetect/maldet -b -r /var/www/vhosts/?/httpdocs 2 >> /dev/null 2>&1
                /usr/local/maldetect/maldet -b -r /var/www/vhosts/?/subdomains/?/httpdocs 2 >> /dev/null 2>&1
        elif [ -d "/usr/local/directadmin" ]; then
                # DirectAdmin
                /usr/local/maldetect/maldet -b -r /var/www/html/?/ 2 >> /dev/null 2>&1
                /usr/local/maldetect/maldet -b -r /home?/?/domains/?/public_html 2 >> /dev/null 2>&1
        else
                # cpanel, interworx and other standard home/user/public_html setups
                /usr/local/maldetect/maldet -b -r /home?/?/public_html 2 >> /dev/null 2>&1
        fi

        # scan default apache docroot paths
        if [ -d "/var/www/html" ]; then
                /usr/local/maldetect/maldet -b -r /var/www/html 2 >> /dev/null 2>&1
        fi
        if [ -d "/usr/local/apache/htdocs" ]; then
                /usr/local/maldetect/maldet -b -r /usr/local/apache/htdocs 2 >> /dev/null 2>&1
        fi
fi

#

You can edit it to add your web server specific layout if it does not looks like: cPanel, DirectAdmin, Plesk or any of other shown above.

nginx: [warn] 4096 worker_connections exceed open file resource limit: 1024

~ tadmin

If you see error messages like:
nginx: [warn] 4096 worker_connections exceed open file resource limit: 1024

Then checking nginx web server configuration using command:

# service nginx configtest

It can be fixed permanently (after system reboot) by editing /etc/security/limits.conf configuration file:

#@student        -       maxlogins       4

* soft nofile 65536
* hard nofile 65536

# End of file

Default Linux system limits can be viewed using command:

# ulimit -a

core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 14586
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 14586
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

To fix open files limit value temporary (it works until server reboot) use command:

# ulimit -n 65536

To view new values of system limits run once again:

# ulimit -a

core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 14586
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 65536
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 14586
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

Then checking nginx web server configuration use command:

# service nginx configtest

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

And, finally, restart nginx web server using command:

# service nginx restart

Stopping nginx: [ OK ]
Starting nginx: [ OK ]
#

Errors running mysqldump related to LOCK TABLES

~ tadmin

If you see such error patterns when running mysqldump backup:

# mysqldump -u wordpress_u -p wordpress_db > ./tmp/db_backups/wordpress_db-`date +'%Y%m%d%H%M'`.sql

Enter password:
mysqldump: Got error: 1016: Can’t open file: ‘./wordpress_db/wp_1050_postmeta.frm’ (errno: 24) when using LOCK TABLES

# mysqldump -u wordpress_u -p wordpress_db > ./tmp/db_backups/wordpress_db-`date +'%Y%m%d%H%M'`.sql

Enter password:
mysqldump: Got error: 23: Out of resources when opening file ‘./wordpress_db/wp_1050_options.MYD’ (Errcode: 24) when using LOCK TABLES

try to add –lock-tables=false to dump command.

Work with davfs2 on Fedora CentOs RHEL

~ tadmin

This post can be useful if you need to work with remote file system using davfs2 protocol.

To start work with it you should install davfs2 support using command:

# sudo yum install davfs2 -y

To start work you have to create mount point using command:

# sudo mkdir /mnt/drive_davfs2

Mount drive using command:

# sudo mount -t davfs https://file-store.shkodenko.com/ /mnt/drive_davfs2

You will need to provide username and password to connect.

Now, you can work with remote DAVFS folder /mnt/drive_davfs2 as with local folder.

To view list of currently mounted filesystems use command:

# sudo mount -l

Then you don’t need remote folder you should unmount it using command:

# sudo umount /mnt/drive_davfs2

RHEL Downloadonly plugin for yum

~ tadmin

If you need to get package source code only, and you don’t want to install it, you can use Downloadonly plugin for yum.

Install the package including “downloadonly” plugin:

(RHEL5)

# yum install yum-downloadonly

(RHEL6)

# yum install yum-plugin-downloadonly

Run yum command with “–downloadonly” option as follows:

# yum install --downloadonly --downloaddir=<directory> <package>

Confirm the RPM files are available in the specified download directory.

Source: https://access.redhat.com/solutions/10154

Import SSL certificate from web server Apache OpenSSL to Tomcat format

~ tadmin

To import SSL certificate into pkcs12 format you can use command:

# openssl pkcs12 -export -in /home/taras/ssl/shkodenko.com.crt -inkey /home/taras/ssl/shkodenko.com.key -out /home/taras/ssl/shkodenko.com.p12 -name tomcat -CAfile /home/taras/ssl/shkodenko.com.ca.crt -caname root -chain
Enter Export Password:
Verifying - Enter Export Password:

You will be prompted to input password.

# keytool -importkeystore -srckeystore /home/taras/ssl/shkodenko.com.p12 -srcstoretype PKCS12 -deststoretype JKS -destkeystore /home/taras/ssl/shkodenko.com.jks
Enter destination keystore password:  
Re-enter new password: 
Enter source keystore password:  
Entry for alias tomcat successfully imported.
Import command completed:  1 entries successfully imported, 0 entries failed or cancelled

Now, you can use /home/taras/ssl/shkodenko.com.jks certificates storage in your Apache Tomcat SSL virtual host configuration.