Announcement

#16 2018-01-15 19:42:22

PolyWogg
Member
2014-08-17
72

Re: High I/O usage revisited...

This is an old post of mine, but I finally tripped over a solution.

We were on the right track -- the WordPress security plugins up high were corrupting my install, but simply disabling them wasn't fixing anything. So we eliminated them.

But it turned out that one of the most popular Security plugins in WP has a glitch from time to time -- it leaves the .HTACCESS file with all the inserted codes in it, even after it's disabled. So shutting it down, even deleting the plugin wasn't fixing the problem -- because it wasn't cleaning up the file before it left.

Which meant my .HTACCESS file was creating a recurring load loop, trying to recall itself over and over somehow everytime it loaded. So I might get "one" hit from say "China", but after that, the protection would create a recurring load of the file so it looked like China was hammering my site.

I didn't even know that was POSSIBLE.

However, I switched to another service provider, was fine for a few months, and then the security plugin created the same loop again when I put in new settings and finally it had it "tweaked" for a non-basic install. The tech support people at this new provider found the problem in less than two minutes...it was my .htaccess file that was messing everything up. Sheesh.

So much angst and work, and it was the WP plugin after all.

PolyWogg

Offline

 

#17 2018-01-15 19:58:54

flop25
Piwigo Team
2006-07-06
7037

Re: High I/O usage revisited...

wow what a story! Thank you for sharing the final reults and have fun now!


To get a better help : Politeness like Hello-A link-Your past actions precisely described
Check my extensions : more than 30 available
who I am and what I do : http://fr.gravatar.com/flop25
My gallery : an illustration of how to integrate Piwigo in your website

Offline

 

#18 2021-05-06 18:54:44

Tharius
Member
2020-05-08
13

Re: High I/O usage revisited...

Hi everyone!

I also have the "cheap hosting provider wants me to upgrade to solve my infrequent High I/O use" problem.

My environment is:

Piwigo 11.4.0
Operating system: Linux
PHP: 7.4.16 (Show info) [2021-05-06 12:39:13]
MySQL: 5.7.34 [2021-05-06 12:39:13]
Graphics Library: ImageMagick 7.0.10-10

Oddly my problems started precisely on May 01 which leads me to suspect a server side change instead of anything else. There was no Piwigo update at that time, the previous one being in March. I changed PHP versions from 5.6 to 7.4 on May 03. so this is also not the source of the issues.

I have followed this, and other threads, but since my I/O faults specifically happen when I generate thumbnails on album pages I have gone ahead and generated them offline.

I downloaded my uploads and data directory to my local machine through ftp to my hosting provider, i.e. entirely externally to Piwigo.  I ran exiftrans -ai on the folders to rotate the pictures en masse prior to generating thumbnails.

I'm a relatively casual Linux user, but I created this BASH script to run over the structure to generate all the files I required. Tested and used in the Window 10 Bash subsystem. No warranty is provided or implied. I hope this is helpful, feedback and comments are appreciated. If you improve the script, please share back.

Code:

# A script for offline creation of Piwigo thumbnails and alternate sized files.
#
# Easily expanded for more sizes, defined for default thumbnail, square, and medium sizes.
# Requires djpeg, pnmscale, and cjpeg which are found in the libjpeg/libturbojpeg package.
#
# Only processes jpg/jpeg files, please feel free to expand and share back.
#
# Directory assumptions: run from the piwigo/upload folder. Will iterate over years, months,
# and days but makes no attempt to crawl any further structure.
#
# The script assumes that the _data/i/upload/ directory exists and is referenced in BASE but
# will create the thumbnail directories as needed. Explicit or relative paths should work fine.
#
# If you manipulate the tree structure while the script is running, you own the broken results.
#
# I use the commandline ./makeThumbs.sh | tee thumb.txt for logging purposes. If you wish to speed
# the script up simply remove or comment out the echo lines.
#
# credits to https://www.cyberciti.biz/tips/howto-linux-creating-a-image-thumbnails-from-shell-prompt.html
# for the initial code

TYPES="*.jpg *.jpeg"
BASE=$(pwd)"/../_data/i/upload/"   # must be terminated with trailing "/"

for y in */; do
    echo "Processing uploads from year $y"
    cd $y
    if [ ! -d "$BASE$y" ]; then
        echo "Creating data directory $BASE$y"
        mkdir "$BASE$y"
    fi

    for m in */; do
        echo "Processing uploads from $y$m"
        cd $m
        if [ ! -d "$BASE$y$m" ]; then
            echo "Creating data directory $BASE$y$m"
            mkdir "$BASE$y$m"
        fi
        for d in */; do
            echo "Processing uploads from $y$m$d"
            cd $d
            if [ ! -d "$BASE$y$m$d" ]; then
                echo "Creating data directory $BASE$y$m$d"
                mkdir "$BASE$y$m$d"
            fi
               for f in $TYPES
               do
                 [ -f "$f" ] || continue
                 
                 NAME="${f%.*}"
                 EXTENSION="${f##*.}"
                 DIR=$BASE$y$m$d
                 TN=$NAME"-th."$EXTENSION
                 SQ=$NAME"-sq."$EXTENSION
                 ME=$NAME"-me."$EXTENSION
                 
                 echo "Checking "$(pwd)"/"$f
                 if [ ! -f $DIR$TN ]; then
                    FN=$DIR$TN
                    echo "Creating "$FN
                    djpeg $f | pnmscale -width 144 | cjpeg -optimize -progressiv -quality 75 > $FN
                 fi

                 if [ ! -f $DIR$SQ ]; then
                    FN=$DIR$SQ
                    echo "Creating "$FN
                    djpeg $f | pnmscale -xysize 120 120 | cjpeg -optimize -progressiv -quality 75 > $FN
                 fi

                 if [ ! -f $DIR$ME ]; then
                    FN=$DIR$ME
                    echo "Creating "$FN
                    djpeg $f | pnmscale -xysize 792 594 | cjpeg -optimize -progressiv -quality 95 > $FN
                 fi
               done
           cd ..
        done
        cd ..
    done
    cd ..
done

Offline

 

Board footer

Powered by FluxBB

github twitter newsletter Donate Piwigo.org © 2002-2024 · Contact