🌍
English
Hello/Hi/Greetings,
I recently installed piwigo using the netinstall script. It's gone fairly smoothly until recently. I'm slowly transfering about 50,000 photos from my static website into piwigo. My hosting server (lunarpages, whom I'm otherwise very happy with) disabled my account due to high CPU usage. I'd like to avoid this in the future. Generating thumbnails also takes a while on a shared server. One way to deal with this is to only move over a few photos a day; not a reasonable solution with as many pictures as I have or when I add 2,000 photos at once.
My preferred solution is to use my powerful home workstation to do the imagemagick converting and upload the results.
I imagine this working in a few steps:
1. Upload pictures as normal through the administrator web interface.
2. rsync a certain portion of the files down to my local workstation
3. run a batch script to process the files into the correct size and location LOCALLY
4. rsync the resulting generated images back to their appropriate location on the HOST server
5. continue using Piwigo as if I had done the conversion on the HOST
A few issues or potential problems:
A. Does the database need to be updated with any information about the resized images? Will it know where to look for them if the conversion is done outside of the administration interface?
B. Where do I find out which commands Piwigo runs to generate the images? I know it's /usr/bin/convvert $OPTIONS file-in file-out. Alternatively, could I just run the correct php script locally to do this? Is image.class.php the right file? It appears to have the pwg_resize function defined in it.
I'd like to avoid setting up a mirror server to do this. I know that uploaded photos go in ./_data/i/upload/ with the file name formatted yyyy/mm/dd-hash?-suffix? What else would I need to know?
Thanks!
Piwigo version: Piwigo 2.8.6
Operating system: Linux
PHP: 5.6.28 (Show info) [2017-03-03 17:28:46]
MySQL: 5.6.33 [2017-03-03 17:28:46]
Graphics Library: External ImageMagick 7.0.1-10
Piwigo URL: http://orobouros.net/piwigo
Offline
An update:
I tried writting a replacement for convert which would simply pass along the arguments as text into the file, instead of generating an actual image. The idea being that I could replace the images by locally running the contents of the files. For some reason, when I try to replace the convert binary with a script, pwigo no longer recognizes the imagemagick installation and the status says I have GD installed.
Is this a bug in Piwigo? It's not recoginizing the imagemagick installation.
Offline
I've found a solution that's pretty much a dirty hack of the code. Details will follow below, but I basically created a script to use in place of convert, which spits out shell scripts instead of images. I copy it all to my local machine, generate the images, and then copy it all back.
Here's the basic steps:
Patch the file ./admin/include/image.class.php as follows:
--- image.class.orig_php 2017-03-04 10:53:01.000000000 -0500
+++ image.class.php 2017-03-04 10:53:45.703998863 -0500
@@ -636,7 +636,7 @@
$this->add_command('sampling-factor', '4:2:2' );
}
- $exec = $this->imagickdir.'convert';
+ $exec = $this->imagickdir.'fakeconv';
$exec .= ' "'.realpath($this->source_filepath).'"';
foreach ($this->commands as $command => $params)Also copy the whole directory where convert lives (probably /usr/local/bin or similar) to your own area, say
rsync /usr/local/bin/ /home/$(whoami)/bin/imagemagick
Then include the following shell script, changing the !# if necessary. Name it "fakeconv" and place it in your new personal imagemagick director. It's essentially a "fake convert" script that intercepts the calls to the binary and produces an empty file and a shell script instead.
#!/bin/sh
REPURL="/home/$(whoami)/public_html/piwigo"
function echo_inargs {
echo "# All args were: $@"
echo echo Converting: $OUTURL
echo convert $@
echo "# Convert redirected at $(date)"
}
function fakeversion {
shift
convert --version $@
}
OUTURL=$(echo "${@: -1}" | sed "s|$REPURL|.|g")
if [[ "$1" == "--version" ]]; then
fakeversion $@
else
echo_inargs $@ | sed "s|$REPURL|.|g" > ${@: -1}.pwg.sh
touch ${@: -1}
fiFinally, pull all this data down to your local machine, run the shell scripts locally (assuming you have imagemagick installed), and then upload it all again. So far, it semes like Piwigo doesn't know the difference. I wrote a small script to do this easily.
#!/bin/sh
BASEURL="username@hostprovider.xyz:/home/username/public_html/piwigo"
PICDATA='upload/2017/02/28'
LOCALDIR="/home/$(whoami)/piwigo"
mkdir -p $LOCALDIR/{,_data/i/}$PICDATA
\rsync -avv $BASEURL/$PICDATA/ $LOCALDIR/$PICDATA/
\rsync -avv $BASEURL/_data/i/$PICDATA/ $LOCALDIR/_data/i/$PICDATA/
cd $LOCALDIR
source <(cat $(find . -name "*.pwg.sh"))
# \rsync -avv $LOCALDIR/$PICDATA/ $BASEURL/$PICDATA/
\rsync -avv $LOCALDIR/_data/i/$PICDATA/ $BASEURL/_data/i/$PICDATA/
cd -I turned off the options to automatically scale down images upon upload because it caused problems. You'll also have to change your local settings for where to find imagemagick. Finally, you'll want to clean up all the *.pwg.sh files laying around. But so far, this is working for me.
Offline