In years past, I had a Piwigo install on my ReadyNAS, and there was a script I could run via CRON that would go through and generate any missing thumbnails on a regular basis, so that they were already present when the site was loaded and did not need to be generated on the fly. This also let me add new photos, and the scheduled script would process them the next time it ran without me needing to worry about it.
I've since then moved from the ReadyNAS over to a TrueNAS, which is running Piwigo in a FreeBSD Jail / VM. I dug up my old notes, but can't seem to get the script to work under FreeBSD.
Anyone have a way to generate thumbnails automatically via a script in a FreeBSD environment, that I can schedule?
Piwigo 13.5.0
Installed on 24 May 2022, 8 months 2 weeks 6 days ago
Operating system: FreeBSD
PHP: 8.0.26
MySQL: 10.3.37-MariaDB
Graphics Library: External ImageMagick 7.1.0-50
Piwigo URL: http://photos.aiskon.net
Thank you!
Offline
This is the old script I had under ReadyNAS, which was a Debian base, if I remember correctly. Obviously, the URL and path(s) would change, but the rest of the syntax is where I am clueless.
+++
wget --keep-session-cookies --save-cookies /home/admin/cookies.txt --delete-after --post-data="username=cron&password=RwpJYB^Zz#mQ^6" "https://gjschaller.homeftp.net/piwigo/ws.php?format=json&method=pwg.session.login"
wget --load-cookies /home/admin/cookies.txt -nv -O missing.json "https://gjschaller.homeftp.net/piwigo/ws.php?format=json&method=pwg.getMissingDerivatives"
while [ `wc -c missing.json | cut -f 1 -d ' '` -gt 50 ]
do
sed -e 's/[\\\"]//g' \
-e 's/{stat:ok,result:{next_page:[0-9]*,urls:\[//' \
-e 's/{stat:ok,result:{urls:\[//' \
-e 's/\]}}/\n/' \
-e 's/,/\n/g' \
-e 's/\&b=[0-9]*//g' missing.json | \
while read line ; do
wget -nv -O /dev/null $line
done
wget --load-cookies /home/admin/cookies.txt -nv -O missing.json "https://gjschaller.homeftp.net/piwigo/ws.php?format=json&method=pwg.getMissingDerivatives"
done
rm /home/admin/cookies.txt
+++
Thank you for any help and advice!
Last edited by GJSchaller (2023-02-14 19:31:34)
Offline
And what in the script does not work?
Quite a good password by the way..
Offline
Heh, I forgot that was in there, in plain text - that's been changed.
OK, so I modified the script to work under the FreeBSD file system, running as root. I made sure wget was installed, and now have some progress. The new script:
wget --keep-session-cookies --save-cookies /root/cookies.txt --delete-after --post-data="username=cron&password=REDACTED" "https://photos.aiskon.net/piwigo/ws.php?format=json&method=pwg.session.login"
wget --load-cookies /root/cookies.txt -nv -O missing.json "https://photos.aiskon.net/piwigo/ws.php?format=json&method=pwg.getMissingDerivatives"
while [ `wc -c missing.json | cut -f 1 -d ' '` -gt 50 ]
do
sed -e 's/[\\\"]//g' \
-e 's/{stat:ok,result:{next_page:[0-9]*,urls:\[//' \
-e 's/{stat:ok,result:{urls:\[//' \
-e 's/\]}}/\n/' \
-e 's/,/\n/g' \
-e 's/\&b=[0-9]*//g' missing.json | \
while read line ; do
wget -nv -O /dev/null $line
done
wget --load-cookies /root/cookies.txt -nv -O missing.json "https://photos.aiskon.net/piwigo/ws.php?format=json&method=pwg.getMissingDerivatives"
done
rm /root/cookies.txt
The output when I run it:
root@piwigo:~ # ./piwigo-generate.sh
--2023-02-21 09:14:58-- https://photos.aiskon.net/piwigo/ws.php … sion.login
Resolving photos.aiskon.net (photos.aiskon.net)... 10.0.0.241
Connecting to photos.aiskon.net (photos.aiskon.net)|10.0.0.241|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: ‘ws.php?format=json&method=pwg.session.login.tmp’
ws.php?format=json& [ <=> ] 8.90K --.-KB/s in 0.001s
2023-02-21 09:14:58 (5.84 MB/s) - ‘ws.php?format=json&method=pwg.session.login.tmp’ saved [9113]
Removing ws.php?format=json&method=pwg.session.login.tmp.
2023-02-21 09:14:58 URL:https://photos.aiskon.net/piwigo/ws.php?format=json&method=pwg.getMissingDerivatives [9121] -> "missing.json" [1]
[: -gt: unexpected operator
root@piwigo:~ #
So it looks like it's failing on the "While" line of the script?
Offline
Make sure that the shell pulled by the script actually understands the -gt operator without forcing arithmetic expressions. Create a script with
[ 100 -gt 50 ] && echo yes
and run it; should output yes and not fail. If it fails then try with
[[ 100 -gt 50 ]] && echo yes
or
if ((100 > 50)); then echo yes; fi
You get the idea..
Or force the shell with a shebang as first line, like
#!/usr/bin/bash
But maybe your FreeBSD comes only with the default Bourne Shell /bin/sh installed; with exceptionally bad luck you get a tcsh instead ...
Other than that, make sure that wc and cut are installed and deliver expected results, i.e.
wc -c /dev/null
must display 0 /dev/null and
wc -c /dev/null | cut -f 1 -d ' '
must display 0
Offline
Thank you, this is helpful!
It looks like they're installed, and wc works. cut doesn't give expected output - I did install it via pkg install cut, then re-ran the test, this is what I have:
(MOTD Text)
root@piwigo:~ # [ 100 -gt 50 ] && echo yes
yes
root@piwigo:~ # wc -c /dev/null
0 /dev/null
root@piwigo:~ # wc -c /dev/null | cut -f 1 -d ' '
root@piwigo:~ #
Thank you for your help on this - if I need to move this over to a FreeBSD support forum, I can.
Offline
Your problem apparently is that wc prefixes its output with spaces (to right align?) so cut delivers an empty first field. Instead, you could use
wc -c missing.json | awk '{print $1}'
Offline
Thank you. Making sure I'm not making any mistakes, the full line would be:
while [ `wc -c missing.json | awk '{print $1}'` -gt 50 ]
I'm still new to FreeBSD and my linux is rusty, so I want to be sure I've got all the syntax correct with another set of eyes.
Thank you!
Offline
Looks good.
Offline
I realized I had a wrong URL in there - the "(domain)/piwigo/" was from the old NAS, once I updated with your syntax and corrected the URL, it seems to be working!
Offline