Announcement

  •  » Miscellaneous
  •  » Script to generate thumbnails (2023 edition)

#1 2023-02-14 19:20:17

GJSchaller
Member
2022-05-26
16

Script to generate thumbnails (2023 edition)

In years past, I had a Piwigo install on my ReadyNAS, and there was a script I could run via CRON that would go through and generate any missing thumbnails on a regular basis, so that they were already present when the site was loaded and did not need to be generated on the fly.  This also let me add new photos, and the scheduled script would process them the next time it ran without me needing to worry about it.

I've since then moved from the ReadyNAS over to a TrueNAS, which is running Piwigo in a FreeBSD Jail / VM.  I dug up my old notes, but can't seem to get the script to work under FreeBSD.

Anyone have a way to generate thumbnails automatically via a script in a FreeBSD environment, that I can schedule?

Piwigo 13.5.0
Installed on 24 May 2022, 8 months 2 weeks 6 days ago
Operating system: FreeBSD
PHP: 8.0.26
MySQL: 10.3.37-MariaDB
Graphics Library: External ImageMagick 7.1.0-50


Piwigo URL: http://photos.aiskon.net

Thank you!

Offline

 

#2 2023-02-14 19:26:29

GJSchaller
Member
2022-05-26
16

Re: Script to generate thumbnails (2023 edition)

This is the old script I had under ReadyNAS, which was a Debian base, if I remember correctly.  Obviously, the URL and path(s) would change, but the rest of the syntax is where I am clueless.

+++

wget --keep-session-cookies --save-cookies /home/admin/cookies.txt --delete-after --post-data="username=cron&password=RwpJYB^Zz#mQ^6" "https://gjschaller.homeftp.net/piwigo/ws.php?format=json&method=pwg.session.login"

wget --load-cookies /home/admin/cookies.txt -nv -O missing.json "https://gjschaller.homeftp.net/piwigo/ws.php?format=json&method=pwg.getMissingDerivatives"
while [ `wc -c missing.json | cut -f 1 -d ' '` -gt 50 ]
do
  sed -e 's/[\\\"]//g' \
  -e 's/{stat:ok,result:{next_page:[0-9]*,urls:\[//' \
  -e 's/{stat:ok,result:{urls:\[//' \
  -e 's/\]}}/\n/' \
  -e 's/,/\n/g' \
  -e 's/\&b=[0-9]*//g' missing.json | \
  while read line ; do
    wget -nv -O /dev/null $line
  done
  wget --load-cookies /home/admin/cookies.txt -nv -O missing.json "https://gjschaller.homeftp.net/piwigo/ws.php?format=json&method=pwg.getMissingDerivatives"
done

rm /home/admin/cookies.txt

+++

Thank you for any help and advice!

Last edited by GJSchaller (2023-02-14 19:31:34)

Offline

 

#3 2023-02-15 02:39:34

erAck
Only trying to help
2015-09-06
2035

Re: Script to generate thumbnails (2023 edition)

And what in the script does not work?
Quite a good password by the way..


Running Piwigo at https://erack.net/gallery/

Offline

 

#4 2023-02-21 15:22:30

GJSchaller
Member
2022-05-26
16

Re: Script to generate thumbnails (2023 edition)

Heh, I forgot that was in there, in plain text - that's been changed.

OK, so I modified the script to work under the FreeBSD file system, running as root.  I made sure wget was installed, and now have some progress.  The new script:

wget --keep-session-cookies --save-cookies /root/cookies.txt --delete-after --post-data="username=cron&password=REDACTED" "https://photos.aiskon.net/piwigo/ws.php?format=json&method=pwg.session.login"

wget --load-cookies /root/cookies.txt -nv -O missing.json "https://photos.aiskon.net/piwigo/ws.php?format=json&method=pwg.getMissingDerivatives"

while [ `wc -c missing.json | cut -f 1 -d ' '` -gt 50 ]
do
  sed -e 's/[\\\"]//g' \
  -e 's/{stat:ok,result:{next_page:[0-9]*,urls:\[//' \
  -e 's/{stat:ok,result:{urls:\[//' \
  -e 's/\]}}/\n/' \
  -e 's/,/\n/g' \
  -e 's/\&b=[0-9]*//g' missing.json | \
  while read line ; do
    wget -nv -O /dev/null $line
  done
  wget --load-cookies /root/cookies.txt -nv -O missing.json "https://photos.aiskon.net/piwigo/ws.php?format=json&method=pwg.getMissingDerivatives"
done

rm /root/cookies.txt

The output when I run it:

root@piwigo:~ # ./piwigo-generate.sh
--2023-02-21 09:14:58--  https://photos.aiskon.net/piwigo/ws.php … sion.login
Resolving photos.aiskon.net (photos.aiskon.net)... 10.0.0.241
Connecting to photos.aiskon.net (photos.aiskon.net)|10.0.0.241|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: ‘ws.php?format=json&method=pwg.session.login.tmp’

ws.php?format=json&     [ <=>                ]   8.90K  --.-KB/s    in 0.001s

2023-02-21 09:14:58 (5.84 MB/s) - ‘ws.php?format=json&method=pwg.session.login.tmp’ saved [9113]

Removing ws.php?format=json&method=pwg.session.login.tmp.
2023-02-21 09:14:58 URL:https://photos.aiskon.net/piwigo/ws.php?format=json&method=pwg.getMissingDerivatives [9121] -> "missing.json" [1]
[: -gt: unexpected operator
root@piwigo:~ #

So it looks like it's failing on the "While" line of the script?

Offline

 

#5 2023-02-21 18:39:14

erAck
Only trying to help
2015-09-06
2035

Re: Script to generate thumbnails (2023 edition)

Make sure that the shell pulled by the script actually understands the -gt operator without forcing arithmetic expressions. Create a script with

[ 100 -gt 50 ] && echo yes

and run it; should output yes and not fail. If it fails then try with

[[ 100 -gt 50 ]] && echo yes

or

if ((100 > 50)); then echo yes; fi

You get the idea..
Or force the shell with a shebang as first line, like
#!/usr/bin/bash

But maybe your FreeBSD comes only with the default Bourne Shell /bin/sh installed; with exceptionally bad luck you get a tcsh instead ...

Other than that, make sure that wc and cut are installed and deliver expected results, i.e.
wc -c /dev/null
must display 0 /dev/null and
wc -c /dev/null | cut -f 1 -d ' '
must display 0


Running Piwigo at https://erack.net/gallery/

Offline

 

#6 2023-02-21 19:45:34

GJSchaller
Member
2022-05-26
16

Re: Script to generate thumbnails (2023 edition)

Thank you, this is helpful!

It looks like they're installed, and wc works.  cut doesn't give expected output - I did install it via pkg install cut, then re-ran the test, this is what I have:

(MOTD Text)
root@piwigo:~ # [ 100 -gt 50 ] && echo yes
yes
root@piwigo:~ # wc -c /dev/null
       0 /dev/null
root@piwigo:~ # wc -c /dev/null | cut -f 1 -d ' '

root@piwigo:~ #

Thank you for your help on this - if I need to move this over to a FreeBSD support forum, I can.

Offline

 

#7 2023-02-22 17:19:14

erAck
Only trying to help
2015-09-06
2035

Re: Script to generate thumbnails (2023 edition)

Your problem apparently is that wc prefixes its output with spaces (to right align?) so cut delivers an empty first field. Instead, you could use

Code:

wc -c missing.json | awk '{print $1}'

Running Piwigo at https://erack.net/gallery/

Offline

 

#8 2023-02-22 17:26:20

GJSchaller
Member
2022-05-26
16

Re: Script to generate thumbnails (2023 edition)

Thank you.  Making sure I'm not making any mistakes, the full line would be:

while [ `wc -c missing.json | awk '{print $1}'` -gt 50 ]

I'm still new to FreeBSD and my linux is rusty, so I want to be sure I've got all the syntax correct with another set of eyes.

Thank you!

Offline

 

#9 2023-02-22 21:06:13

erAck
Only trying to help
2015-09-06
2035

Re: Script to generate thumbnails (2023 edition)

Looks good.


Running Piwigo at https://erack.net/gallery/

Offline

 

#10 2023-02-22 21:19:53

GJSchaller
Member
2022-05-26
16

Re: Script to generate thumbnails (2023 edition)

I realized I had a wrong URL in there - the "(domain)/piwigo/" was from the old NAS, once I updated with your syntax and corrected the URL, it seems to be working!

Offline

 
  •  » Miscellaneous
  •  » Script to generate thumbnails (2023 edition)

Board footer

Powered by FluxBB

github twitter newsletter Donate Piwigo.org © 2002-2024 · Contact