#1 2025-01-20 18:34:20

dougienisbet
Member
2025-01-02
16

best practice for large amount of local photos

Hello,

I've installed piwigo on a few Debian bookworm machines and on my home server I decided to try and syncrhonise my entire photo collection. I did this using the 'ftp' method - except I used rsync to copy everything into ./galleries with the view to exploring how how I might automate the process a little in future (similar to https://piwigo.org/forum/viewtopic.php?id=31297 perhaps).

I have a lot of photos (130K / 578GB ) so I wasn't altogether surprised that things didn't go smoothly. An earlier trial with a smaller number was ok (27GB). I got a php error on my first attempt at synchronisation which I resolved by increasing the value in php.ini (it was looking for 586MB and I increased the value from 512 to 1024MB).

Now I'm getting a mysql error
[EDIT: here's the error-

Code:

Fatal error: Uncaught mysqli_sql_exception: Column 'name' cannot be null in /denweb/photos/include/dblayer/functions_mysqli.inc.php:132 Stack trace: #0 /denweb/photos/include/dblayer/functions_mysqli.inc.php(132): mysqli->query() #1 /denweb/photos/include/dblayer/functions_mysqli.inc.php(550): pwg_query() #2 /denweb/photos/admin/include/functions.php(1750): mass_inserts() #3 /denweb/photos/admin/site_update.php(881): tag_id_from_tag_name() #4 /denweb/photos/admin.php(406): include('...') #5 {main} thrown in /denweb/photos/include/dblayer/functions_mysqli.inc.php on line 132

]

that I suspect may be related to bad filename or corrupt file, but, I think I've just found the answer to the question I was about to ask ... which was ...

How to only synchronise part of the directory structure (mostly for debugging purposes to find out where the problem is), and I've just had a play with the 'Site Manager' tab and realise it allows me to create mini sites within the overall ./galleries/ structure, if I understand this correctly. So I think I can use this to narrow down where the problem will be, although it might take a while.

But the question about best practice remains. I'm perfectly comfortable with rsync, cron etc and would find it the most straightforward way to maintain a piwigo gallery, but I'm not sure if it's (still) the best way to do this. And like the poster in https://piwigo.org/forum/viewtopic.php?id=31297 I'm curious to investigate whether piwigo would be a feasable option for automating timelapse or periodical updates from webcams.

Environment

    Piwigo 15.3.0 Check for upgrade
    Installed on 8 January 2025, 1 week 4 days ago
    Operating system: Linux
    PHP: 8.2.26 (Show info) [2025-01-20 17:33:21]
    MySQL: 10.11.6-MariaDB-0+deb12u1 [2025-01-20 17:33:21]
    Graphics Library: ImageMagick ImageMagick 6.9.11-60
    Cache size 129.98 MB right now Refresh

Activated plugin list 5

    LocalFiles Editor
    OpenStreetMap
    Piwigo Failed Login logger
    Read Metadata
    Take A Tour of Your Piwigo

Last edited by dougienisbet (2025-01-20 19:31:16)

Offline

 

Board footer

Powered by FluxBB