Several users have reported the same problem: the "upload" directory has been deleted after an upgrade to Piwigo 13.3.0. We now have quite a clear view about what happened. Here is the timeline:

[November 28th 2022] version 13.3.0 is released, with the archive to upgrade from 13.x to 13.3.0

[December 8th 2022] erAck, very active contributor, tells me this upgrade archive is not complete, see [Github] Piwigo issue #1818 I release a new archive by using the described procedure.

[December 11th 2022] YannB (on the French forum) tells us some photos have been deleted from his "upload" directory. At this time I think it's a one-time issue, not linked to the upgrade itself. I don't investigate any further.

[December 15th 2022] other users boge90 and FaBay89 report the same problem on [Github] Piwigo issue #1827

[December 15th 2022, still] I don't believe in coincidence. This issue becomes my top priority. I work all the afternoon with ddtddt to investigate and find solutions. We quickly find the problem (in the upgrade archive, we find an "obsolete.list" file with a single line "upload" in it, which leads to the automatic deletion of directory "upload" during upgrade). I can't reproduce the creation of this obsolete.list file and I know the upgrade archive has not been hacked. I (plg) have created this file (more precisely the script that generate the upgrade archive, but it's me who wrote/run this script) but I can't understand why yet.

We immediately update the archive to remove this obsolete.list file. All Piwigo installations who automatically updated a 13.x version to 13.3.0 between December 8th and 15th are concerned. Drops of sweat bead on my forehead.

We change the upgrade archive generator so that this problem can't happen ever again. We also think about the most efficient way to warn concerned users. Happily the vast majority of upgrades to 13.3.0 occured before December 8th.

I enhanced [extension by plg] Check Uploads so that it can check every photo listed in the database exists on the file sytem.

[December 16th 2022] I implement an automatic check on a sample of photos to make sure they're available on the filesystem, see [Github] Piwigo commit 97b665d8 if you like details. If any photo is missing you get a huge warning in the administration panel, asking to find details with the Check Uploads plugin.

Because we care a lot about privacy, we don't know who did the upgrade and when. It would have been very nice to have a list of email address and be able to contact them directly. But we can't do that, because of privacy.

Piwigo 13.4.0 is released. Of course, I tripled check the upgrade archive.

[December 19th 2022] I think I have an explanation for the arrival of the "upload" line in the obsolete.list file of the upgrade archive. When I create these upgrade archives, I create a /tmp/13.3.0 directory on my MacOS (I make it explicit because it's important). Then I extract files from piwigo-13.2.0.zip and piwigo-13.3.0.zip and run the create_patch.pl script to create the upgrade archive. The script compares directory 1320 and 1330: if a file is new or modified, it's added in the upgrade archive, if a file is not in 1330 it is added to the obsolete.list file.

Why would the "upload" be gone from the "1330" directory? It seems that MacOS, once a day, automatically removes all empty directories older than X days. My guess is that I may have used the same unzipped 1330 directory from November 28th 2022 to regenerate the patch on December 8th 2022. Due to automatic removal by MacOS, the "upload" directory was no longer there. Please note that I don't remember having never regenerate an upgrade archive. The lack of "1330/upload" was considered as an obsolete directory, thus to remove :-/

This whole situation "sucks". I obviously did not want it to happen and I transparently provide here the details. What I deeply hope is that concerned users, and all users in general, follow our recommendations:

upgrade guide wrote:

Always have a backup of your database and files. The best is to have them made automatically on a regular basis. If anything bad happens during the update, you would be able to retore a backup.

You see here it's a combination of "worst case scenario". If the bug had been obvious I would have detected it during my upgrade test even before releasing it (because yes, all the upgrade archives are tested). Maybe I should have created a version 13.4.0 on December 8th instead of updating the archive, but I didn't see the point since there was absolutely no change compared to 13.3.0.

We have improved the checks during upgrade archive creation and we're also going to put all upgrades (and plugins install/upgrade/deletion) in a specific history. This way will be able to provide this information on the maintenance page.