Announcement

#1 2024-03-10 06:03:25

macstrat
Member
2016-05-10
13

Unable to sort FTP uploaded albums

Hello/Hi/Greetings,

If I upload files via ftp into the galleries folder, I cannot sort the albums in the album manager...However if I upload via the java uploader (when it works, it has not been uploading all the files and erroring out due to an error) or by the web interface to the uploads folder, I can sort them in the album manager.

Its a fresh install to play around with and everything else seems to be working fine except being able to sort albums.

Am I missing something?

Not having the ability to sort albums or properly upload files is a real killer.

Environment

    Piwigo 14.3.0 Check for upgrade
    Installed on 10 March 2024, 3 hours ago
    Operating system: Linux
    PHP: 8.1.2-1ubuntu2.14 (Show info) [2024-03-10 04:57:56]
    MySQL: 5.5.5-10.6.16-MariaDB-0ubuntu0.22.04.1 [2024-03-10 04:57:56]
    Graphics Library: External ImageMagick 6.9.11-60
    Cache size N/A   never calculated Refresh

Activated plugin list 0

    No plugin activated

Piwigo URL: http://38.102.87.173/

Offline

 

#2 2024-03-10 18:33:13

windracer
Member
St. Pete, FL
2014-12-28
489

Re: Unable to sort FTP uploaded albums

On the Album Manager page, if you should be able to drag-and drop the folders into the sort order you want. I only use physical folders (i.e., FTP sync) and can do this. There's also the "Automatic Sort Order" which can be used to apply a sort order if you want.

Offline

 

#3 2024-03-10 19:09:27

macstrat
Member
2016-05-10
13

Re: Unable to sort FTP uploaded albums

windracer wrote:

On the Album Manager page, if you should be able to drag-and drop the folders into the sort order you want.

Normally yes, I SHOULD  be able to do this but cannot. This is what I'm trying to figure out.

Images are here: https://imgur.com/a/hvnvYOu

So in the first photo, you see that I have the folder "FFP_Chocolate" in the galleries folder. (The permissions are set to macstrat:www-data. I have tried setting the folder and its contents to www-data:www-data also with no success)

In the second photo, you can see that it was able to synchronize, and we can see the "FFP_Chocolate" album and its images in the album manager. There is also an album named "Test".

I cannot move "FFP_Chocolate" into "Test".

In the third photo, you can see files from "FFP Whiskey" uploaded via the java uploader. They upload to the uploads folder, and the album "FFP Whiskey" shows up in the Album Manager.

The fourth image shows that "FFP Whiskey" CAN be moved into the "Test" Album.

In the fifth image, "FFP_Chocolate" appears like it can be moved, but doesn't ACTUALLY move. I cannot move FFP_Chocolate out of the root of the hierarchy.

Images are here: https://imgur.com/a/hvnvYOu

Offline

 

#4 2024-03-10 20:21:39

erAck
Only trying to help
2015-09-06
2128

Re: Unable to sort FTP uploaded albums

Figure out what user account creates directories and files (i.e. for ftp it seems to be macstrat and not the web server's www-data account) and the corresponding file system permissions. If a chown from macstrat to www-data did not help but the Java upload as www-data owner works, then likely the ftp owner's permissions are not sufficiently set. You did not mention what they are in both cases. The image (you linked twice the same but speak of four images, there aren't) shows that there the permissions for FFP_Chocolate are drwxrwsr-x (so with the group sticky bit set) but it's not clear what the owner and group even are or for which case the image is.


Running Piwigo at https://erack.net/gallery/

Offline

 

#5 2024-03-10 22:59:48

macstrat
Member
2016-05-10
13

Re: Unable to sort FTP uploaded albums

erAck wrote:

Figure out what user account creates directories and files (i.e. for ftp it seems to be macstrat and not the web server's www-data account) and the corresponding file system permissions. If a chown from macstrat to www-data did not help but the Java upload as www-data owner works, then likely the ftp owner's permissions are not sufficiently set. You did not mention what they are in both cases. The image (you linked twice the same but speak of four images, there aren't) shows that there the permissions for FFP_Chocolate are drwxrwsr-x (so with the group sticky bit set) but it's not clear what the owner and group even are or for which case the image is.

so the owner of FFP_Chocolate and one of the files is:
drwxrwsr-x 2 www-data www-data 4096 Mar 10 02:06 FFP_Chocolate
-rwxrwxr-x 1 www-data www-data  788118 Mar 10 02:06 FFP_Chocolate_0047.jpeg

and the FFP Whiskey files are:
-rw-r--r-- 1 www-data www-data  214834 Mar 10 17:47 20240310174720-5b1dd23e.jpg

all files and folders are owned by www-data. macstrat is a member of the sudo and www-data groups:

macstrat@ubuntu:/$ groups macstrat
macstrat : macstrat sudo www-data

As a test, I created a new user just for uploading and put it into the www-daa group and nothing else. I uploadied a folder called "Arabian_Nights_Baghdad" and when I went to sync the files I get:

Warning: opendir(./galleries/Arabian_Nights_Baghdad): Failed to open directory: Permission denied in /var/www/html/admin/include/functions.php on line 608

Warning: opendir(./galleries/Arabian_Nights_Baghdad): Failed to open directory: Permission denied in /var/www/html/admin/site_reader_local.php on line 70

the piwigo directories for _data, galleries, and upload are symlinked to folders in teh /data directory
the symlinks are owned by www-data also:

macstrat@ubuntu:/$ ls -l /var/www/html/galleries
lrwxrwxrwx 1 www-data www-data 23 Mar 10 01:21 /var/www/html/galleries -> /data/piwigo_galleries/



erAck wrote:

The image (you linked twice the same but speak of four images, there aren't)

There are 5 images in that album. originally the link was at the end but i moved it to the top adn forgot to delete it off the end of the post.

Offline

 

#6 2024-03-11 01:51:28

erAck
Only trying to help
2015-09-06
2128

Re: Unable to sort FTP uploaded albums

macstrat wrote:

erAck wrote:

The image (you linked twice the same but speak of four images, there aren't)

There are 5 images in that album. originally the link was at the end but i moved it to the top adn forgot to delete it off the end of the post.

Darn, I didn't even notice there were several images under that same URL, sorry. I usually never scroll down on imgur because it's all shite.

Warning: opendir(./galleries/Arabian_Nights_Baghdad): Failed to open directory: Permission denied in /var/www/html/admin/include/functions.php on line 608

the piwigo directories for _data, galleries, and upload are symlinked to folders in teh /data directory
the symlinks are owned by www-data also:

macstrat@ubuntu:/$ ls -l /var/www/html/galleries
lrwxrwxrwx 1 www-data www-data 23 Mar 10 01:21 /var/www/html/galleries -> /data/piwigo_galleries/

And what are the directory permissions for /data/piwigo_galleries/ ?

Also make sure that you have

<Directory /var/www/html/>
    Options FollowSymLinks
</Directory>

in your httpd.conf or have an AllowOverride for it to be able to change options per directory .htaccess, i.e. in /var/www/html/.htaccess


Running Piwigo at https://erack.net/gallery/

Offline

 

#7 2024-03-11 05:09:35

macstrat
Member
2016-05-10
13

Re: Unable to sort FTP uploaded albums

erAck wrote:

And what are the directory permissions for /data/piwigo_galleries/ ?

so the directory permissions for /data/piwigo_galleries are:
drwxrwxr-x 4 www-data www-data  4096 Mar 10 21:52 piwigo_galleries

erAck wrote:

Also make sure that you have

<Directory /var/www/html/>
    Options FollowSymLinks
</Directory>

in your httpd.conf or have an AllowOverride for it to be able to change options per directory .htaccess, i.e. in /var/www/html/.htaccess

I do not have a .htaccess file so I created one and added the lines to it, restarted the server, no success
I added the lines of code to the apache2.conf as a new set with the other <directory> listings, and got a 500 error. removed the .htaccess file, restarted the server, and the error went away, but did not fix the issue.
I also tried adding a full string:
<Directory /var/www/html>
        Options Indexes FollowSymLinks
        AllowOverride All
        Require all granted
</Directory>

restarted the server, and still have the same issue.

Offline

 

#8 2024-03-11 13:05:18

erAck
Only trying to help
2015-09-06
2128

Re: Unable to sort FTP uploaded albums

Using AllowOverride in a .htaccess file won't work anyway, that has to be in the apache2.conf, you'd have to add it to the existing <Directory /var/www/> or <Directory /var/www/html/> whatever you have there as document root.

Anyhow, does it work if you have a directory /var/www/html/galleries/ and a symlink
/var/www/html/galleries/Arabian_Nights_Baghdad -> /data/piwigo_galleries/Arabian_Nights_Baghdad

I'm out of guesses.


Running Piwigo at https://erack.net/gallery/

Offline

 

#9 2024-03-11 15:37:16

macstrat
Member
2016-05-10
13

Re: Unable to sort FTP uploaded albums

EDIT: I am currently using piwigo import tree perl file as as substitute and its working, but it doesnt fix the core issue.

erAck wrote:

Anyhow, does it work if you have a directory /var/www/html/galleries/ and a symlink
/var/www/html/galleries/Arabian_Nights_Baghdad -> /data/piwigo_galleries/Arabian_Nights_Baghdad

the only reason I can't do that is because I have 2000 albums of 50 images each with 20+ albums being added every month. The first 2000 are sorted into alphabetical folders, but the new uploads don't have that luxury. You can see what I'm talking about at my current site here: https://cardscans.piwigo.com/. (These were all uploaded using the Java uploader so it's never been an issue. They did have to backdate my site so that the java uploader would continue to work)


erAck wrote:

I'm out of guesses.

So last night I set up a VM running Ubuntu server 22.04 LTS with 2 hard drives (50gb each), 4gb ram, and 4 vCPUs. Set up and mounted one of the drives to /mnt/piwigo

1. Installed ssh, vsftp, and a LAMP stack with apache2, mariadb, and php8.1. Also installed were php-zip, unzip, ImageMagick, and mbstring.

2. Created a new user that is in the sudo and www-data groups so I'm not logging in with root.

3. Set up the database and user for piwigo.

4. wget and unzipped piwigo into the /var/www/html folder. Version 14.3

5. Set up up symlinks for _data, galleries, and upload to corresponding folders in /mnt/piwigo and copied the files from the existing directories into these.

6. Set up permissions for the folders and symlinks for www-data, and chmod g+s on the folders so new files would inherent the www-data group properties.

7. Added the <directory> lines to apache2.conf for /var/www/html

8. Ran the piwigo setup and uploaded via the same techniques: one uploaded with the Java uploader (about 15 of the files didnt actually upload and errored out, this is an issue that has been brought up before and is a known issue with the java uploader currently), and one folder uploaded with ftp.

Ran into the same issue where the ftp folder would not move but the java uploaded files will. So unless im doing something wrong during the initial setup, it is a reproducible issue.

I put an issue in on the git repository and will update that issue with a link to this thread.

Last edited by macstrat (2024-03-11 21:23:46)

Offline

 

#10 2024-03-12 10:52:18

gueba
Member
2021-12-31
57

Re: Unable to sort FTP uploaded albums

Now I hope I understand:
you want to nest a physical album into a virtual album?!
This is not possible, AFAIK (due to the file structure). You can link photos from a physical album to a virtual, using the batch manager OR you can convert physical albums to virtual ones using an extension like this:
https://piwigo.org/ext/extension_view.php?eid=878

hth

Offline

 

#11 2024-03-12 16:39:33

macstrat
Member
2016-05-10
13

Re: Unable to sort FTP uploaded albums

gueba wrote:

Now I hope I understand:
you want to nest a physical album into a virtual album?!
This is not possible, AFAIK (due to the file structure). You can link photos from a physical album to a virtual, using the batch manager OR you can convert physical albums to virtual ones using an extension like this:
https://piwigo.org/ext/extension_view.php?eid=878

hth

I was experimenting with this concept and it raises a question of futility (I didn't know it separated virtual from physical albums so thank you for that).  Assuming that:

1. If I upload via FTP, I have to use naming conventions that differ from that of the uploader's host OS (i.e uploading from Windows to Linux and not being able to use spaces in the file name)
2. If I use the Virtualize plugin it shifts the files from /galleries/ to /uploads/ and changes the file name in the process.
3. It places them within the file structure of the uploads folder (YYYY>MM>DD)

It seems like an exercise in futility to make sure that all my files are named properly for upload, then have the name and folder structure changed for the sake of creating a virtual album when I can use the Perl script to send the files straight to the uploads folder without having to change the names on the uploaders host OS, creates the folder structure (YYYY>MM>DD), and creates the virtual albums automatically on upload?

If everything is going to wind up in the /uploads/ folder anyway, it seems like having a dedicated upload folder to place your files in and then running a .bat file that contains the Perl script command fixes all the above issues and removes the time that would be spent:
-Renaming the files for FTP upload
-Logging in
-Running the plugin or using the batch manager to link it to a virtual album
-SSH into the server to remove the now empty folders in the /galleries/ folder.

The Perl script makes the /galleries/ folder and plugin redundant for the sake of album organization unless you are using the /galleries/ folder as a backup.

Offline

 

#12 2024-03-13 00:53:12

erAck
Only trying to help
2015-09-06
2128

Re: Unable to sort FTP uploaded albums

macstrat wrote:

I was experimenting with this concept and it raises a question of futility (I didn't know it separated virtual from physical albums so thank you for that).

That's the concept. Virtual albums are fundamentally different from physical album folders. You can assign any image to any album and more than one album.

Assuming that:

1. If I upload via FTP, I have to use naming conventions that differ from that of the uploader's host OS (i.e uploading from Windows to Linux and not being able to use spaces in the file name)

You can extend the accepted set of characters. Please read the local/config/config.inc.php documentation for $conf['sync_chars_regex'], and for example to add a space to the default add this to your local config

Code:

$conf['sync_chars_regex'] = '/^[a-zA-Z0-9-_. ]+$/';

2. If I use the Virtualize plugin it shifts the files from /galleries/ to /uploads/ and changes the file name in the process.
3. It places them within the file structure of the uploads folder (YYYY>MM>DD)

Yes.


Running Piwigo at https://erack.net/gallery/

Offline

 

#13 2024-03-13 02:45:20

macstrat
Member
2016-05-10
13

Re: Unable to sort FTP uploaded albums

erAck wrote:

macstrat wrote:

I was experimenting with this concept and it raises a question of futility (I didn't know it separated virtual from physical albums so thank you for that).

That's the concept. Virtual albums are fundamentally different from physical album folders. You can assign any image to any album and more than one album.

Im a little confused then. So whats the benefit of physical albums if all the advantages go to virtual albums when it sounds like all it does is lock the file/folder in place and remove the versatility of dynamic organization?

Offline

 

#14 2024-03-13 10:53:29

erAck
Only trying to help
2015-09-06
2128

Re: Unable to sort FTP uploaded albums

Because many people already have their photos organized in folders before they start with Piwigo? Also, bulk uploading to a server or doing it scripted is easier via sftp/scp/rsync/whatever.


Running Piwigo at https://erack.net/gallery/

Offline

 

#15 2024-03-13 11:05:27

gueba
Member
2021-12-31
57

Re: Unable to sort FTP uploaded albums

... So whats the benefit of physical albums ...

You're joking?

Did you ever hear of a clean logical folder-file-structure and proper file-naming?

Offline

 

Board footer

Powered by FluxBB

github twitter newsletter Donate Piwigo.org © 2002-2024 · Contact