Piwigo.org

yoDan · 2009-06-16 23:32:58

Hi,

I used pLoader to upload photos to my gallery. Pictures are best ordered by time of creation or simply by filename (0123.jpg format). However, they remain ordered by, if I recall correctly, the order I added them by drag'n'drop in pLoader. Even if I explicitly select "order by filename" in the drop box, photos are still in that "arbitrary" order.

Is it a UI bug or a backend bug? ;-)

(the gallery contains family photos, that's why I don't give its address right away. But if you need it to diagnose the problem, sure, I will share the address)

Best,

plg · 2009-06-17 00:43:23

Hi yoDan :-)

Pictures are best ordered by time of creation or simply by filename (0123.jpg format). However, they remain ordered by, if I recall correctly, the order I added them by drag'n'drop in pLoader. Even if I explicitly select "order by filename" in the drop box, photos are still in that "arbitrary" order.

When using pLoader, the filename is automatically changed to something like 20090617002205-abc123.jpg (where 20090617002205 is the timestamp of posted date and abc123 are the 6 first characters of the original file md5sum) so you can't really trust the filename order. First conclusion: with photos added by pLoader, sorting by filename is a duplicate of sorting by posted date.

Sorting by time of creation won't work because it only takes the date into account, not the time. All you photos probably have the same date within a single category. That won't work :-/ [Bugtracker] ticket 561 (I really think this bug should be fixed quite soon, I mean that I should stop writing in the forum and open my code editor...)

Now, the solution (or the "workaround" if you prefer). In Piwigo 2.0 we have added manual rank on photos within the same category. When browsing a category from the gallery side, logged on as an administrator, you see a "edit category informations" icon. Click on it and you'll be thrown into the administration page for this category. In this administration page, you'll see a "manage image ranks" icon. Click on it. Here comes the new screen. Here you can define order of your photos, the way you want. Just change the numeric value for each of your photos and validate.

Conclusion for me:

1. we need another sort option "by name" (which is not the same as "by filename")
2. [Bugtracker] ticket 561 must be fixed
3. manual ranks management must be better documented (and UI improved with drag'n drop sorting for example)

VDigital · 2009-06-17 08:19:13

Don't use a workaround just for dates.

If you already use a config_local.inc.php and if you coded:

$conf['order_by'] = .....;

Don't forget to add behind (or adapt it to your preferences):

$conf['order_by_inside_category'] = $conf['order_by'];

Because this one (..._inside_category) still contains the previous value of $conf['order_by'] from the default conf.
(This point has been explained several times in our French forum).

Enjoy!

VDigital · 2009-06-17 10:11:47

My previous answer looks a bit out of the subject, I wrote it quickly this morning but it is often the problem in case of ignored sort order.

You're both right, on pLoader side.

I need to check which recent options we have for picture name within pLoader
Name can be used in $conf['order_by'] or $conf['order_by_inside_category']
So, instead of the default:

$conf['order_by'] = ' ORDER BY date_available DESC, file ASC, id ASC';

// order_by_inside_category : inside a category, images can also be ordered
// by rank. A manually defined rank on each image for the category.
//
// In addition to fields of #images table, you can use the
// #image_category.rank column

// $conf['order_by_inside_category'] = ' ORDER BY rank';
// will sort images by the manually defined rank of images in this category.
$conf['order_by_inside_category'] = $conf['order_by'];

Currently you should have:

$conf['order_by'] = ' ORDER BY file ASC, id ASC';

Or something like in your config_local.inc.php

Another workaround could be:

$conf['order_by'] = ' ORDER BY date_creation DESC, name DESC';

But again don't forget to add:

$conf['order_by_inside_category'] = $conf['order_by'];

Note for beginners: date_creation doesn't include time for technical reasons, so we need the real pic name which is always ordered except if you have several cameras.

yoDan · 2009-06-17 23:27:43

Hi,

Thanks plg for the thorough explanation and workarounds. In my case, manually reordering the pictures is not really an option (too many of them), and I've just noticed that pLoader uploaded rather low-res versions of the photos, so I will just resend them. (yeah, I know, I should have checked the settings first! :o)

In the mean time, I'd like to comment on your conclusions. Yes, including time in sort-by-creation-date would be a nice feature (I trust it's not as easy as it sounds). Also, sorting by name would be a welcome addition.

But I would like to stress that the behaviour of "sort by filename" would still be quite confusing for all visitors. My initial file is called 0123.jpg, once in the gallery the label below the thumbnail is 0123, and yet its full path on the server is something like 20090614120139-93a74128.jpg. Just curious, why did you use that scheme? Wouldn't ${orig_filename}.${something_unique} (for instance, 0123.jpg.000) be enough, and preserve filename order?

Cheers,
PS: VDigital, sorry, no direct editing of PHP files. I'm too old for that! ;-P

VDigital · 2009-06-18 07:38:03

Evidence it is! Thxs to yoDan.

Kind regards,
PS: Don't forget I am much older than you... ;-)

plg · 2009-06-19 01:42:11

Hi yoDan,

yoDan wrote:
In my case, manually reordering the pictures is not really an option (too many of them)

I understand :-/ see [Bugtracker] ticket 940 I don't think you have so many clics to do on your photo set even with the current reordering form.

yoDan wrote:
, and I've just noticed that pLoader uploaded rather low-res versions of the photos, so I will just resend them. (yeah, I know, I should have checked the settings first! :o)

By default, pLoader doesn't send the high-res photos, only the thumbnail+"web resized" photos. This makes upload much much faster (uploading 6,000KB is 14 times slower than uploading 430KB, which are the D700 average file size VS pLoader average filesize on your photos).

You can't resend them because pLoader has a nice feature which says "hey Piwigo, do you already have this photo?" based on your photo md5sum. If you send your photo once again, it will only update photo properties. Another feature request would be to force thumbnail/web resized/high resolution re-upload). So if you really want to re-upload, start by removing all photos:

1. at the root of your gallery, activate the flat mode (icon with tooltip "display all elements in all sub-categories")
2. add all photos in your caddie
3. in the administration>pictures>caddie>Deletions>target "all" + confirm + deleted selected photos

I've never checked removing more than 10 photos at once, you will tell us how good it behaves :-)

yoDan wrote:
In the mean time, I'd like to comment on your conclusions. Yes, including time in sort-by-creation-date would be a nice feature (I trust it's not as easy as it sounds). Also, sorting by name would be a welcome addition.

it's not as easy as is sounds but it's not impossible :-)

yoDan wrote:
But I would like to stress that the behaviour of "sort by filename" would still be quite confusing for all visitors. My initial file is called 0123.jpg, once in the gallery the label below the thumbnail is 0123, and yet its full path on the server is something like 20090614120139-93a74128.jpg. Just curious, why did you use that scheme?

There are several excellent reasons for this :-)

1. security, if your file is called 0123.jpg, whatever the permissions you set, anyone can directly browse to 0124.jpg and display your very private photo
2. simplicity, we don't care about forbidden characters in filesystem and http
3. concurrency, 2 admins can upload "my_sleeping_kitten.jpg" at the same time

While writing these reasons, I realize that we could partially solve the problems with :

2009 / 06 / 19 / 93a74128 / 0123.jpg
2009 / 06 / 19 / cacf6356 / 0124.jpg

Issues 1 and 3 can be fixed this way. So my real problem is 2. What if:

2009 / 06 / 19 / 93a74128 / Описание.jpg
2009 / 06 / 19 / cacf6356 / 说明在中国.jpg

yoDan · 2009-06-19 18:00:13

By default, pLoader doesn't send the high-res photos, only the thumbnail+"web resized" photos. This makes upload much much faster

I definitely agree with that behaviour. It was just my mistake.

You can't resend them because pLoader has a nice feature which says "hey Piwigo, do you already have this photo?" based on your photo md5sum.

That's great. So, presumably, if I modify some pictures, but not all, I can ask pLoader to re-upload them all, and it will actually upload only the modified picts? :)
Maybe you just need to make the question more precise "hey Piwigo, do you already have this photo's thumbnail/web size/hires size?" That's certainly not critical anyway.

What if
2009 / 06 / 19 / 93a74128 / Описание.jpg
2009 / 06 / 19 / cacf6356 / 说明在中国.jpg

Some serious testing with major hosting sites is needed here, but could it just work as is (just avoid \, / and newlines in filenames, basically)? Am I overly optimistic?

plg · 2009-06-19 23:25:38

yoDan wrote:
So, presumably, if I modify some pictures, but not all, I can ask pLoader to re-upload them all, and it will actually upload only the modified picts? :)

On your computer, if you modify 0123.jpg, it will have another md5sum. So Piwigo will answer "I don't know you yet". The existence check only uses the md5sum, not the filename. The "new" 0123.jpg will be considered as a new photo and fully uploaded. The "old" 0123.jpg will stay on Piwigo side.

Maybe you just need to make the question more precise "hey Piwigo, do you already have this photo's thumbnail/web size/hires size?" That's certainly not critical anyway.

OK, interesting improvement. First Piwigo identifies the photo, then Piwigo compares the md5sum of each file (3 files come with a md5sum to check that upload was successful). If the md5sum differs, the file is overwritten. I'll ask ron to come and give his opinion.

What if
2009 / 06 / 19 / 93a74128 / Описание.jpg
2009 / 06 / 19 / cacf6356 / 说明在中国.jpg
Some serious testing with major hosting sites is needed here, but could it just work as is (just avoid \, / and newlines in filenames, basically)? Am I overly optimistic?

I suppose it may work with a clean filesystem. I also think about pLoader on Windows/Piwigo on Linux. I admit it sounds to me like "watch out, you gonna have many problems if you list forbidden characters instead of authorized characters". It may not be a very common situation but imagin you have pLoader on a Linux box and 2 filenames : abc.jpg and ABC.jpg. On windows side, it's a problem because filenames are case insensitive (hum, that would not be a real problem if the md5sum is used as parent directory).

The question on filename constraints is sometimes asked, for example, see [Bugtracker] ticket 414 and [Forum, topic 13423] Synchronizing with large storage area - works now, a few hints?

Another related topic is [Bugtracker] ticket 967

ron · 2009-06-21 22:13:27

plg wrote:
OK, interesting improvement. First Piwigo identifies the photo, then Piwigo compares the md5sum of each file (3 files come with a md5sum to check that upload was successful). If the md5sum differs, the file is overwritten. I'll ask ron to come and give his opinion.

Current API pwg.images.exist uses the original photo md5 checksum. I agree that it should also take thumbnail and resized md5 checksum as parameters. It should return which md5 is different in order to tell to pLoader which file has to be sent.

plg · 2009-07-14 22:45:18

* [Bugtracker] ticket 1051
* [Bugtracker] ticket 1052

VDigital · 2009-07-15 08:50:50

ron wrote:
plg wrote:
OK, interesting improvement. First Piwigo identifies the photo, then Piwigo compares the md5sum of each file (3 files come with a md5sum to check that upload was successful). If the md5sum differs, the file is overwritten. I'll ask ron to come and give his opinion.
Current API pwg.images.exist uses the original photo md5 checksum. I agree that it should also take thumbnail and resized md5 checksum as parameters. It should return which md5 is different in order to tell to pLoader which file has to be sent.

ron,

I am not sure what your intentions are but look at this case.

You and your "brother" are sharing a gallery.
Both send pictures by pLoader.
Your brother sent his DSC_12345.jpg
What will append the day you send your DSC_12345.jpg?

Override is an option (Possible or Never).
If possible, the controls should it be based on filename, createdDateTime, and md5checksum of each individual file (thumbnail, resized, HD).

plg · 2009-07-15 09:06:50

VDigital wrote:
You and your "brother" are sharing a gallery.
Both send pictures by pLoader.
Your brother sent his DSC_12345.jpg
What will append the day you send your DSC_12345.jpg?

The unicity of the photo is verified with the md5sum of the original photo. The original filename is absolutely not a constraint.

VDigital · 2009-07-15 09:13:51

plg wrote:
VDigital wrote:
You and your "brother" are sharing a gallery.
Both send pictures by pLoader.
Your brother sent his DSC_12345.jpg
What will append the day you send your DSC_12345.jpg?
The unicity of the photo is verified with the md5sum of the original photo. The original filename is absolutely not a constraint.

I know.

plg · 2009-07-15 09:19:37

Anyway, I agree that the overwrite should be an option in some way (the problem with options, is that it makes the whole application complicated).

Using the original filename as a key for uniqueness sounds dangerous to me.

Piwigo.org

Announcement

#1 2009-06-16 23:32:58

Ignored sort order

#2 2009-06-17 00:43:23

Re: Ignored sort order

#3 2009-06-17 08:19:13

Re: Ignored sort order

#4 2009-06-17 10:11:47

Re: Ignored sort order

#5 2009-06-17 23:27:43

Re: Ignored sort order

#6 2009-06-18 07:38:03

Re: Ignored sort order

#7 2009-06-19 01:42:11

Re: Ignored sort order

yoDan wrote:

yoDan wrote:

yoDan wrote:

yoDan wrote:

#8 2009-06-19 18:00:13

Re: Ignored sort order

#9 2009-06-19 23:25:38

Re: Ignored sort order

yoDan wrote:

#10 2009-06-21 22:13:27

Re: Ignored sort order

plg wrote:

#11 2009-07-14 22:45:18

Re: Ignored sort order

#12 2009-07-15 08:50:50

Re: Ignored sort order

ron wrote:

plg wrote:

#13 2009-07-15 09:06:50

Re: Ignored sort order

VDigital wrote:

#14 2009-07-15 09:13:51

Re: Ignored sort order

plg wrote:

VDigital wrote:

#15 2009-07-15 09:19:37

Re: Ignored sort order

Board footer