Announcement

  •  » Engine
  •  » Multiple-size

#1 2011-12-26 12:08:27

plg
Piwigo Team
Nantes, France, Europe
2002-04-05
13496

Multiple-size

At the beginning with version 1.0 in 2002, each photo was available in 2 sizes: thumbnail + web size. Then with version 1.4 in 2005, we introduced the high definition, each photo had 3 sizes. No change since then. The Multiple-size project will reset the way we manage image sizes.

About 1 or 2 years ago, I've talked with rvelices about this feature (during a "face to face" meeting). rvelices has been thinking about it and has implemented a first draft recently. We have discussed about this draft with P@t and grum (meeting in Paris last week) and asked rvelices to first distribute it as a plugin : http://piwigo.org/svn/extensions/derivatives (anybody can already try it on a Piwigo trunk right now) But this plugin is planned for going into the core, the plugin is just for "early tests".

What does Piwigo lose?

Remote sites. The Multiple-size feature implies Piwigo to generate the sizes from the original photo, this is not possible with remote sites. As far as I'm concerned, I think Remote sites was a great feature on version 1.2 in 2002 (when you had only 100MB of storage on your free hosting) but it became useless with years (the same free hosting now offers 10GB, that's enough for 2,000 photos of 5MB or 40,000 wallpaper sized photos)

How does it work? (without going to much into details)

* the user uploads the original photo, be it a 4,000 x 3,000 pixels or a smaller 1,600 x 1,200 pixels (if you want to increase transfer speed or reduce disk usage)

* Piwigo stores it in upload/2012/01/01/ (web upload or pLoader/Digikam/Lightroom) or galleries (FTP upload)

* when asked by a visitor, if the size is not available, Piwigo generates the file "on the fly" in a cache, located in local/i

* each photo can be displayed in several sizes: square, thumbnail, small, medium, large, extra large... Piwigo proposes a basic configuration and proposes to edit each size and deactivate sizes.

How does it improve Piwigo?

* the ability to show the biggest photo, depending on the screen size of your visitor

* separate the "original photos" directory (which needs to be backed up) with the "generated sizes" (which is "just a cache")

* more flexibility for themes: a theme can ask for specific size/ratio, the image will be generated "on the fly"

Possible issues:

* too much image processing at once. If a visitor opens for the first time a page with 200 thumbnails, it means 200 thumbnails to generate.

* higher disk usage if the cache is never purged or if many visitors ask the "extra large" size

* image processing becomes nearly mandatory, so it has to work on your hosting provider


After a quick review by me (plg) and deep testing by Grum, we said "GO" to rvelices for integration into Piwigo core for a wider test. [Bugtracker] ticket 2541

I doubt the feature will stay "as is" for Piwigo 2.4.0 in April, but it's a good start for 2.4.0beta1 next week. Based on reviews and problems encountered, we'll improve the way multiple-size works.

edit on December 27th 2011: we can't "add" sizes

Offline

 

#2 2011-12-26 13:25:19

flop25
Piwigo Team
2006-07-06
7036

Re: Multiple-size

Great news! I'm very excited about this !

as usual ^^ just a few question before using it :
"each photo can be displayed in several sizes: square, thumbnail, small, medium, large, extra large... Piwigo proposes a basic configuration and proposes to edit each size, remove sizes or add new sizes." => these official/legacy sizes are for what exactly : for the picture page, for the themes, to avoid hotlinking with otehr sizes... And the pictures will be generated on the fly too ?
Can an extension add its own sizes or generate tis own sizes ? or it will register its sizes to the new system ?

the "* too much image processing at once" issue is the biggest one : why not create a new action in the batch (and also during the upload) ?

ps : we might need a feature list to see what exactly and technically has to be done : I've seen the plugin. The base is here, but the interaction with extension/user has to be define

Last edited by flop25 (2011-12-27 13:53:33)


To get a better help : Politeness like Hello-A link-Your past actions precisely described
Check my extensions : more than 30 available
who I am and what I do : http://fr.gravatar.com/flop25
My gallery : an illustration of how to integrate Piwigo in your website

Offline

 

#3 2011-12-26 13:34:30

mol
Piwigo Team Germany
germany
2010-05-26
108

Re: Multiple-size

that sounds great.
i know a similar feature in the old, file structure orientated gallery script yappa-ng (projekt stopped at 2006), i used between 2004-2006 as main gallery, i loved it (also the user managment, much better than piwigo today). but no development and no modern feauture like tagging system, searching, ugly templates, and so on :-)

too much image processing at once. If a visitor opens for the first time a page with 200 thumbnails, it means 200 thumbnails to generate.

in yappa-ng there was a "album initialise" script for the admin, similar the generate thumbnail function in piwigo, to fill the "cache", with this important hint to the admin:

"It makes the user with the sizes from 1024x1024 little difference if the image will be directly in the consideration or will be converted in advance!
Also, I can report from experience that even after a long time to be considered by far only a few pictures in maximum resolution. Thus, if all images are pre-converted au suspected, then the only costs unnecessarily storage!"

(sorry for google translate)

Offline

 

#4 2011-12-26 13:42:02

flop25
Piwigo Team
2006-07-06
7036

Re: Multiple-size

also, that could be even bigger : we can hide the full path (id or hash request), for privacy issue, if someone want to sell picture etc


To get a better help : Politeness like Hello-A link-Your past actions precisely described
Check my extensions : more than 30 available
who I am and what I do : http://fr.gravatar.com/flop25
My gallery : an illustration of how to integrate Piwigo in your website

Offline

 

#5 2011-12-26 14:00:54

rvelices
Piwigo Team
2005-12-29
1960

Re: Multiple-size

plg wrote:

* each photo can be displayed in several sizes: square, thumbnail, small, medium, large, extra large... Piwigo proposes a basic configuration and proposes to edit each size, remove sizes or add new sizes.

The admin will be allowed to disable standard sizes, but he will not be able to create new ones. The main reason is that themes and plugins should be aware that the new size exists and there will be an ugly dependency between admin and themes/plugins

Offline

 

#6 2011-12-26 14:02:46

flop25
Piwigo Team
2006-07-06
7036

Re: Multiple-size

rvelices wrote:

plg wrote:

* each photo can be displayed in several sizes: square, thumbnail, small, medium, large, extra large... Piwigo proposes a basic configuration and proposes to edit each size, remove sizes or add new sizes.

The admin will be allowed to disable standard sizes, but he will not be able to create new ones. The main reason is that themes and plugins should be aware that the new size exists and there will be an ugly dependency between admin and themes/plugins

you mean like wordpress, only extension will be able to add sizes ?

Last edited by flop25 (2011-12-26 14:03:12)


To get a better help : Politeness like Hello-A link-Your past actions precisely described
Check my extensions : more than 30 available
who I am and what I do : http://fr.gravatar.com/flop25
My gallery : an illustration of how to integrate Piwigo in your website

Offline

 

#7 2011-12-26 14:04:26

rvelices
Piwigo Team
2005-12-29
1960

Re: Multiple-size

flop25 wrote:

these official/legacy sizes are for what exactly : for the picture page, for the themes, to avoid hotlinking with otehr sizes...

the picture page will give urls for every size - up to the theme to use the right one (the default theme will propose all of them and will remember the user choice for the session)

flop25 wrote:

Can an extension add its own sizes or generate tis own sizes ? or it will register its sizes to the new system ?

Yes. This is called a "custom derivative". The plugin / theme can generate what size they want ...

Offline

 

#8 2011-12-26 14:09:00

flop25
Piwigo Team
2006-07-06
7036

Re: Multiple-size

thx for all your work and your answers


To get a better help : Politeness like Hello-A link-Your past actions precisely described
Check my extensions : more than 30 available
who I am and what I do : http://fr.gravatar.com/flop25
My gallery : an illustration of how to integrate Piwigo in your website

Offline

 

#9 2011-12-27 11:09:20

plg
Piwigo Team
Nantes, France, Europe
2002-04-05
13496

Re: Multiple-size

rvelices wrote:

The admin will be allowed to disable standard sizes, but he will not be able to create new ones.

OK, I've edited my first post.

Offline

 

#10 2011-12-27 11:16:25

plg
Piwigo Team
Nantes, France, Europe
2002-04-05
13496

Re: Multiple-size

flop25 wrote:

[...] for the picture page, for the themes, to avoid hotlinking with otehr sizes...

This makes me think about an important point: hotlinking! If I hotlink a "web size" photo from my Piwigo 2.3 into a forum post, the hotlink must not be broken, and it must not show the original size instead (that would be very bad for the forum visitors and for the bandwidth of the gallery owner).

flop25 wrote:

And the pictures will be generated on the fly too ?

All sizes are generated on the fly if Piwigo doesn't file the expected file in its cache.

flop25 wrote:

the "* too much image processing at once" issue is the biggest one : why not create a new action in the batch (and also during the upload) ?

Yes, this is possible. I think it can be important for thumbnails, but regenerating all "extra large" would be meaningless and should be avoided.

Offline

 

#11 2011-12-27 11:21:48

plg
Piwigo Team
Nantes, France, Europe
2002-04-05
13496

Re: Multiple-size

mol wrote:

Also, I can report from experience that even after a long time to be considered by far only a few pictures in maximum resolution. Thus, if all images are pre-converted au suspected, then the only costs unnecessarily storage!"

That is interesting. Depending on...

1) what are the dimensions of the large and extra-large sizes
2) what is the algorithm to display sizes (automatic or on demand)
3) what is the screen resolution of your visitors

... there may be very few large and extra-large in the cache.

(On Piwigo.com, we will be able to generate some statistics)

Offline

 

#12 2011-12-27 17:55:20

mol
Piwigo Team Germany
germany
2010-05-26
108

Re: Multiple-size

plg wrote:

1) what are the dimensions of the large and extra-large sizes
2) what is the algorithm to display sizes (automatic or on demand)
3) what is the screen resolution of your visitors

... there may be very few large and extra-large in the cache.

(On Piwigo.com, we will be able to generate some statistics)

1) the default sizes for yappa are:  thumbnail, 50x50, 640x640, 1024x1024 and 1600x1600, on my gallery i have added "original size" (unconverted upload)
2) on demand, a radio button. ( but i would prefer a scaling like in the stripped theme or the autosize plugin)
3) in the year 2004/5? lower than today. 95% with 640 and 1024. :-)

Offline

 

#13 2011-12-28 14:16:27

rvelices
Piwigo Team
2005-12-29
1960

Re: Multiple-size

The remaining things to do (in order of importance from my point of view) are:

a. drop remote sites

b. rewrite metadata sync (don't look at high anymore) + ftp synchronisation

c. review upload (forms + ws)

d. write admin GUI + persistence for choosing sizes

e. center of interest (if a derivative might be cropped as square for example, the admin must be able to choose the part which he wants to keep)

f. rotation based on Exif orientation (see below)

g. migration of database + files (drop has_high, high_xxx, tn_ext, move files from pwg_high one dir above, move thumbs to pwg_representative for non images that do not have a representative, remove thumbnails)

h. nice to have

  - improve default themes (for example on index give to users the possibility to choose thumbs, small or mediums ...)

  - allow themes to ges custom derivatives very easy ...

  - sharpening, watermarks (text or images, for derivatives larger than some size or not ...), grayscale effects


Concerning f (rotation / orientation) we have two ways:

1. Rotate the original image based on exif orientation - the admin will be able to rotate afterwards if he wants

Advantage: One easy step, no db change required, simpler code in piwigo core
Disadvantage: We loose metadata if only gd is available

2. Save the rotation angle in the db from Exif during sync/upload process. Let the admin eventually manually overwrite it. Perform computation of resulting derivative sizes and rotate the derivative based on this value.

Advantage: Originals are not touched
Disadvantage: Complicates a lot the piwigo core (during meta sync, ask if the admin chose a manual rotation angle?, crop/sizing algo more cumbersome, center of interest need also rotation etc ...)

Any feedback on point f?

Offline

 

#14 2011-12-28 15:16:26

rub
Piwigo Team
Lille
2005-08-26
5019

Re: Multiple-size

plg wrote:

What does Piwigo lose?

Remote sites. The Multiple-size feature implies Piwigo to generate the sizes from the original photo, this is not possible with remote sites. As far as I'm concerned, I think Remote sites was a great feature on version 1.2 in 2002 (when you had only 100MB of storage on your free hosting) but it became useless with years (the same free hosting now offers 10GB, that's enough for 2,000 photos of 5MB or 40,000 wallpaper sized photos)

Ok, but original photos can be save on "remote site". IE, original photo can be save on a specific site with other url, cache will be on local.


plg wrote:

How does it work? (without going to much into details)

Original photos can be save on a local directory not accesible by public http?
Trigger/event must be add in order to apply effect on picture.

For example, photos must be saved on a private directory and trigger add watermark on picture.

Offline

 

#15 2011-12-28 15:30:42

rvelices
Piwigo Team
2005-12-29
1960

Re: Multiple-size

rub wrote:

Ok, but original photos can be save on "remote site". IE, original photo can be save on a specific site with other url, cache will be on local.

The problem is that piwigo needs access to the original photos in order to build resizes. This is going to be extremely slow.

rub wrote:

Original photos can be save on a local directory not accesible by public http?

Yes: you need to protect your directory with a .htaccess, make sure you have enabled_high=false for everyone and make sure your original photos are larger than the biggest resize

rub wrote:

Trigger/event must be add in order to apply effect on picture.
For example, photos must be saved on a private directory and trigger add watermark on picture.

No trigger because the php script from piwigo that generate the resizes does not connect to the database. But check my point h. above (sharpening, watermark, grayscale effects etc...)
Basically in the very first version there will be no such thing (it's the last point in the remaining to do list)

Offline

 
  •  » Engine
  •  » Multiple-size

Board footer

Powered by FluxBB

github twitter newsletter Donate Piwigo.org © 2002-2021 · Contact