Announcement

  •  » Engine
  •  » To coders ... to many unnecesary derivatives

#1 2017-08-13 14:41:22

eliz82
Member
Romania
2016-04-27
281

To coders ... to many unnecesary derivatives

I have observed that many extensions that use justified style or masonry style for showing the thumbnails, are building new custom size thumbnails (derivatives), without asking admin to chose an existing derivative or automatically checking if similar sized thumbnails exist already (builded by Piwigo core or other extensions).

Let me give an example:
1) Piwigo core => Admin will disable all derivatives (except square, thumbnail and medium) and will set Piwigo "thumbnails" size to be "540x216px / no crop mode" . This will create the file: IMG_37885-th.jpg with 216px height.
2) Gdthumb plugin => "216px" height is set in the plugin options. This will create the file: IMG_86154-cu_s9999x216.jpg with 216px height.
3) SmartPocket mobile theme => will automatically create custom derivatives with 216px height (no option to change this). This will create the file "IMG_86154-cu_648x216_z_0x216.jpg"

So 3 identical derivatives with the same image height (and width). Are you seeing the problem? Extensions should give the option to use the core derivatives if it's possible !

The list of extension I tested are:
-theme Smart Pocket (mobile theme) by P@t (or plg now ... no idea)
-plugin Gthumb+ by P@t (or plg now ... no idea)
-plugin Gdthumb by SergeD
... probably they are many others.

Many hosting companies are limiting user account not only by the used space but also by "Maximum inodes". The inodes are the number of "files+folders".
With such bad optimization of the piwigo extensions you can easily hit the number of inodes before you reach your space limit.

Last edited by eliz82 (2017-08-13 20:01:37)

Offline

 

#2 2017-08-14 08:58:31

eliz82
Member
Romania
2016-04-27
281

Re: To coders ... to many unnecesary derivatives

Actually I made more tests and it seems SmartPocket theme is more smarter then GDthumb or Gthumb+

If you set Piwigo thumbnails size to "648x216px" and "crop" option is disabled, SmartPocket will not generate another derivatives but use the core derivative instead.
Custom size "648x216_z_0x216" will appear in the Piwigo derivatives list, but if you check the /_data/i/... they are no generated files with this name. Instead if "filename-th.JPG" not exist, they will be generated when SmartPocked albums are first accessed.

I had a look at the code
/themes/smartpocket/themeconf.inc.php

Code:

class SPThumbPicker
{
  var $candidates;
  var $default;
  var $height;
  
  function init($height)
  {
    $this->candidates = array();
    foreach( ImageStdParams::get_defined_type_map() as $params)
    {
      if ($params->max_height() < $height || $params->sizing->max_crop)
        continue;
      if ($params->max_height() > 3*$height)
        break;
      $this->candidates[] = $params;
    }
    $this->default = ImageStdParams::get_custom($height*3, $height, 1, 0, $height );
    $this->height = $height;
  }
  
  function pick($src_image)
  {
    $ok = false;
    foreach($this->candidates as $candidate)
    {
      $deriv = new DerivativeImage($candidate, $src_image);
      $size = $deriv->get_size();
      if ($size[1]>=$row_height-2)
      {
        $ok = true;
        break;
      }
    }
    if (!$ok)
    {
      $deriv = new DerivativeImage($this->default, $src_image);
    }
    return $deriv;
  }
}

I'm not to good with php and piwigo but probably "ImageStdParams::get_defined_type_map()" build a array with existing sizes and loop "foreach($this->candidates as $candidate)" check if similar size exist.

then in "/themes/smartpocket/template/thumbnails.tpl"

Code:

{$derivative=$thumb_picker->pick($thumbnail.src_image)}

"pick" function is used

Offline

 

#3 2017-08-14 10:02:50

homdax
Member
Sweden
2015-02-02
294

Re: To coders ... to many unnecesary derivatives

My host does not use inodes, but basically what you are saying is that Piwigo is, via some plugins you list, creating a number of unnecessary files and that better control of this is warranted since it could result in increased cost for the website owner, often without understanding why.

My /i/* directory has 2014 files, but the uploaded images in albums are only 405 so that is the GD functions creating thumbnails and image caching for presentation. Which would cost 1500 inodes more I guess?

I guess better control of the image sizes and how many files that create is warranted.

Offline

 

#4 2017-08-14 12:54:58

eliz82
Member
Romania
2016-04-27
281

Re: To coders ... to many unnecesary derivatives

From what I saw Piwigo 2.9.1 comes with all derivatives enabled by default. They are 9 derivatives the last one XXL has 1656 x 1242px by default.
That means if you upload a picture bigger then 1656x...px other 9 pictures will be build.
---
So if you upload 1000 pictures in your gallery => will have 9000 derivatives => 10000 image files in total.
If you have SmartPocket theme and GThumb+ plugin then you will have a total of 12000 image files for 1000 uploaded pictures.

Each derivative type is build on first access (so maximum number of files will accumulate in time). If you use a plugin like Autosize will probably build faster if the site have many visitors, because Autosize automatically checks user display resolution and return a picture size according to this.
---
The best you could optimize Piwigo if you use SmartPocket and GThumb+ is to go to Administration => Configuration => Option => Photo sizes => view details =>

1) disable all derivatives except square, thumbnail, medium (those cannot be disabled).
2) set a big size for medium (bigger then the resolution of the pictures you want to upload). for example if you upload pictures 1200x800px then set medium to 1400x...
3) set 648x216 for thumbnail (so smartpocket not generate new derivatives). you can use a smaller value in the option but you must manually edit "themes\smartpocket\template\thumbnails.tpl" and set the $row_height. for example 480x160 and set $row_height=160.

In this way only square, thumbnail and S9999x200px are build. So for 1000 pictures you will have 3000 derivatives => 4000 total files.
---
I think the hosts companies that make automatic backups often are putting limits of inodes.
If 2 users have the same space (6GB) but one has 1000 files and the other one has 100.000 files => when automatic backup are made (either copy or archive the files) => the second user will use more processor, more IO (disk usage), more power and it takes more time.

Last edited by eliz82 (2017-08-15 10:12:54)

Offline

 
  •  » Engine
  •  » To coders ... to many unnecesary derivatives

Board footer

Powered by FluxBB

github twitter newsletter Donate Piwigo.org © 2002-2024 · Contact