Announcement

  •  » Engine
  •  » Custom Derivatives-missing image info in DB to facilitate rebuild?

#1 2017-08-21 00:00:15

CalgarySteveWilliams
Member
2017-08-19
7

Custom Derivatives-missing image info in DB to facilitate rebuild?

Hi,

I am new to piwigo (2.9.1) and am impressed with it.  However, I noticed there is no built in ability to build custom thumbnails such as those used by my chosen theme Bootstrap Darkroom which uses 260x180 thumbnail.

I was investigating if it would be possible integrate this functionality into the Batch Manager.

The first stumbling block I came across is that the dimensions of the custom size aren't stored in the database when a new custom size is created.  I feel like this is likely a bug/oversight, but don't really know.

Details
-------
Looking at the database:
select * from piwigo_config where param = 'derivatives';

For built in sizes, I can see information such as:
"small";O:16:"DerivativeParams":3:{s:13:"last_mod_time";i:1501993489;s:6:"sizing";O:12:"SizingParams":3:{s:10:"ideal_size";a:2:{i:0;i:576;i:1;i:432;}s:8:"max_crop";i:0;s:8:"min_size";N;}s:7:"sharpen";i:0;}s:6:

For the "custom" sizes, there is only information such as:
"c";a:10:{
    s:8:"e260x180";i:1503113392;
    s:9:"e1008x756";i:1502782626;
    ...

This means if I have a custom size such as e260x180, I cannot get any information after the fact to try to build any additional thumbnails!

Looking into:
include/derivative_std_params.inc.php -> ImageStdParameters::get_custom()

There is only code to store the timestamp:
    if ( @self::$custom[$key] < time() - 24*3600)
    {
      self::$custom[$key] = time();
      self::save();
    }

I modified the code so that it actually stores the entire custom derivative when the custom size is generated:
  static function get_custom($w, $h, $crop=0, $minw=null, $minh=null)
  {
    $params = new DerivativeParams( new SizingParams( array($w,$h), $crop, array($minw,$minh)) );
    self::apply_global($params);

    $params->last_mod_time = time();

    $key = array();
    $params->add_url_tokens($key);
    $key = implode('_',$key);

    // For now, unconditionally save
    // Development only.  Implement original logic in production.
    self::$custom[$key] = $params;
    self::save();

    return $params;
  }

This results in a much more useful database entry:
"c";a:1:{
    s:8:"e260x180";O:16:"DerivativeParams":3:{s:13:"last_mod_time";i:1503255886;s:6:"sizing";O:12:"SizingParams":3:{s:10:"ideal_size";a:2:{i:0;i:260;i:1;i:180;}s:8:"max_crop";i:1;s:8:"min_size";a:2:{i:0;i:260;i:1;i:180;}}s:7:"sharpen";i:0;}    }

Then I was able to create a simple method:
  /**
   * @param string $type
   * @return DerivativeParams
   */
  static function get_custom_by_type($type)
  {
    if (array_key_exists($type, self::$custom))
    {
      return self::$custom[$type];
    }
    return null;
  }

This way I can pass in e260x180 and get the DerivativeParams object which can be used to make a DerivativeItem.  For example:
    $derivative = new DerivativeImage(ImageStdParams::get_custom_by_type($type), $src_image);


My thoughts were that this could (almost) be incorporated into the existing "Generate multiple size images".  There could be a checkboxes for all the built in sizes, then another section of checkboxes for the custom sizes.  Really, the treatment is the same for the built in sizes and custom sizes, so the logic in ws_getMissingDerivatives is almost identical for both.

Alternatively, use a plugin to extend the "Batch Manager" Action list to generate them.  Using a plugin approach presents an issue with all the duplication of code to achieve the same functionality, and lack of ability to use ws.php (ws_getMissingDerivatives)

Initially, I was trying to do this in a plugin (without modifying the main code base). 


If I keep going down this road, is there any chance (assuming appropriate code review/approval, etc) of getting the changes to the main code base?

Is there any guidance on the "best" way to proceed with this?

Thanks,
Steve Williams

Offline

 

#2 2017-08-21 00:27:55

flop25
Piwigo Team
2006-07-06
7037

Re: Custom Derivatives-missing image info in DB to facilitate rebuild?

Hello
For me since I have a theme heavily using custom sizes I'm very interested
Let's wait the opinion of  Rvelices


To get a better help : Politeness like Hello-A link-Your past actions precisely described
Check my extensions : more than 30 available
who I am and what I do : http://fr.gravatar.com/flop25
My gallery : an illustration of how to integrate Piwigo in your website

Offline

 

#3 2017-08-21 07:06:57

CalgarySteveWilliams
Member
2017-08-19
7

Re: Custom Derivatives-missing image info in DB to facilitate rebuild?

Hi,

Doing a little further investigation, I figured out how "i.php" works with the custom derivatives.  It in fact does reverse engineer the parameters from the (for example) e260x180.

If the custom parameters were stored in the database, this whole routine could be reduced to one line of code and a few supporting routines could be removed.

Now I look further, the scope of this change is a bit larger than I anticipated.

i.php:

function parse_custom_params($tokens)
{
  if (count($tokens)<1)
    ierror('Empty array while parsing Sizing', 400);

  $crop = 0;
  $min_size = null;

  $token = array_shift($tokens);
  if ($token[0]=='s')
  {
    $size = url_to_size( substr($token,1) );
  }
  elseif ($token[0]=='e')
  {
    $crop = 1;
    $size = $min_size = url_to_size( substr($token,1) );
  }
  else
  {
    $size = url_to_size( $token );
    if (count($tokens)<2)
      ierror('Sizing arr', 400);

    $token = array_shift($tokens);
    $crop = char_to_fraction($token);

    $token = array_shift($tokens);
    $min_size = url_to_size( $token );
  }
  return new DerivativeParams( new SizingParams($size, $crop, $min_size) );
}

Figuring out how i.php works enabled me to make a shell script that will regenerate my custom images :)   I'm not saying I don't want to make the changes to piwigo, but for the moment, this will speed up my website which is running on a PCEngines alix low power board (12 watts?).

I've included my shell script in case it helps anyone else.  This is running on a Unix (OpenBSD) box with shell access to the gallery.

Save the script as "custom" in the gallery folder.  Tweak as necessary.
-------------------------------------
!/bin/sh

# Note the / on the end
where="/var/www/piwigo/galleries/"

# The path and name of this script.  For now, it's just in the gallery folder
custom=${where}custom

# Space separated list of sizes to regenerate
sizes="e260x180"

if [ ! -d $where ]; then
  echo "Unable to locate $where"
  exit 1
fi

cd $where
if [ $? -ne 0 ]; then
  echo "Unable to change directory to $where"
  exit 1
fi

# If no folder specified, then do something cool/nasty and fork a process
# for each folder in the main level of gallery.  This will HAMMER the system, but in
# the interest of expediency... :)
#
# On systems with many root folders, might run out of processes trying this.
# Should do something smarter, but it works for me :)

# If no parameters, for a process for each folder
if [ $# -eq 0 ]; then
  for dir in *; do
    if [ -d "$dir" ]; then
      echo "Forking to process $dir"
      echo "Executing ${where}custom $dir"
      ${where}custom "$dir" > /tmp/custom_out.$dir 2>&1 &
    else
      echo "Skipping non-directory $dir"
    fi
  done
  # Done forking, just exit
  exit 0
fi

# Just to make sure
if [ ! -d "$1" ]; then
  echo "Unable to locate $1"
  exit 1
fi

# Temporary working directory
mkdir /tmp/custom.$$

find "$1" \( ! -name '*.htm' -a ! -name '*.php' -a ! -name '*.html' \) -type f -print | while read file; do
  echo "Processing $file"
  dir=`dirname "$file"`
  f=`basename "$file" | awk -F. '{ print substr($0,1,index($0,".")-1)}'`
  ext=`basename "$file" | awk -F. '{ print substr($0,index($0,".")+1)}'`
  for size in $sizes; do
    cd /tmp/custom.$$
    if [ $? -ne 0 ]; then
      echo "Unable to change directory to /tmp/custom.$$"
      exit 1
    fi

    # Deal with spaces in the URL
    # TODO - This isn't working :( - but piwigo doesn't show galleries/photos with spaces,
    # so not a big deal right now.  For me, legacy of importing from gallery3
    url="https://piwigo.williamsitconsulting.com/i.php?/galleries/${dir}/${f}-cu_${size}.${ext}"
    url=`echo "$url" | sed 's/ /\&nbsp;/g'`
    echo $url
    ftp $url
    rm -f ${f}-cu_${size}.${ext}
  done
done

cd $where
rm -f /tmp/custom.$$
exit 0

Offline

 

#4 2017-08-21 07:21:02

CalgarySteveWilliams
Member
2017-08-19
7

Re: Custom Derivatives-missing image info in DB to facilitate rebuild?

Thinking about the shell script a bit more... it's kind of a cool solution!

With some tweaking, it could be used to run under cron and just update newly added files.

find /var/www/piwigo/galleries -mtime -5 -print...

Would find files added in the last 5 days and could be fed to the loop that regenerates the custom derivatives.

Hum...  too much idle time on my hands!  lol

Offline

 

#5 2018-12-28 22:59:50

skaldenhoven
Member
2018-12-28
2

Re: Custom Derivatives-missing image info in DB to facilitate rebuild?

Hi CalgarySteveWilliams,

I found your post when also trying to get the customderivatices script from Bootstrap Darkroom to work. (Also had the error "Trying to get property of non-object") After turning on curl errors to be logged got to know that it is caused by a malformed url. (Only cannot determine what would be wrong)
I also tried your shell script but didn't get it working right now.
Anyway did you make any progress on this matter, perhaps a plugin or something like that?
I also contacted the author of the original plugin and he directed me here since the custom derivatives is not part of the core of the theme.

Sander.

Offline

 
  •  » Engine
  •  » Custom Derivatives-missing image info in DB to facilitate rebuild?

Board footer

Powered by FluxBB

github twitter newsletter Donate Piwigo.org © 2002-2024 · Contact