Announcement

#31 2015-01-06 00:29:44

SnotRocket
Member
2014-08-15
63

Re: Raw File SUpport

So... this plugin has done something to confuse the rest of piwigo.. or at least some of my other plugins.

I can't process / use video files any more.  mpg, mp4, etc when I try to sync, or load an album containing a video file.

Notice: Undefined variable: conf in /var/www/2000transam.com/gallery/upload_plugins/ffmpeg.php on line 41 Notice: Undefined variable: second in /var/www/2000transam.com/gallery/upload_plugins/ffmpeg.php on line 43 Unable to handle upload of ./galleries/CTS-V/2011_CTS-V_Dyno.mpg

Offline

 

#32 2015-01-06 01:17:40

wsloand
Member
2014-11-17
31

Re: Raw File SUpport

SnotRocket, The first issues appear to be permission problems.  Can you set the permissions so that the web server group can write to the directory?

The reason that clicking the raw thumbnail worked is due to the dynamic resizing.  You will be able to have a thumbnail and small images created each time, but that will be very hard computationally on the server.  That also would most likely be fixed by setting permissions on the directory.

For the issue with upload_plugins/ffmpeg.php, I'd not tested ffmpeg yet.  The $conf problem is a bug with the code:

$conf['ffmpeg_dir']

should be changed to

default_conf('ffmpeg_dir', '')

the $second missing is an issue with the code.  When copying this code, I missed setting that variable.  Adding the following as a new line just below the prepare_directory line, should fix the problem:

$second = 1;

The errors probably went away because you were viewing a page without ffmpeg-related images.

Offline

 

#33 2015-01-06 01:25:55

SnotRocket
Member
2014-08-15
63

Re: Raw File SUpport

Thanks for the reply.

Here's what the patch did:

// move the uploaded file to pwg_representative sub-directory
  $representative_file_path = original_to_representative($filename,
                                                         $representative_ext);
  prepare_directory(dirname($representative_file_path));
   
  $ffmpeg = $conf['ffmpeg_dir'].'ffmpeg';
  $ffmpeg.= ' -i "'.$filename.'"';
  $ffmpeg.= ' -an -ss '.$second;
  $ffmpeg.= ' -t 1 -r 1 -y -vcodec mjpeg -f mjpeg';
  $ffmpeg.= ' "'.$representative_file_path.'"';
  $ffmpeg.= ' 2>&1';


And, per your above instructions, I've changed it to:

// move the uploaded file to pwg_representative sub-directory
  $representative_file_path = original_to_representative($filename,
                                                         $representative_ext);
  prepare_directory(dirname($representative_file_path));

  $second = 1;

  $ffmpeg = default_conf('ffmpeg_dir', '').'ffmpeg';
  $ffmpeg.= ' -i "'.$filename.'"';
  $ffmpeg.= ' -an -ss '.$second;
  $ffmpeg.= ' -t 1 -r 1 -y -vcodec mjpeg -f mjpeg';
  $ffmpeg.= ' "'.$representative_file_path.'"';
  $ffmpeg.= ' 2>&1';


Look right? :p

Offline

 

#34 2015-01-06 01:29:33

wsloand
Member
2014-11-17
31

Re: Raw File SUpport

That looks good to me.  :)

Offline

 

#35 2015-01-06 01:30:16

SnotRocket
Member
2014-08-15
63

Re: Raw File SUpport

Bad ass!

Thank you... it's already working!

Sort of :p

So... I'm now able to process my videos, and thumbnails are generated.

However... MOV's and M4V's aren't identified as videos to piwi (?) and it's broken my VideoJS plugin.  It used to work ;)

Seems that MP4's work fine.

Thoughts?

Last edited by SnotRocket (2015-01-06 01:51:19)

Offline

 

#36 2015-01-06 03:49:05

wsloand
Member
2014-11-17
31

Re: Raw File SUpport

For m4v, if you add that in the list at the bottom of ffmpeg.php, it should work (look for 'wmv,'mov', ... and add it as another quoted string there).  Anything that ffmpeg supports can go there, and it should work.  But, "work" here is defined as "make a thumbnail and allow the video to be downloaded when downloading the original file".  "work" does not mean "display the video like VideoJS does/did".

For fixing VideoJS, I don't know how it works.  I just emailed xbgmsharp to see if he knows what part of this broke the plugin.

Offline

 

#37 2015-01-09 16:58:56

SnotRocket
Member
2014-08-15
63

Re: Raw File SUpport

So, I've been using this for a few days now and have processed ~10,000 raw images or so into my gallery...

With the above code changes.. it's working flawlessly... with the only exception being the MPG file's are broken.

Awesome work... thank you!!!!

Offline

 

#38 2015-01-12 01:55:28

wsloand
Member
2014-11-17
31

Re: Raw File SUpport

SnotRocket, what is the difficulty with mpg files?

Offline

 

#39 2015-01-13 15:06:05

plg
Piwigo Team
Nantes, France, Europe
2002-04-05
13786

Re: Raw File SUpport

Hi wsloand,

I've reviewed your code (not tried yet, just reviewed). It's a big piece of code, you must have studied quite some time to understand the way Piwigo works :-)

I like that you cleanly segmented the code, with extracting handlers of TIFF/PDF/VIDEO in a clean functions.

You have "invented" a new modular system with your upload_plugins and shown how to handle RAW files with this system. You could have simply extended the code in admin/include/functions_upload.inc.php:add_uploaded_file just like I did to generate pwg_representative for PDF/TIFF/VIDEO files. But your system is better because it also works for the sync process. Good work! I'm sure all users of synchronization will appreciate :-)

What I don't like so much is the way "upload_plugins" works, because I don't think it's really necessary and I know that it will require some extra work to manage the upload_plugins like we manage the plugins (on piwigo.org/ext and on Piwigo administration). What is important is that we can "extend" the system with additional handlers and that we can add them with plugins.

Here is another way to manage those "handlers", that don't require to have a new "upload_plugins" system.

Code:

$conf['representative_handlers'] = array(
  'upload_tiff' => array('tiff', 'tif'),
  'upload_ffmpeg' => array('mp4', 'mpeg'),
  'upload_pdf' => array('pdf'),
);

Then we use these handlers at the appropriate places (add_uploaded_file and sync). Any plugin can modify the $conf['representative_handlers']. For example, you can have a "RAW support" plugin that do:

Code:

$conf['representative_handlers']['upload_dcraw'] = array('arw', 'cr2', 'crw', 'dcr', 'mrw', 'nef', 'orf', 'raf', 'x3f');

(at the appropriate trigger, such as "init")

What do you think? Do you want to try this change or do you prefer I do it while implementing your patch?

Offline

 

#40 2015-01-13 15:45:09

mistic100
Former Piwigo Team
Lyon (FR)
2008-09-27
3277

Re: Raw File SUpport

my two cents without reading the patch

I suggest to just rely on the existing triggers system

where uploaded files are "converted" :

Code:

$handled = trigger_change('upload_file', false, $file);

somewhere in Piwigo core (I use Closure for simplification, but we can't use them) :

Code:

add_event_handler('upload_file', function($handled, $file)
{
  if ($handled)
  {
    return $handled;
  }

  if (in_array(get_extension($file), array(/** core formats **/))
  {
    // do something with JPEG, PNG, GIF, BMP
    
    return true;
  }

  return false;
});

(here we must define multiple handlers, one for image files, one fore video containing PLG code, ...)

And plugins use the same type of handler for raw files, pdf files, etc.

---

this is how pwg_login works to allow third party login systems

Offline

 

#41 2015-01-13 15:51:01

plg
Piwigo Team
Nantes, France, Europe
2002-04-05
13786

Re: Raw File SUpport

The simpler, the better! mistic's solution is simpler than mine :-)

Offline

 

#42 2015-01-14 01:30:42

wsloand
Member
2014-11-17
31

Re: Raw File SUpport

I'm happy to have the simple solution, and I'll admit that I didn't get that far in my spelunking in the code.  I was wanting to use the existing trigger system, but I didn't see exactly how.  The one thing that I'm not sure I see in the below example is the ability to have different priorities for different plugins with fallback to less preferred ones.

The reason that I think that priorities are important is the example of extracting the image from the raw file.  I know that some of my raw files have thumbnails and some don't.  So, I may want the first priority to extract the .jpg that is embedded within the raw using exiftool, and if that doesn't succeed, then fall back to dcraw conversion.

I also agree that the way that I setup the upload_plugin directory was rather clunky.  It worked, it allowed a simple method for a different file per file handler, and so that's what I used.  If it's simple to put it in as part of the normal plugin interface, then I'm all for it.

Regardless, I definitely would prefer someone more knowledgeable putting together the final patch.  (Thank you plg!)

Offline

 

#43 2015-01-14 04:05:38

SnotRocket
Member
2014-08-15
63

Re: Raw File SUpport

wsloand wrote:

SnotRocket, what is the difficulty with mpg files?

Since there is more information above that I don't completely understand - maybe the solution is already there.

Before I implemented your patch, mpeg files used to play in my gallery.

Now, they do not.

Here's a quick example...

http://thehouseofjohnson.net/gallery/in … tegory/274

All of the files/thumbs/images in that link are videos...  The ones that you click on and only get an image are mpeg (actual extension of .mpeg) files (the top portion of that album are all mpegs)... scroll down closer to the bottom, and they are all mp4's.  When you click on an MP4 (actual extension of .mp4), VideoJS (I assume/think) plays the video.

Offline

 

#44 2015-03-09 04:20:13

wsloand
Member
2014-11-17
31

Re: Raw File SUpport

Hi,

I have a bit more time to work on the raw support patch for the next week or so if no one else has had time to progress it toward integration.  And, I have new motivation.  (I'm a new dad!  Grandparents want pictures!)

I didn't quite follow mistic's suggested method for how I could use the trigger_change and add_event_handler functions.

Also, I'm not quite sure how plg would like to have the new file types added.  They should not be handled with the existing plugin system-- should we assume that they're integrated into the piwigo trunk?  Where would you recommend that the files go?  To me, the most obvious two locations are either under admin somewhere or creating a new plugin-related directory.

Have a good day,

Bill

Offline

 

#45 2015-03-12 17:01:21

plg
Piwigo Team
Nantes, France, Europe
2002-04-05
13786

Re: Raw File SUpport

Hi wsloand,

Sorry for this late reply.

wsloand wrote:

I have a bit more time to work on the raw support patch [...]

Cool :-)

wsloand wrote:

(I'm a new dad!  Grandparents want pictures!)

Congrats. This is a bit why Piwigo was designed at the beginning ;-)

wsloand wrote:

I didn't quite follow mistic's suggested method for how I could use the trigger_change and add_event_handler functions.

He gave the general form of how the file formats could be implemented. He simply says we should not invent a new system while we have triggers which works perfectly fine.

wsloand wrote:

Also, I'm not quite sure how plg would like to have the new file types added.  They should not be handled with the existing plugin system-- should we assume that they're integrated into the piwigo trunk?  Where would you recommend that the files go?  To me, the most obvious two locations are either under admin somewhere or creating a new plugin-related directory.

We can have some handlers in code and be able to add handlers with plugins. mistic suggests to see how pwg_login works. If you want I can write the core handlers, then you can create a plugin for another handler (or several handlers, if it's relevant to group them)

Offline

 

Board footer

Powered by FluxBB

github twitter newsletter Donate Piwigo.org © 2002-2024 · Contact