Announcement

#16 2014-12-01 12:39:13

noyaudabricot
Member
Brussels
2014-12-01
6

Re: Raw File SUpport

Handling raw files in a transparent way, a dream comes true...

Offline

 

#17 2014-12-02 00:10:49

wsloand
Member
2014-11-17
31

Re: Raw File SUpport

The current issue that I'm having with the patch is the following:

During the upload process, it generates a pwg_representative .jpg file, but when I go to the page, it sets the following for the data-src in the img tag

data-src="i.php?/upload/2014/11/21/20141121213009-be293b89-th.tif"

When I go to http://server/piwigo/i.php?/upload/2014/11/22/pwg_representative/20141122063225-be293b89-th.jpg , it works (note that I added the pwg_representative).

What file contains the logic for using the pwg_representative or the normal file path when setting the data-src?

Edit: I found the issue that I was having.  I set too many extensions as $conf['picture_ext'].  That is now resolved, and I should have an updated patch by this weekend.

Last edited by wsloand (2014-12-02 14:14:51)

Offline

 

#18 2014-12-06 08:48:29

Analog Kid
Member
2014-12-03
6

Re: Raw File SUpport

$conf['breath_state']='bated';

Last edited by Analog Kid (2014-12-06 08:48:52)

Offline

 

#19 2014-12-07 01:09:26

wsloand
Member
2014-11-17
31

Re: Raw File SUpport

I just uploaded a new patch to the bug in the tracker (http://piwigo.org/bugs/view.php?id=3182)

Please review the upload_plugin_support.diff as a change relative to the current svn trunk for addition of raw support.  It currently has only been integrated into the web form upload, but it should be relatively straight forward to integrate into synchronization and app uploads.

For raw file conversion, it requires dcraw, convert from ImageMagick, and exiftool to convert the file from raw to jpg and to copy the tags from the raw to the jpg file.  I found that just using dcraw and convert missed a lot of tags that I'd prefer to have carried over.

I think that as the patch continues to come together, it will likely change the way that file extensions are tracked in piwigo:

$conf['picture_ext'] will remain as the idea of "anything that can be displayed by a browser natively.
$conf['file_ext'] will become "anything that should be uploaded, but does not have an upload plugin installed for it"
$conf['upload_form_all_types'] will likely go away (don't put it into file_ext if you don't want it uploaded)

The to-do list is:

[*] Additional testing (currently I've only tested jpg, tiff, and Canon CR2 files)

[*] Integration with synchronization

[*] Document how to create an upload plugin

[*] Create an option generate representative images during synchronization ($conf['generate_representative_sync'])

[*] Create another option to generate the representative image when you go to the web page for the first time ($conf['generate_representative_view'])

[*] Remove $conf['upload_form_all_types'] through all sources

[*] Make a generalized ImageMagick tool that will take any format that convert supports and convert the image.

Offline

 

#20 2014-12-31 18:39:10

wsloand
Member
2014-11-17
31

Re: Raw File SUpport

At long last, the patch is ready for general testing.  The patch now attached to the bug (http://piwigo.org/bugs/view.php?id=3182, file named upload_plugin_support-20141231.diff) should do everything needed to generalize support for different file types.  The patch does the following:

* If a pwg_representative already exists (because you uploaded it manually), then don't generate a new one.
* Allow pwg_representative generation when
** Files are uploaded directly
** Files are synchronized (ftp+synchronization, optional)
** Files are viewed for the first time
* File extensions are now all case insensitive in $conf['picture_ext'] and $conf['file_ext']
* $conf['file_ext'] only needs to have extensions that you want to be able to upload, but that do not have an upload_plugin.  It doesn't need to have all extensions.

You should now no longer need to create a representative extension if an upload_plugin is available to support the file extension, and upload_plugins are easy to write (essentially, take in a file name, run a shell command to generate your pwg_representative, return an array with success and the representative extension).

The patch should apply cleanly to the current subversion trunk.

A few notes for the more seasoned Piwigo developers:
* I don't think that the function is_valid_image_extension in admin/include/functions_upload.inc.php is needed anymore.  Can you please verify that, and if not, delete it?
* There are a few places where I'm needing to include the new files that may be a bit processing-intensive.  If you see a way to simplify that, I'd definitely be interested.

As far as I can tell, everything is done to make raw images work in Piwigo and to enable many more file types to be easily supported.

Last edited by wsloand (2014-12-31 18:41:28)

Offline

 

#21 2014-12-31 18:46:41

plg
Piwigo Team
Nantes, France, Europe
2002-04-05
13180

Re: Raw File SUpport

Nice, I have to test it! Thank you wsloand.


Latest blog post (November 9th 2018) Why Flickr could not remain free for ever

Offline

 

#22 2014-12-31 20:13:41

SnotRocket
Member
2014-08-15
63

Re: Raw File SUpport

I would love to try this out...

What's the best way to import this functionality into my stable release?

Offline

 

#23 2015-01-02 23:08:38

NoProbs
Member
2015-01-01
6

Re: Raw File SUpport

Sounds really promising - I have just started to migrate away from menalto Gallery (after many years of happy use) and TIFF/RAW support (upload and sync) was one of the functions I needed

Also happy to test if I can add to 2.7.2 as a plugin

Offline

 

#24 2015-01-02 23:20:13

wsloand
Member
2014-11-17
31

Re: Raw File SUpport

SnotRocket and NoProbs, since this is relatively untested, I'm not sure that it won't horribly break your installation, delete all your pictures, corrupt the database, and cause insurrection in a nearby country.  That said...

If you download the diff file from the bug report and then run the following at the command line, it should work:

cd /path/to/piwigo
patch -p0 < /path/to/upload_plugin_support-20141231.diff

You'll also need to ensure that dcraw and exiftool are on your path (exiftool is optional, but it will maintain the picture tags).

Offline

 

#25 2015-01-04 20:44:35

NoProbs
Member
2015-01-01
6

Re: Raw File SUpport

wsloand - I have tested sync and upload in a test environment and in my limited testing it all appeared to work as expected.  I uploaded and synchronised TIFF, JPG and NEF files

I look forward to seeing this is in a production release - if I can help with more testing then please advise.

Thanks

Offline

 

#26 2015-01-04 21:40:11

wsloand
Member
2014-11-17
31

Re: Raw File SUpport

Thanks NoProbs!

What was the capitalization of the extensions that you uploaded?  (You wrote them as all caps, but I want to verify that.)  The reason I ask is that I tried to make it no longer sensitive to capitalization of the file extension (jpg = JPG = JpG), but that is the part that I was least certain I'd caught all the uses.

Offline

 

#27 2015-01-04 22:04:12

NoProbs
Member
2015-01-01
6

Re: Raw File SUpport

I have successfully uploaded upper and lower case (jpg, tiff and nef) extensions

Offline

 

#28 2015-01-05 17:38:13

SnotRocket
Member
2014-08-15
63

Re: Raw File SUpport

So.. I'm having a few issues.

Working on my own to resolve them.

But, I'm getting:

Unable to handle upload of ./galleries/RandomImages/20DMacroShots/IMG_0297.CR2

Those were taken with my old 20D

I'm also having an issue with:

[prepare_directory] cannot create directory "./galleries/RandomImages/RawTest/pwg_representative"

Offline

 

#29 2015-01-05 18:01:46

SnotRocket
Member
2014-08-15
63

Re: Raw File SUpport

Both appear to be my issues...

Seems to be working... still testing!

Offline

 

#30 2015-01-05 18:11:39

SnotRocket
Member
2014-08-15
63

Re: Raw File SUpport

Just to preface - I'm a FULLY FTP managed user... I don't use webupload...

So.. seeing some error messages...

I added a few raw images to an existing folder...

I went to view the folder... and the thumbnails were not auto generated at all.

I clicked on the "RAW" image and my re-sized imaged worked (Awesome!!).

I hit the back button in my browser and the thumbnails now show (Awesome!!)

But I have an error message at the top of the page:

Notice: Undefined variable: conf in /var/www/2000transam.com/gallery/upload_plugins/ffmpeg.php on line 41 Notice: Undefined variable: second in /var/www/2000transam.com/gallery/upload_plugins/ffmpeg.php on line 43

Everything seems to be working however....

edit: after doing it a few times (as described above)... I no longer see the error message.

Last edited by SnotRocket (2015-01-05 18:50:27)

Offline

 

Board footer

Powered by FluxBB

github twitter facebook google+ newsletter Donate Piwigo.org © 2002-2019 · Contact