Announcement

  •  » Miscellaneous
  •  » Process for autmating video processing and uploading?

#1 2020-11-10 16:16:18

mosher
Member
2020-11-10
4

Process for autmating video processing and uploading?

I have a workflow for videos that I'm looking to automate. Does anyone know of an existing solution before I try implementing it myself?

My basic steps are:
1. Select a set of videos.
2. Process the videos with Handbrake/ffmpeg to get a smaller file size.
3. Generate poster images for the videos.
4. Upload the videos via ftp.
5. Sync the uploaded videos.

The idea is I could choose my set of videos, and have them automatically processed and uploaded. I expect I might have to manually do the sync step still.

I use the videojs extension for videos. For image uploading I use Shotwell but it isn't aware Piwigo can accept videos so it won't allow me to upload them. Plus I couldn't automate the processing step with Shotwell anyway.

Offline

 

#2 2020-11-10 17:12:09

TheDoc
Member
Munich / Germany
2015-10-18
173

Re: Process for autmating video processing and uploading?

Listening in tho this thread :-) I have exactly the same workflow without any cool automation...

Offline

 

#3 2021-01-03 20:25:04

mosher
Member
2020-11-10
4

Re: Process for autmating video processing and uploading?

So I've made a solution that works for myself. I believe all but one part should be generally useful.

First off, I found Piwigo Import Tree when I was 95% done with this. I'm not using it in part as I was basically done with my solution, and there were some gaps in the documentation that left me wondering if it was a good fit for my flow. But it might work for you, so check it out.

As for my solution, in short I make a text file list of all the videos I want to upload, and then process these files with fileProc, a Python script I made to run a set of commands on a set of files. I have it configured to:
1. Transcode a video to a level a quality I've found to be a good balance of size and quality with HandBrake-CLI.
2. Generate a poster image with ffmpeg.
3. sftp the directory structure to my server. I have a public/private ssh key in use, so no login is required.
4. Use piwigo-actions, another set of scripts I made, to login to piwigo, run synchronization, and log out. I also found piwigo-scripts which does the same and more. But I believe it is intended for older piwigo sites as the URLs used don't match what my site uses.
5. Clean-up my local directory as I don't care to have backups of the files.

I have more thorough documentation on the sites for each tool, but there are a few points worth calling out here:
1. fileProc is hard coded assuming you want to use my file structure. Photos are organized by date locally in folders of yyyy/mm/dd. And you want them in folders on piwigo like yyyy/mm_Month. I'd like to make this more generic so others can translate their folder structure however they want, but currently you'd need to modify the code to handle something else.
2. I've tested this only on my linux machine. I suspect on Windows or Mac you'd have trouble getting them to work. The key dependencies are bash and curl for piwigo-actions and python3 for fileProc. But all the configured commands I run to do everything else within fileProc also assume bash and other unix commands.
3. This ends up with the videos available on your site, but they are missing checksums. I'd like to make this automatic. But should probably just write a bug report (if I can ever access the bug site) as it seems like a deficiency in Piwigo's synchronization.

Hopefully this is helpful. I'll do what I can to answer questions. Unfortunately, I probably won't have much time to improve the tools in the short term. Though the hard coded assumptions on directory structures is really bugging me.

Offline

 

#4 2021-03-02 15:26:20

blacksmith
Member
2021-03-01
2

Re: Process for autmating video processing and uploading?

Hi mosher,

It´s really interesting your work!

As I said on other post (https://piwigo.org/forum/viewtopic.php?id=31091) I´m interested in the syncronization part. If I understand well, your  piwigo-actions/piwigo.sync.sh makes the same job as Piwigo´s tools/remote_sync.pl but without the Perl dependency. Am I right?

I´m looking for the best/efficient way of doing this sync job (for images and videos) so I can develop an automatic process in a cronjob. I´ll try your piwigo-actions...

Thanks!!

Last edited by blacksmith (2021-03-02 15:29:16)

Offline

 

#5 2021-03-10 11:56:36

mosher
Member
2020-11-10
4

Re: Process for autmating video processing and uploading?

Hi blacksmith,
piwigo-actions/piwigo.sync.sh is the same doing a Synchronize action through your piwigo website. So it doesn't do much without the other actions and something to tie them together, like my fileProc script. I haven't tried tools/remote_sync.pl, or found documentation for it, so I don't know all the ways they may differ.

I should add I haven't tested my tools with Piwigo 11, so they may be broken. I'm waiting on key extensions to support it, or a chance to test it out myself. But don't have a lot of time for it, so it may be a while before I upgrade.

Offline

 
  •  » Miscellaneous
  •  » Process for autmating video processing and uploading?

Board footer

Powered by FluxBB

github twitter newsletter Donate Piwigo.org © 2002-2024 · Contact