Announcement

#1 2016-09-11 01:44:10

Jessie
Member
2016-09-04
31

Tutorial to correctly FTP-upload pictures to Piwigo

Hello,

I've struggled a little to upload my pictures/videos to Piwigo, so I thought I'll share here what I've done.
I'm using Debian Stable, Nginx, Piwigo 2.8.2 and the Boostrap Darkroom theme.

First of all, I have chosen to upload pictures manually (i.e. not through the web interface) because uploading thousands of pictures through the web interface is not practical. A simple rsync/ftp is much more easier and reliable.

The first thing to do is to allow video files:
https://github.com/tkuther/piwigo-boots … eo-support

Prepare the pictures/videos before upload:
  - photo/videos in portrait mode need to be correctly rotated before the upload
  - filenames should not contains special characters
  - file timestamp should be correctly set
  - video format should be mp4 encoded with h.264/aac

After that you can upload the pictures into /var/www/piwigo/galleries and change the owner to www-data (don't forget access mode 755 for directories and 644 for files).

Then use the Tools/Synchronize to make Piwigo take into account your pictures/videos.

The pictures will be there but the first guy to visit your gallery will have to generate all thumbnails and other image size (which can take a long time if your web server is not powerful enough).
So let's help your poor visitor by pre-generating as much as possible.

Temporarily install the Videojs plugin, and in the plugin configuration (Synchronize tab) generate the thumbnails for the videos. Deactivate the plugin after that (Bootstrap Darkroom uses its own video plugin).
Use the Batch Manager to pre-generate some thumbnails, at least square, medium and XXL.

After that the only problems remaining should be:
  - to generate the thumbnails for the gallery (but you can generate them manually by visiting your galleries):
http://piwigo.org/forum/viewtopic.php?id=26710
  -some big videos won't appear correctly (video controls outside the display):
http://piwigo.org/forum/viewtopic.php?p … 33#p164533

Here is the python script I am using to prepare the pictures/videos before upload:

Code:

#!/usr/bin/env python
# Dependencies: python-pyexiv2, python-magic, exiftran, exiftool, ffmpeg>2.7

import os
import sys
import unicodedata
import argparse
import glob
import subprocess
import shutil

import pyexiv2
import magic

custom_image_extensions = ['.jpeg', '.png', '.jpg']
custom_substitutions = [('&', 'et'), ('(',''), (')',''), (' !', ''), ("'", ' '), (' ', '_')]
custom_video_mime_types = ['media', 'video']

def remove_accents(name):
    """Remove accents from a filename"""
    nkfd_form = unicodedata.normalize('NFKD', name)
    return u''.join([c for c in nkfd_form if not unicodedata.combining(c)])

def remove_special_characters(name):
    """Substitue some special characters"""
    new_name = name
    for a,b in custom_substitutions:
        new_name = new_name.replace(a,b)
    return new_name

def normalize_name(path, name):
    """Remove accents and special characters"""
    old_name = name
    name = remove_accents(name)
    name = remove_special_characters(name)
    if name != old_name:
        print 'rename %s to %s' % (os.path.join(path,old_name), os.path.join(path,name))
        os.rename(os.path.join(path,old_name), os.path.join(path,name))
    return name

def find_case_insensitive(dirname, extensions):
    """Helper function to find file extensions with case insensitive"""
    file_list = []
    for filename in glob.glob(os.path.join(dirname,"*")):
        base, ext = os.path.splitext(filename)
        if ext.lower() in extensions:
            file_list.append(filename)
    return file_list

def identify_rotated_files(file_list):
    """Identify which files are rotated"""
    rotated_files = []
    for name in file_list:
        metadata = pyexiv2.metadata.ImageMetadata(name)
        metadata.read();
        if "Exif.Image.Orientation" in metadata:
            orientation = metadata.__getitem__("Exif.Image.Orientation")
            orientation = int(str(orientation).split("=")[1][1:-1])
            if orientation != 1:
                rotated_files.append(name)
    return rotated_files

def correct_images(directory):
    """Correct pictures if necessary"""
    file_list = find_case_insensitive(directory, custom_image_extensions)
    if file_list:
        # Correct timestamp
        subprocess.check_output(["exiftool", "-overwrite_original", "-filemodifydate<createdate"] + file_list)
    rotated_files = identify_rotated_files(file_list)
    if rotated_files:
        backup_directory = os.path.join(directory, "original_private")
        if not os.path.isdir(backup_directory):
            os.mkdir(backup_directory, 0755)
        for name in rotated_files:
            shutil.copy2(name, os.path.join(backup_directory, os.path.basename(name)))
        subprocess.check_output(["exiftran", "-aip"] + rotated_files)

def convert_video(magic_loader, directory):
    """Convert video if not in mp4 format"""
    file_list = []
    for name in os.listdir(directory):
        path = os.path.join(directory,name)
        mime_type = magic_loader.file(path)
        if mime_type.split('/')[0] in custom_video_mime_types:
            file_list.append(path)

    for name in file_list:
        # Preserve timestamp
        subprocess.check_output(["exiftool", "-overwrite_original", "-TrackCreateDate<CreateDate", "-TrackModifyDate<CreateDate", "-MediaCreateDate<CreateDate", "-MediaModifyDate<CreateDate", "-ModifyDate<CreateDate", name])
        basename, extension = os.path.splitext(name)
        if extension != '.mp4':
            if (basename + '.mp4') not in file_list:
                # Convert video and rotate automatically (with ffmpeg > 2.7)
                subprocess.check_output(["ffmpeg", "-i", name, "-strict", "-2", "-acodec", "aac", "-vcodec", "libx264", "-preset", "veryslow", "-map_metadata", "0", "-y", basename + '.mp4'])
            backup_directory = os.path.join(directory, "original_private")
            if not os.path.isdir(backup_directory):
                os.mkdir(backup_directory, 0755)
            shutil.move(name, backup_directory)
        else:
            if extension =='.MP4':
                os.rename(name, basename + '.mp4')
                name = basename + '.mp4'
            # Rotate video if necessary
            rotation = subprocess.check_output(['exiftool','-Rotation',name]).split()
            if rotation and rotation[2] != "0":
                os.rename(name, basename + '.MP4')
                subprocess.check_output(["ffmpeg", "-i", basename + '.MP4', "-strict", "-2", "-acodec", "copy", "-vcodec", "libx264", "-preset", "veryslow", "-map_metadata", "0", "-y", name])
                os.remove(basename + '.MP4')
    if file_list:
        # Restore timestamp
        subprocess.check_output(["exiftool", "-overwrite_original", "-FileModifyDate<CreateDate", "-ext", "mp4", directory])

def prepare_directory(directory):
    """Prepare directory for upload to Piwigo"""
    magic_loader = magic.open(magic.MAGIC_MIME_TYPE)
    magic_loader.load()
    if not os.path.isdir(directory):
        raise ValueError("Directory '%s' does not exist" % directory)
    for path, subdirs, files in os.walk(unicode(directory), topdown=False):
        for name in files:
            normalize_name(path,name)
        for name in subdirs:
            normalize_name(path, name)
    for path, subdirs, files in os.walk(unicode(directory), topdown=False):
        for name in subdirs:
            if "private" not in name:
                correct_images(os.path.join(path, name))
                convert_video(magic_loader, os.path.join(path, name))
    if "private" not in directory:
        correct_images(directory)
        convert_video(magic_loader, directory)
    

if __name__ == "__main__":
    parser = argparse.ArgumentParser(description='Rename files')
    parser.add_argument('directory', help='Name of the directory to process')
    args = parser.parse_args()

    prepare_directory(args.directory)

Offline

 

#2 2017-05-04 08:21:01

blacktea
Member
2017-05-03
11

Re: Tutorial to correctly FTP-upload pictures to Piwigo

Nice tutorial, thank you!

Offline

 

#3 2017-05-04 16:39:55

vikozo
Member
suisse
2016-11-20
41

Re: Tutorial to correctly FTP-upload pictures to Piwigo

Hello
nice tutorial, would be nice to have a Option in Piwigo to synch it automatic, so you only would Need to FTP your files an finish...
have a nice day
vinc


Piwigo version: the newest since 2.8.2 / PHP: 5.6.27-0+deb8u1 /  MySQL: 5.5.5-10.0.27-MariaDB-0+deb8u1
Piwigo URL:              http://www.kocher.photos/piwigo/
Lightroom --> Piwigo extensions: https://alloyphoto.com/plugins/piwigo/

Offline

 

Board footer

Powered by FluxBB

github twitter newsletter Donate Piwigo.org © 2002-2024 · Contact