Announcement

  •  » Extensions
  •  » Quick local synchronization from command line

#1 2017-02-11 11:23:43

rob777
Member
2017-02-11
9

Quick local synchronization from command line

Hello/Hi/Greetings,

I would like to perform a quick sync regularly (daily) from the server (in the crontab).
So it would be great if I could call the quick sync action from bash .
Is this something feasible ?
Thanks.
Regards

        Piwigo 2.8.6
        Operating system: Linux
        PHP: 7.0.13-0ubuntu0.16.04.1 (Show info) [2017-02-11 10:30:53]
        MySQL: 5.5.5-10.0.29-MariaDB-0ubuntu0.16.04.1 [2017-02-11 10:30:53]
        Graphics Library: GD 2.1.1

Offline

 

#2 2017-02-27 11:23:01

kiteboy
Member
2017-02-08
17

Re: Quick local synchronization from command line

I would alsoo like to see this facility - not sure if its possible though??

Offline

 

#3 2017-02-28 17:17:57

rob777
Member
2017-02-11
9

Re: Quick local synchronization from command line

it looks like this not possible via API.
This method would be a nice to have feature.

I don't know if this is possible via an HTTP POST command....


Has anyone tried something in this way ? any clue ?

Offline

 

#4 2017-03-03 11:26:41

torsten
Member
2017-03-03
2

Re: Quick local synchronization from command line

I wrote a python script that watches your images folder and syncs it via piwigo webservice if it recognized a change. The script uses a linux kernel module named inotify to watch the folder (most effective one).


#!/usr/bin/python
# coding: utf-8
import sys
import inotify.adapters
import inotify.constants
import time
import requests
import multiprocessing

__author__ = 'torsten'

dir = <your images dir>

def refreshPiwigo():
    user = "piwigo username"
    password = "piwigo password"
    base_url = "http://localhost/piwigo"

    loginData = {}
    loginData["method"] = "pwg.session.login"
    loginData["username"] = user
    loginData["password"] = password

    session = requests.Session()
    loginResponse = session.post(base_url + '/ws.php?format=json', data=loginData)
    print(loginResponse.text)

    syncData = {}
    syncData["sync"] = "files"
    syncData["display_info"] = 1
    syncData["add_to_caddie"] = 1
    syncData["privacy_level"] = 0
    syncData["sync_meta"] = 1
    syncData["simulate"] = 0
    syncData["subcats-included"] = 1
    syncData["submit"] = 1

    startTime = time.time()
    syncResponse = session.post(base_url + '/admin.php?page=site_update&site=1', data=syncData, timeout=600.0)
    endTime = time.time()
    print(syncResponse.text)
    print(endTime - startTime)

if __name__ == '__main__':
    try:
        refreshPiwigoProcess = None

        i = inotify.adapters.InotifyTree(dir, mask=(inotify.constants.IN_CREATE | inotify.constants.IN_MODIFY), block_duration_s = 3)

        lastChangeTime = None
        for event in i.event_gen():
            if event is None:
                if lastChangeTime is not None and time.time() - lastChangeTime > 10 and (refreshPiwigoProcess is None or not refreshPiwigoProcess.is_alive()):
                    print("run check")
                    lastChangeTime = None
                    refreshPiwigoProcess = multiprocessing.Process(target=refreshPiwigo)
                    refreshPiwigoProcess.start()
            else:
                filename = event[3]
                if not filename.startswith('.'):
                    lastChangeTime = time.time()

    except KeyboardInterrupt:
        print "stopping..."

    sys.exit(0)


To run it install the following python libs:
   pip install requests
   pip install inotify

I installed it as a service on my linux machine so it will starts when booted.

Offline

 

#5 2017-03-04 21:55:57

rob777
Member
2017-02-11
9

Re: Quick local synchronization from command line

Hi Torsten,
Great !!! I will definitely test it pretty soon since this is actually exactly what I need.
In my case I will not need the inotify feature since my process pushes a lot of updates all at once, and I will rather trigger it from another script once all changes are made.
It looks promising, I'll keep you in the loop.
Thank you very much for this script.

Offline

 

#6 2017-03-05 12:16:15

torsten
Member
2017-03-03
2

Re: Quick local synchronization from command line

Hi rob777,

actually this implementation waits until there are no changes made for 10 sec (including running uploads), but if you have a trigger to start the script, that is better. I am using SFTP to upload it and do not have a chance to start a script when done.

by the way: to sync the piwigo-videoJS plugin and generate poster images for the videos

#!/usr/bin/python
# coding: utf-8

__author__ = 'torsten'


import requests
import time

user = "your username"
password = "your password"
base_url = "http://yourserver/piwigo/"

loginData = {}
loginData["method"] = "pwg.session.login"
loginData["username"] = user
loginData["password"] = password

session = requests.Session()
loginResponse = session.post(base_url + '/ws.php?format=json', data=loginData)
print(loginResponse.text)

syncData = {}
syncData["mediainfo"] = "mediainfo"
syncData["ffmpeg"] = "ffmpeg"
syncData["metadata"] = 1
syncData["poster"] = 1
syncData["postersec"] = 4
syncData["output"] = "jpg"
syncData["posteroverlay"] = 1
syncData["subcats_included"] = 1
syncData["submit"] = 1
syncData["cat_id"] = 0

startTime = time.time()
syncResponse = session.post(base_url + 'admin.php?page=plugin&section=piwigo-videojs%2Fadmin%2Fadmin.php&tab=sync', data=syncData, timeout=600.0)
endTime = time.time()
print(syncResponse.text)
print(endTime - startTime)

Offline

 

#7 2019-07-28 12:21:49

d3mon01
Member
2019-07-28
1

Re: Quick local synchronization from command line

first of all great job with the script works perfectly one small question is there a way to assign a group to new albums when they are created i have multiple users with there own gallery but have no way of keeping the albums separate with out manually assign the album to a group if i could add a line in i was thinking maybe sync data command (just not sure what it would be) and running scripts for each use thanks in advance

Offline

 

#8 2020-01-09 15:13:27

harry69
Member
2020-01-09
1

Re: Quick local synchronization from command line

Hello,

can you tell me please which path has to be given here ?

dir = <your images dir>

It shoud'nt be the base_url

thanks for clarification


torsten wrote:

I wrote a python script that watches your images folder and syncs it via piwigo webservice if it recognized a change. The script uses a linux kernel module named inotify to watch the folder (most effective one).


#!/usr/bin/python
# coding: utf-8
import sys
import inotify.adapters
import inotify.constants
import time
import requests
import multiprocessing

__author__ = 'torsten'

dir = <your images dir>

def refreshPiwigo():
    user = "piwigo username"
    password = "piwigo password"
    base_url = "http://localhost/piwigo"

    loginData = {}
    loginData["method"] = "pwg.session.login"
    loginData["username"] = user
    loginData["password"] = password

    session = requests.Session()
    loginResponse = session.post(base_url + '/ws.php?format=json', data=loginData)
    print(loginResponse.text)

    syncData = {}
    syncData["sync"] = "files"
    syncData["display_info"] = 1
    syncData["add_to_caddie"] = 1
    syncData["privacy_level"] = 0
    syncData["sync_meta"] = 1
    syncData["simulate"] = 0
    syncData["subcats-included"] = 1
    syncData["submit"] = 1

    startTime = time.time()
    syncResponse = session.post(base_url + '/admin.php?page=site_update&site=1', data=syncData, timeout=600.0)
    endTime = time.time()
    print(syncResponse.text)
    print(endTime - startTime)

if __name__ == '__main__':
    try:
        refreshPiwigoProcess = None

        i = inotify.adapters.InotifyTree(dir, mask=(inotify.constants.IN_CREATE | inotify.constants.IN_MODIFY), block_duration_s = 3)

        lastChangeTime = None
        for event in i.event_gen():
            if event is None:
                if lastChangeTime is not None and time.time() - lastChangeTime > 10 and (refreshPiwigoProcess is None or not refreshPiwigoProcess.is_alive()):
                    print("run check")
                    lastChangeTime = None
                    refreshPiwigoProcess = multiprocessing.Process(target=refreshPiwigo)
                    refreshPiwigoProcess.start()
            else:
                filename = event[3]
                if not filename.startswith('.'):
                    lastChangeTime = time.time()

    except KeyboardInterrupt:
        print "stopping..."

    sys.exit(0)


To run it install the following python libs:
   pip install requests
   pip install inotify

I installed it as a service on my linux machine so it will starts when booted.

Offline

 
  •  » Extensions
  •  » Quick local synchronization from command line

Board footer

Powered by FluxBB

github twitter newsletter Donate Piwigo.org © 2002-2024 · Contact