Piwigo.org

plg · 2008-07-31 00:21:02

web API refers to webservices (french documentation only, translators wanted)

[Bugtracker] ticket 839

In order to make remote uploader easier to create, I have added a basic (currently as a proof of concept) pwg.images.add web API method on Butterfly. tools/piwigo_remote.pl is a Perl prototype for remote client using the web API. Here is the way I call the remote client:

Code:

 ./piwigo_remote.pl --action=pwg.images.add --file=test.jpg --thumbnail=TN-test.jpg --name="raspberry" --category_id=579

This will send the files content base64 encoded in POST parameters. On the Piwigo side, ws_images_add dump the files in upload/<year>/<month>/<day>/<datetime>-<random>.jpg

- upload = "upload" directory at the root of the Piwigo installation
- <year> = current year
- <month> = current month
- <day> = current day
- <datetime> = current datetime formatted as CCYYMMDDhhmmss
- <random> = 6 first characters of the hexadecimal md5 of the photo file

Code:

$ tree ~/public_html/piwigo/trunk/upload
/home/pierrick/public_html/piwigo/trunk/upload
`-- 2008
    `-- 07
        `-- 30
            |-- 20080730225003-ce00292c.jpg
            `-- thumbnail
                `-- TN-20080730225003-ce00292c.jpg

The photo has no #images.storage_category_id. This is a problem on some places like admin/picture_modify.php because Piwigo has always considered it was a "not null" field. I have considered that such upload did not require to have a physical category behind it (ie the directory is an upload directory, not a way to organize the display of photos).

There is no "synchronization" step, in ws_images_add, I simply insert a line in #images and another one in #image_category.

- What's your opinion about this feature?
- Do you think #images.storage_category_id should be required?
- Do you think an image must be linked to its storage category ? (I have some doubts know)

rub · 2008-07-31 14:26:40

IMO, #images.storage_category_id is necesary... For maintenance for exemple, when directories are computed.

trunc/upload is not a idea, directory must be associated to a site.

For me, sub-directory if not necessary, a option will be best. And for each directory a category must created.

I prefer to keep standart at moment but perhaps a new way borned and I don't see all the avantages.

VDigital · 2008-07-31 15:15:32

storage_category

Storage_category is really useful. Mainly, when you have to transfer you website from one host to another. In these cases, we are used to run maintance to rebuilt our website.

With your proposal, pictures in /upload/yyyy/mm/...
would ever be in this structure and never be seen elsewhere.

Example to be clear enough:
Previous site (#_sites table):
1 ./galleries/

New site (#_sites table):
1 ../old/ (<= will contain all previous ./galleries/)
2 ./galleries/ (<= new structure)

Maintenance is the simpliest way to step by step reorganize our galleries.
Advanced users are familar with this procedure.

8-)

VDigital · 2008-07-31 15:32:02

IMO, let webmasters choosing their file organization.

By Admin, define default upload subdirectory like:
cat=25 and a random-subcat
or cat=26 and a random-file-prefix
or cat=27 and override file

(Only storage_categories till now).

(A next step could be to associate an existing storage_category to some virtual categories as their upload subdirectory. Uploaded image should only be visible via the storage_category.
A further step again, uploaded images could be linked (or not by option) to the used virtual category).

8-)

VDigital · 2008-07-31 15:44:02

Last remark maybe, what's about current synchronization?

Synchro which site is related to these uploaded pictures? None, is that correct?

--category_id=579
if cat=579 is a physical cat related to site=1

At the first running synchro directories+files, your uploaded pictures will be deleted, don't you think so?

8-)

plg · 2008-08-01 00:05:52

VDigital wrote:
--category_id=579
if cat=579 is a physical cat related to site=1

At the first running synchro directories+files, your uploaded pictures will be deleted, don't you think so?

Not at all. A physical category can be virtually linked to any photo (virtual links were "invented" before virtual categories).

(I have read several times your opinions Rub and VDigital, I have to think more before replying)

VDigital · 2008-08-01 07:30:34

z0rglub wrote:
VDigital wrote:
--category_id=579
if cat=579 is a physical cat related to site=1

At the first running synchro directories+files, your uploaded pictures will be deleted, don't you think so?
Not at all. A physical category can be virtually linked to any photo (virtual links were "invented" before virtual categories).

(I have read several times your opinions Rub and VDigital, I have to think more before replying)

That's correct. Link is made via storage_category_id. So in that case no problem.
I delete the folder and synchronize, what happens: Unlinked pictures are created.
How many users are aware of their unlinked pictures today?

8-)

(Indeed, we have to think more too, because I have already strong arguments with a lot of unexplored consequences.)

rub · 2008-08-01 11:52:02

With storage_category_id, we can check if the category is uploadable.
And define on category upload options like VDigital wrote (random-subcat, random-file-prefix, override file)

plg · 2008-08-01 23:56:10

VDigital wrote:
That's correct. Link is made via storage_category_id. So in that case no problem.

Not really, in fact I'm not so sure to understand what you mean. The link between an image and a category is made with table #image_category, not with #images.storage_category_id. #images.storage_category_id is used to know if such a link is a "storage" link or not and to rebuild the #images.path when performing the [Admin>Maintenance>Update images informations] action.

VDigital wrote:
I delete the folder and synchronize, what happens: Unlinked pictures are created.

What is an "unlinked" picture? A picture with no "virtual" link to a category (as opposed to "storage" link). If you delete a folder with files inside, the images corresponding to the files will be deleted in the database (and of course all related data such as links to categories, links to tags, related comments, and so on).

VDigital wrote:
How many users are aware of their unlinked pictures today?

That's right, very few users know it's possible to know which images have no "virtual" link.

I've been thinking about the current data model of the categories and I think that we have this as a constraint, I mean the link between a directory and a category. I'm still in favor of saying that by default "1 directory = 1 category" so that the gallery category hierarchy structure can reflect the filesystem directory tree, but I would also like to have 1 directory with no related category. I know what I'm saying is a huge change in the basics of the application but I currently feel frustrated with the data model concerning categories.

The changes I'm thinking about are too big for Butterfly. We won't have enough time for this, and I don't want to postpone the release. So l will propose some major changes about the data model once Butterfly will be released, or before as a discussion topic.

For now, I think I will add 2 configuration variables :

- upload_site. "1" by default, this site exists as soon as Piwigo is installed.
- upload_directory. "upload" by default

I don't use a storage "category_id" as default place for upload because it would mean there is at least one existing storage category, which is not the case right after Piwigo installation and I don't want to force the user to have one before using the web API method pwg.images.add. So ./galleries/upload will be automatically created if it does not exists at first upload (and a corresponding category to avoid breaking current rules). The corresponding category will be private by default (as long as permission are not managed at image level).

The uploaded file will be dumped as ./galleries/upload/20080730225003-ce00292c.jpg because I don't want to have subdirectories if there are corresponding categories (it would quickly produce an heavy category hierarchy). We may have many files in the same directory which is a bad thing if there are too many of them. This is why I wanted to dispatch files in a 3 levels directory tree. If the user has too many files, he will have to change the upload_directory to "upload2" for example :-/

rvelices · 2008-08-02 00:32:41

Pierrick I agree with the solution for butterfly and with the discussion for after that ...

In theory I'm not against getting rid of storage_category_id; However on my personal gallery I'm using the storage categories as real visible categories, so I would not be happy with this change

VDigital · 2008-08-02 08:52:19

rvelices, I agree.

z0rglub, I understand your need. I am missing maybe some points :

you said:
- upload_site. "1" by default, this site exists as soon as Piwigo is installed.
- upload_directory. "upload" by default

1 - Does that mean you are creating a rule where site = 1 has to exist (Why in the future you can't removed it to leave only 2, 3, ... which could ever be local sites)?
2 - Does that mean your users would be able to upload in a distant site (a non local one)?
3 - If today, without your new API, your gallery is structured like this:
./galleries/upload/2005-05/xxxxxx.jpg with only 1 directory already called upload with many subfolders
What about upgrade? Is that your upload directory suit you for the API?

From my point of view, your "upload directory" must one of the current available local directory anywhere in the tree and decided by the webmaster.
Examples:
Today I have decided that my "upload directory" is ./galleries/real/2008-08/
Next month, I am the webmaster, and I could decide to switch it to ./galleries/real/2008-09/

It isn't us to choose directory names in case of upload, it is the webmaster.
We can play with the data model, but we have to leave totally open-minded the physical model.

It can be powerful to make it free, don't you agree?

[My complete mindset is:
- ./galleries can be local but changed in ./folders/ or ../albums/ (Webmaster has to prepare it, procedure).
- sites can be moved, renamed (without any loss, but Webmaster need a procedure again).
- /thumbnail can be changed in location, can be renamed too (a tool for Webmaster).
- thumbnail prefix can be changed or removed (just by a simple Webmaster decision but asynchronously by the simple fact a thumb is going to be displayed on a picture page).
- /pwg_high can be changed in location, can be renamed too (Webmaster has to prepare it, procedure).

All contraints we have already introduced have to be removed.
So don't create new ones.]

Thanks.
8-)

plg · 2008-08-02 15:08:03

VDigital wrote:
1 - Does that mean you are creating a rule where site = 1 has to exist

No new rule. During install.php, site "1" is created. Read install.php :-)

VDigital wrote:
(Why in the future you can't removed it to leave only 2, 3, ... which could ever be local sites)?

You can, but who said you couldn't change the configuration parameter for upload too?

VDigital wrote:
2 - Does that mean your users would be able to upload in a distant site (a non local one)?

Certainly not, it would be another level of complexity that I don't want to solve right now.

VDigital wrote:
3 - If today, without your new API, your gallery is structured like this:
./galleries/upload/2005-05/xxxxxx.jpg with only 1 directory already called upload with many subfolders
What about upgrade? Is that your upload directory suit you for the API?

If the user keep the default configuration parameters, web API uploaded photos will be uploaded in ./galleries/upload.

VDigital wrote:
From my point of view, your "upload directory" must one of the current available local directory anywhere in the tree and decided by the webmaster.

What if the user don't ever want to FTP connect and create a directory? This "web API way" to upload photo is designed to be used exclusively, I don't want to force the user to use FTP to upload anything.

VDigital wrote:
It isn't us to choose directory names in case of upload, it is the webmaster.
We can play with the data model, but we have to leave totally open-minded the physical model.

This is the way it was designed since the beginning. But it was also because directories were strongly linked to categories. If we change this link from mandatory to optionnal, I think it's ok because what the user really wants it to be free on the category hierarchy, I am not so sure he minds the directory tree if another tool do the job automatically (thanks to the web API method pwg.images.add).

The way Piwigo dispatches photo files could be considered as internal mechanics.

VDigital wrote:
All contraints we have already introduced have to be removed.

My proposition is to remove constraints, not to add some!

For now (with minimum changes) another way to solve the problem can be that photo must be uploaded in a "storage category". Another web API could be "pwg.category.create" with an option to create the corresponding directory.

Please consider this web API method to be used by the webmaster, not by gallery guests. My goal is that webmasters use the web API to upload their photos, no FTP upload, no thumbnail creation, no synchronization. Such a tool is under preparation, I think it will make first steps with Piwigo much more easy than the current process.

VDigital · 2008-08-02 15:19:53

z0rglub wrote:
For now (with minimum changes) another way to solve the problem can be that photo must be uploaded in a "storage category". Another web API could be "pwg.category.create" with an option to create the corresponding directory.

Please consider this web API method to be used by the webmaster...

Must be uploaded in a "storage category" is fine and "pwg.category.create" will sound well.
Web API method to be used by the webmaster: yes, but not only, you use many every day without knowing they are.
8-)

plg · 2008-08-02 16:02:30

VDigital wrote:
Web API method to be used by the webmaster: yes, but not only, you use many every day without knowing they are.

Uh? If this method is authorized for user with the "admin" status, how could it be used by a "non admin" user? (I realize I haven't placed this contraint yet, but I should have).

VDigital · 2008-08-02 16:17:20

Good point.

8-)

Piwigo.org

Announcement

#1 2008-07-31 00:21:02

[web API] add a photo

Code:

Code:

#2 2008-07-31 14:26:40

Re: [web API] add a photo

#3 2008-07-31 15:15:32

Re: [web API] add a photo

#4 2008-07-31 15:32:02

Re: [web API] add a photo

#5 2008-07-31 15:44:02

Re: [web API] add a photo

#6 2008-08-01 00:05:52

Re: [web API] add a photo

VDigital wrote:

#7 2008-08-01 07:30:34

Re: [web API] add a photo

z0rglub wrote:

VDigital wrote:

#8 2008-08-01 11:52:02

Re: [web API] add a photo

#9 2008-08-01 23:56:10

Re: [web API] add a photo

VDigital wrote:

VDigital wrote:

VDigital wrote:

#10 2008-08-02 00:32:41

Re: [web API] add a photo

#11 2008-08-02 08:52:19

Re: [web API] add a photo

#12 2008-08-02 15:08:03

Re: [web API] add a photo

VDigital wrote:

VDigital wrote:

VDigital wrote:

VDigital wrote:

VDigital wrote:

VDigital wrote:

VDigital wrote:

#13 2008-08-02 15:19:53

Re: [web API] add a photo

z0rglub wrote:

#14 2008-08-02 16:02:30

Re: [web API] add a photo

VDigital wrote:

#15 2008-08-02 16:17:20

Re: [web API] add a photo

Board footer