Announcement

#31 2014-11-03 08:41:59

JT
Guest

Re: PIWIGO crashes freezes servers

Observations and Recommendations

1. If PIWIGO is sluggish due to image uploads or syncs, simply RENAME i.php to something else before doing a SYNC and before going to "Recent Photos" or "Batch Manager". Then employ the STEPS 1-5 that PLG mentioned on this thread in previous post to force i.php resizing to run on only one image at a time. While i.php is still disabled renamed, go to Batch Manager and select images to resize and before clicking submit of course re-enable i.php by renaming it back to i.php and then click submit and run the manual resizing. Resize to the largest size you're going to use and/or M(medium) and then resize the smaller ones from that size. You don't want to create Large, and then XL and then XXL basically, only the other way around.

2. Know that when users "browse" piwigo and load an image based on their chosen 'preferred' default size, piwigo will automatically create the image size dynamically as they try to view it but seems not only for that current one, but also for the next image. Sometimes this is good but other times may overload the server if tons of users are trying to view a new album at the same time and triggering lots of resizes.

3. For lower end servers etc I found it best to disable XXL and for some setups even XL in piwigo "Photo Sizes" options. Many people are on HD 1920x1080 displays so the other resolutions are really not needed, they can just download the FULL resolution if they want better than "Large" (or XL max..) so there's no need for XXL at all in most cases.

I'll wait to hear from Plg if there's an easy way we can temporarily DISABLE piwigo's dynamic image resizing that happens when users try to view photo galleries/albums etc AND also how to disable the resizing that tries to happen when loading Batch Manager or Recent Photos, since if I'm in Batch Manager to MANUALLY CREATE the multiple-sizes then I definitely don't want it to still try to create the PREVIEW THUMBNAILS that it wants to use on the Batch Manager and Recent Photos pages, it should be content to wait until I've chosen to manually generate them and clicked 'submit'.

In my case it turns out what killed the server, froze/crashed it so that I had to reboot was these concurrent resizing attempts triggered by simply loading the Batch Manager or "Recent Photos" pages in the Admin panel.  Hopefully my explanation on the i.php disabling until immediately before running manual batch conversion/resizing will help others avoid similar issues, and maybe at some point there will be some options/switches in piwigo to easily/temporarily disable BatchManager/RecentPhotos from triggering resizes and to disable user/client/browser/viewer triggered resizes from people/bots browsing albums.

Using the trick listed in #1 above, I think I may not find it necessary to create the thumbnails on the client/widnows PC and upload them to the server along with the Original resolution images. Although it would still be useful in some cases so it would still be good to know the exact resolution that piwigo creates for XXL XL L tiny etc (for both portrait/vertical and landscape orientations) or at least the math/formula it uses to create these multiple sizes, if they differ based on input/original image size.

Now that using this method I have PIWIGO running great on even my smallest/weakest setup, a WD My Cloud Mirror, I'm sure I can cope much better on the VPS and dedicated server setups with quad-core 3.5Ghz XEON cpus etc (piwigo was freezing even those before with i.php GONE WILD! )

 

#32 2014-11-03 08:58:42

flop25
Piwigo Team
2006-07-06
7037

Re: PIWIGO crashes freezes servers

As said you can't disable the dynamic resizing but you can upload the resized pictures made elsewhere in data/i. It won't recreate them until that's the right dimension and the file time is newer than the original picture
having such a server so slow for that is not normal. Any other cms like WordPress use server side resizing, because client side resizing would mean a totally different technology
disabling extra sizes, resizing locally etc are advices already provider on this forum


To get a better help : Politeness like Hello-A link-Your past actions precisely described
Check my extensions : more than 30 available
who I am and what I do : http://fr.gravatar.com/flop25
My gallery : an illustration of how to integrate Piwigo in your website

Offline

 

#33 2014-11-03 09:25:59

JT
Guest

Re: PIWIGO crashes freezes servers

I agree with you that client-side resizing would not be a quick/easy route and perhaps not warranted yet since there's some workarounds as you've pointed out.

My point was that none of the other methods fixed my problem other than the specific steps I took as outlined in my previous post, because the "Batch Manager"/"Recent Photos" pages still tried to create THUMBNAILS from the Original high resolution images to show on those pages. That was the key to fixing issues for me, since as I've said I had at least 3 Different installs, two of which were on very high end FAST Xeon datacenter server CPUs and piwigo still brought those servers to their knees when dumped on with tons of high-res images all at one time triggering multiple resizes for each one of those large images. So again this affected my FAST servers as well as my SLOW server (which takes about 50-60seconds to resize a big 7.5K by 4.9K resolution image).

Also, I guess you mean you can't disable the dynamic resizing in options, just like for example you can't really disable 'resizing' unless you just rename i.php, but there's probably a way to do it, I just didn't plan on looking through the code, at least not yet.

I did just notice ONE SMALL PROBLEM now:

For some of my portrait/vertical orientation images, the M(medium/default) size, some of them are stretched saying:

396px × 593px (scaled to 792px × 528px)

While other ones do display correctly:

396px × 593px (scaled to 396px × 594px)

So it seems for some of the piwigo somehow thinks its a landscape mode 792 x 528 image when in fact it is portrait/vertical. How can I have piwigo re-analzye the -me files and realize that and load it properly as 396px × 593px (scaled to 396px × 594px)

 

#34 2014-11-03 09:30:06

JT
Guest

Re: PIWIGO crashes freezes servers

OK, I've fixed that issue.

 

#35 2014-11-03 09:32:33

flop25
Piwigo Team
2006-07-06
7037

Re: PIWIGO crashes freezes servers

you're the first reporting that!

about the servers, it certainly a common system config (os/apache|nginx/gd|im) which cause that with the additional thing that you're uploading very high res pictures on a web server.

ps: what was the issue?


To get a better help : Politeness like Hello-A link-Your past actions precisely described
Check my extensions : more than 30 available
who I am and what I do : http://fr.gravatar.com/flop25
My gallery : an illustration of how to integrate Piwigo in your website

Offline

 

#36 2014-11-03 09:34:23

JT
Guest

Re: PIWIGO crashes freezes servers

Oh, might as well explain in case may help someone else. The problem was that although I had re-run the "Sync" on 'Directories AND files' with Synchronise metadata (filesize, width, height, date_creation, latitude, longitude)
+  including already synchronised photos

well i had simply forgotten to UNCHECK
"perform a simulation only (nothing will be changed in the database)"

So it only did a 'simulation' without actually updating the database :D

 

#37 2014-11-03 09:36:10

flop25
Piwigo Team
2006-07-06
7037

Re: PIWIGO crashes freezes servers

^^ don't worry, it happens to all users at elast once


To get a better help : Politeness like Hello-A link-Your past actions precisely described
Check my extensions : more than 30 available
who I am and what I do : http://fr.gravatar.com/flop25
My gallery : an illustration of how to integrate Piwigo in your website

Offline

 

#38 2014-11-03 09:39:04

JT
Guest

Re: PIWIGO crashes freezes servers

Yea and on this slowest server its using basic setup of apache with GD:

        Piwigo 2.7.1
        Check for upgrade
Environment

        Operating system: Linux
        PHP: 5.4.16 (Show info) [2014-11-03 00:36:24]
        MySQL: 5.1.56 [2014-11-03 00:36:24]
        Graphics Library: GD 2.0.35


Anyway this is great now though, I have piwigo running on my NAS file share and works fine now that i found the i.php/BatchManager/RecentPhotos preview disable workaround etc. Its pretty robust even! :)

 

#39 2014-11-03 20:34:52

JT
Guest

Re: PIWIGO crashes freezes servers

Well I spoke too soon, just did more thorough check this morning and some images are still stretched, so it fixed some but not all. Maybe somehow it 'skips' updating metadata etc under certain circumstances?
Is there a way I can 'force' it to recreate what image 'Scales' to? basically the 396 x 593 portrait images, some of them still try to get scaled to landscape mode 792 x 528, I've even deleted "User cache" and "compiled templates" and some other things in maintenance but somehow piwigo still insists on stretching the M(-me) version of some of these portrait/vertical images. Any ideas?
Thanks!

JT wrote:

I did just notice ONE SMALL PROBLEM now:

For some of my portrait/vertical orientation images, the M(medium/default) size, some of them are stretched saying:

396px × 593px (scaled to 792px × 528px)

While other ones do display correctly:

396px × 593px (scaled to 396px × 594px)

So it seems for some of the piwigo somehow thinks its a landscape mode 792 x 528 image when in fact it is portrait/vertical. How can I have piwigo re-analzye the -me files and realize that and load it properly as 396px × 593px (scaled to 396px × 594px)

 

#40 2014-11-03 21:58:38

JT
Member
2014-11-03
16

Re: PIWIGO crashes freezes servers

OK, finally fixed the image stretching issue I described above.

Steps to Fix:

1. temporarily MOVE the ORIGINAL files from /galleries/mygallery/*.JPG  to /piwigo/TEMPdump
2. Run the Quick SYNC, it will say it removed all the files.
3. MOVE back all the photos from /piwigo/TEMPdump/*.JPG back to /galleris/mygallery/
4. Immediately Re-run the "Quick Sync" in Admin panel again. (default settings etc)

I was VERY thankful that running that SYNC after moving the ORIGINAL files in Step#1 did NOT delete all of my resized images (-sq, -me, -th, etc) inside /piwigo/_data/i/galleries/mygallery/ because that would of been a big bummer considering the hassle I went through explained in this thread to get those multiple-sized images created without overloading the server. So seeing those multiple size images still there was a huge relief! :)

I've tested all of the photos now and none of them are stretched anymore.

Fyi, the reason I had this problem I think is because i had manually CHANGED the ORIENTATION of some photos to vertical/portrait so that people didn't show up sideways etc, but I did this AFTER i had already added the photos using Admin Quick SYNC. So apparently the metadata synchronization does not re-evaluate the orientation of the pictures if you try to Re-Sync later, and piwigo still considered them landscape mode photos instead of portrait and ended up stretching them when viewing.

Perhaps this can be fixed/changed so that a Re-SYNC would have fixed this issue without requiring the Steps #1-4 above? Anyway now that I know better how piwigo 'thinks' I can avoid future issues.

Offline

 

#41 2014-11-03 22:15:10

flop25
Piwigo Team
2006-07-06
7037

Re: PIWIGO crashes freezes servers

Piwigo checks the last_modified time of the original picture. All your issues about the creation again of resized pictures might come to an hypothetical issue to get that time...


To get a better help : Politeness like Hello-A link-Your past actions precisely described
Check my extensions : more than 30 available
who I am and what I do : http://fr.gravatar.com/flop25
My gallery : an illustration of how to integrate Piwigo in your website

Offline

 

#42 2014-11-03 22:47:02

JT
Member
2014-11-03
16

Re: PIWIGO crashes freezes servers

So perhaps when I modified the original images to change their orientation, somehow their 'modified' times did not update properly, otherwise from what you're saying, piwigo would have updated and recognized the new portrait/vertical orientation. 

Interesting because I did change the orientation of the original images from my WINDOWS PC since they were 'shared' over samba/windows share from linux. But I just checked the date/time of the piwigo server and that of my Windows PC and they seem to match just about to the second.  But I guess its possible that the orientation change I made somehow didn't update the modified time.

--UPDATE--strange I just duplicated method I used to change orientation and I do notice that the Modified Time on the file does change, so I'm not 100% convinced that piwigo does indeed re-check file orientation again on re-sync, could still be some other issue but I checked date/time between my windows where I did orientation change and piwigo server and they matched and I just tried again on some other files and do notices Modified Time did change.

Offline

 

#43 2015-06-16 05:03:26

Nonek
Guest

Re: PIWIGO crashes freezes servers

I have been spending some time trying to get Piwigo up and running I thought this would be smooth sailing after reading how someone bragged that they were an expert after a few hours ;-)
I'm coming from gallery3 and I have had a few issues that I have had to work through..
I believe some of them stem from using the absolute latest versions of things.
MariaDB 10
Php 5.6.9
Apache/2.4.6 (CentOS) OpenSSL/1.0.1e-fips mod_fcgid/2.3.9 PHP/5.6.9 mod_wsgi/3.4 Python/2.7.5
GD Version  bundled (2.1.0 compatible) 
Anyway..
Uploading photos via the web is PAINFULLY slow..  I honestly didn't think it was completing until I uploaded 5 photos and passed out in my chair for 30 min...
I'm at a loss as to why this is doing this.
I also had the issue where it was talking about /e being deprecated in the gallery.  It was stated that you shouldn't be showing all errors as a default, but your php default config actually has that set..
Anyway.. I'm not as savy as the guy that this was happening too before, and I know you may or may not have many reports of this, but I'm thinking that there might be an issue with later versions of php and the gallery..
anybody have anything more on this?
Thanks

 

#44 2016-05-23 20:07:42

vitof
Guest

Re: PIWIGO crashes freezes servers

MAJOR KEY ALERT .....in admin panel.....in the maintenance section..........there is link click to Delete Image Sizes thumbs..............Don't do it..........on a gallery with 5321 images.........when it tries to recreate the image sizes afterwards............after visiting the batch manager page..........it brings the server completely down........many "convert" requests.......server is killed..........can't login ssh no more........since 40 minutes.......did reboot server twice from admin panel........still not responding......server is lost.......i don't know what to do..........take a beer..........

 

#45 2016-05-23 23:03:04

flop25
Piwigo Team
2006-07-06
7037

Re: PIWIGO crashes freezes servers

hard reboot


To get a better help : Politeness like Hello-A link-Your past actions precisely described
Check my extensions : more than 30 available
who I am and what I do : http://fr.gravatar.com/flop25
My gallery : an illustration of how to integrate Piwigo in your website

Offline

 

Board footer

Powered by FluxBB

github twitter newsletter Donate Piwigo.org © 2002-2024 · Contact