Announcement

#1 2019-01-04 06:58:27

PolyWogg
Member
2014-08-17
72

Moving a Piwigo install between servers

I'm running the latest Piwigo install, single gallery, and it is virtually hosted on a site that mainly has a WordPress install in the root. It's a basic site, everything works -- but the site has a limit of 200,000 nodes in total. On my little basic install, I was running three separate WP installs plus Piwigo. The gallery has 12000 pics in it.

So I'm thinking of opening second account and sharing the load between the servers. Which would mean moving Piwigo to another server. Everything is working PERFECTLY, I don't really want to touch it, but if I could get it up and running on the other server before deleting the first, I'd feel a bit safer about it. I really don't want to lose all my filenames and custom descriptions.

What's the best way to move a PIWIGO install from one server to another? At least it's within the same company...

I doubt I'm the first to do this, but my searching turned up very little. I tried migration but that was giving me from other software packs to Piwigo; I tried transfer, but that seemed to be more about albums; move was about albums or more likely photos.

Paul
aka PolyWogg

Offline

 

#2 2019-01-04 20:55:13

executive
Member
2017-08-16
1214

Re: Moving a Piwigo install between servers

Offline

 

#3 2019-01-05 03:16:55

PolyWogg
Member
2014-08-17
72

Re: Moving a Piwigo install between servers

Thanks executive...I don't know what I was searching for, but I never found any of those when I tried. And I tried multiple variations of move, moving, hosting, server, gallery. Sigh.

The second one seems to be a good quick overview. I'm not sure there aren't gremlins hiding in there :)

Paul
aka PolyWogg

Offline

 

#4 2019-01-05 06:38:05

executive
Member
2017-08-16
1214

Re: Moving a Piwigo install between servers

Hopefully not, but you'll still have your backup. Come back if you run into trouble and we'll sort you out.

Offline

 

#5 2019-01-05 08:02:03

PolyWogg
Member
2014-08-17
72

Re: Moving a Piwigo install between servers

Okay, looking into it, I seem to have two issues...one that might kill me and one that might save me.

First, I might have fubared myself awhile ago. I have a virtual host package, and I run WordPress in my root at PolyWogg.ca

The Piwigo install is one level down at https://polywogg.ca/pandafamily/.

I have a ton of stuff in the WP site and elsewhere that directly links to that URL structure such as an image at:

https://polywogg.ca/pandafamily/_data/i … 94-la.jpg.

If I move it to another domain, call it "PolyWogg2.ca", then all my existing URLs will be off, won't they? Even if I can copy the database over, move all the files, and get everything up and running in the new site, it won't be PolyWogg.ca in the domain root, it will be PolyWogg2.ca.

Maybe I can reroute any links from polywogg.ca/pandafamily/blah blah blah over to the PolyWogg2 address. Hmm...

Did that setup just screw me up? Should I have installed in root originally?


Second, something seems off with my install, and I wonder if it is the variety of images that are included/generated/resized. The problem I'm having with my host is that it only allows me 200,000 inodes (files and folders). That seems like a lot, and nobody ever goes near it. My main WP install with over 1000 posts has about 20K inodes.

Piwigo, by contrast, is showing 103,490 Files, 1,623 Folders. But I only have 13K photos. If I ignore say 10K for overhead, the rest is about 6-7 times the number of photos I have. Something I never really thought about, but now I wonder if I have it generating sub sizes I don't need...

In my CONFIG / Photo Sizes, it has Square, Thumbnail and Large set as automatic. After that, it also has checked:

-- XXS - tiny        
-- S - small        
-- M - medium        
-- XXL - huge        

Do I need all those? If I cut some of them, would it reduce dramatically the number of files I'm creating or does it manage those other sizes on the fly? Is there also a list somewhere that says where those sizes are shown by default?

Paul

On a separate note, I have about 13K photos in the site but it tells me that Piwigo has

Offline

 

#6 2019-01-05 21:39:58

executive
Member
2017-08-16
1214

Re: Moving a Piwigo install between servers

PolyWogg wrote:

If I move it to another domain, call it "PolyWogg2.ca", then all my existing URLs will be off, won't they?

Yes. I can't think of an easy solution.
Can you keep the old Piwigo installation, and delete all but the images used in old Wordpress posts. Since they're symlinked they won't use any more space right.


PolyWogg wrote:

Do I need all those?

It's your choice.

PolyWogg wrote:

If I cut some of them, would it reduce dramatically the number of files

Probably, but you're not at your limit right.

PolyWogg wrote:

Is there also a list somewhere that says where those sizes are shown by default?

Not sure I understand. Each photo page has a dropdown menu with the sizes.

Last edited by executive (2019-01-05 21:42:32)

Offline

 

#7 2019-01-05 21:58:29

PolyWogg
Member
2014-08-17
72

Re: Moving a Piwigo install between servers

Thanks Executive...I figured it out.

Basically I was asking for the sizes:

- Square        
- Thumbnail        
- XXS - tiny        
- XS - extra small        
- S - small        
- M - medium        
- L - large        
- XL - extra large        
- XXL - huge

Which ones do I *have* to have without breaking my gallery (i.e. where photos wouldn't load on some page). Square, Thumbnail, and Large are required and are pre-selected. But I was wondering if I got rid of XXS, XS, S, M, XL and XXL, would that mean some page wouldn't load i.e. if the admin page needed thumbnails and thumbnails weren't there, obviously that wouldn't load. Hence, I assume, why thumbnails is always required. I'm using Bootstrap Darkroom, and my default on the page is Large, so I was wondering if the others were required. Short answer -- apparently not. I went in, deselected them, pulled up custom settings, and in the explanation of sizes, it showed a whole list of choices and the last time any of them had "hits". Almost all of them showed at least 11 months since they were last viewed for any of the sizes. I will monitor for the future too, there's a couple of weird sizes listed in there that I might be able to get rid of too, but I dumped them.

My 150,000 inodes just dropped to 75,000 overall. I'm nowhere near the limit now!

Thanks for your help...I wouldn't have figured that out without having someone to bounce my thinking off of!

Paul

Offline

 

#8 2019-01-07 09:51:29

eliz82
Member
Romania
2016-04-27
281

Re: Moving a Piwigo install between servers

PolyWogg wrote:

I have a ton of stuff in the WP site and elsewhere that directly links to that URL structure such as an image at:

https://polywogg.ca/pandafamily/_data/i … 94-la.jpg.

If I move it to another domain, call it "PolyWogg2.ca", then all my existing URLs will be off, won't they? Even if I can copy the database over, move all the files, and get everything up and running in the new site, it won't be PolyWogg.ca in the domain root, it will be PolyWogg2.ca.

Maybe I can reroute any links from polywogg.ca/pandafamily/blah blah blah over to the PolyWogg2 address. Hmm...

Moving a site from an old-domain to new-domain and updating the links is basic webmaster knowledge.

1) If your images from Piwigo are linked into Wordpress and the links are absolute (not relative) then you indeed will have problems. Just use a tool that search and replace the data in the Wp database (save as SQL and open in Notepadd++) , or more safe using a specialized plugin (as this ones can also have specialized algorithms that can handle serialized data)
You will find many articles about this, ex. https://blog.templatetoaster.com/update … -database/

2) Extra:
if your old domain is indexed by google for example, or you have links to your old-domain on external sites, you can keep this old domain for a few months and make an webserver 301 (permanent) redirect from the old domain to the new domain.
https://www.google.com/search?q=htacces … new+domain

p.s.  if you make 301 redirect tests make sure you clean your browser cache after each modification to the htaccess file, because the browser will cache the redirect rule.

Offline

 

#9 2019-01-07 19:56:32

PolyWogg
Member
2014-08-17
72

Re: Moving a Piwigo install between servers

Thanks Eliz82...I found a partial way around it...my pressure to do "something" was having too many files. THen I realized that waaaay back when I set up the site, I had chosen too many variations of the photo sizes so it was generating way too many versions of the same file. I reduced and that problem is solved.

Now I have a NEW problem. Apparently when I linked, and they are rather "hardcoded" links from the URL to the blog post, the URL I got was not the default LARGE size but something else. So when I deleted all those other sizes, apparently the linked image went with it. Which means I either have to find the size that disappeared (perhaps Medium) and recreate it, or relink all those photos. Stupid stupid stupid.

PolyWogg

Offline

 

#10 2019-01-08 10:23:58

eliz82
Member
Romania
2016-04-27
281

Re: Moving a Piwigo install between servers

PolyWogg wrote:

Thanks Eliz82...I found a partial way around it...my pressure to do "something" was having too many files. THen I realized that waaaay back when I set up the site, I had chosen too many variations of the photo sizes so it was generating way too many versions of the same file. I reduced and that problem is solved.

i have written a post about that, how to minimize the number of derivatives https://piwigo.org/forum/viewtopic.php?id=28180

PolyWogg wrote:

Now I have a NEW problem. Apparently when I linked, and they are rather "hardcoded" links from the URL to the blog post, the URL I got was not the default LARGE size but something else. So when I deleted all those other sizes, apparently the linked image went with it. Which means I either have to find the size that disappeared (perhaps Medium) and recreate it, or relink all those photos. Stupid stupid stupid.

you could try the same method that i described in previous post. export your Wp database as SQL and open in Notepad++ . Search for "-la.jpg" replace with "-me.jpg"  or with ".jpg" if you want original.

You need to study your gallery files in "_data/i/" or the api output to see what suffixes (to filename) you used before and what suffix you want now
example, http://piwigo.org/demo/ws.php?format=re … .getImages

Offline

 

#11 2019-01-08 18:33:01

PolyWogg
Member
2014-08-17
72

Re: Moving a Piwigo install between servers

Thanks Eliz82,

That link (derivatives) is exactly what was going on with mine.

But now that I understand that better, I'm not sure if my new problem with missing photos embedded in my site IS what I thought it was. Here's my example...I did a blog entry about my honeymoon in Hawaii, with a post about each day with photos of what we saw. Here's the link:

http://polywogg.ca/honeymoon-recap-hawaii-day-5/

It used to work fine, all the photos showed. I wrote it three years ago, it's been fine as far as I know until recently. But now when you load it, you'll see that the first picture loads fine with the link:

https://polywogg.ca/pandafamily/i.php?/ … 3c4-la.jpg

If I understood your derivative post, the "key" part of that filename is the "-la" extension telling me that it is indeed using the LARGE default setting (I keep thumbnail, square and large by default). And it loads fine, which it should.

However, the next image (and all the rest) do NOT load:

https://polywogg.ca/pandafamily/_data/i … b6e-la.jpg

I assumed at first that it was going to have a -ME extension and since I had deleted medium, that was the problem. But no, it too is looking for the large. Except it doesn't load while the first one does.

Now, I haven't fixed the page, so I can see the problem still, but I had the same problem on other pages, and I "fixed" those ones, but I'm not sure how my fix did it. Here's what I did:

a. Opened up the original album in Piwigo
b. Browsed to the single photo page
c. Paged through to the end to see each photo was there and loading
d. Closed out
e. Reloaded the WP page, and all the images loaded.

But I don't know "why" that fixed it...I have two ideas as possible explanations but not sure which is more likely.

1. Does it mean that the gallery doesn't actually have the LARGE ones generated anymore? And by going to each page in the album where LARGE is shown, it IS generating it so now the link works?

2. Or, as my host just instituted a change for something, is it likely to be a first-time caching problem? My host just instituted LiteCache across all the servers. And I likely haven't viewed those pages since caching was rolled out. But that shouldn't matter -- most caching will try to load from the cache, but if it doesn't find it, it loads the original. It doesn't look in the cache and just stop. But, equally, if I browsed through the album, page by page, it would generate it for the cache too. I don't think it can be the caching problem, as it has been installed for almost a month now, and I still had the problem on a page I just created the other day (so the photos would have been in the cache in theory).

Any guesses as to which of the two are likely? Or, I guess more pointedly since you can't guess at my hosts caching protocol, is #1 likely?

PolyWogg

Offline

 

#12 2019-01-08 20:24:23

eliz82
Member
Romania
2016-04-27
281

Re: Moving a Piwigo install between servers

Yes, it seems you are using only the "Large" derivatives in your wordpress posts, because all your photos have "-la.jpg" suffix.

The reason why this appear is because you access the file using "i.php", so the image is generated dynamically
https://polywogg.ca/pandafamily/i.php?/ … 3c4-la.jpg

While this one is just a link to a static image, that do not exist anymore (has been deleted)
https://polywogg.ca/pandafamily/_data/i … b6e-la.jpg

When Piwigo api is interrogated
https://polywogg.ca/pandafamily/ws.php? … .getImages
it will generate static links for the images that have derivatives already build, while it uses "i.php" for the derivatives that where never accessed by a visitor (in Piwigo gallery). That why you first link in the Wordpress is to "i.php" while the rest are direct links to images.

After that you have probably made some changes to Piwigo images derivatives (maybe you have deactivated some, or changed resolution). After that change those "Large" derivatives has been deleted by Piwigo? (never tested this).

You could try to go to your Administration => Photos = Batch manager => Generate multiples sizes => select "Large", so you generate the large derivatives manually.

Last edited by eliz82 (2019-01-09 09:05:32)

Offline

 

#13 2019-01-08 22:35:07

PolyWogg
Member
2014-08-17
72

Re: Moving a Piwigo install between servers

Thanks Eliz82...generation under way! Only 8K more images to go :)

Offline

 

#14 2019-01-09 04:44:24

PolyWogg
Member
2014-08-17
72

Re: Moving a Piwigo install between servers

Okay, making progress. Although it is like 3 steps forward, and 2 back. I regenerated all the LA size ones (it threw me an error on three of them, not sure what that is about) and considerably less than the full 12K (only about 9K according to the update, but maybe 3K were already done).

So I have these three as default:

- Square             120 x 120 pixels, crop
- Thumbnail      144 x 144 pixels
- L - large       1008 x 756 pixels

I am NOT using these in the default:

XXS - tiny     240 x 240 pixels     edit
XS - extra small     432 x 324 pixels     edit
S - small     576 x 432 pixels     edit
M - medium     792 x 594 pixels     edit
XL - extra large     1224 x 918 pixels     edit
XXL - huge     1656 x 1242 pixels     edit

But I still have the following sizes showing under custom:

Custom
Delete e520x360 (Last hit: today)
Delete e1008x756 (Last hit: today)
Delete e120 (Last hit: 2 days ago)
Delete e240 (Last hit: 4 days ago)

Do you think I can delete these? Not sure why they are showing a hit for today?

For PLUGINS, I only see a few dealing with photos that could remotely create the derivatives you mentioned. EASYROTATE but that is only 1 pic at a time when used; FACEBOOK PLUG but I don't think it creates images or crops, just shares; MANAGE PROPERTIES, PHOTOS but again, not touching the photo itself; ROTATEIMAGE, similar to before; and RV THUMB SCROLLER. Again, I don't see anything in there creating extras.

I did fix my WP problem (global search and replace) -- it didn't quite capture everything, but got me from > 1000 to less than 25 to fix (some -me.jpgs didn't replace, and others were links to the original files with no -ME or -LA version for some reason).

Very close to done. Although when I generated the LAs for all of them, it took me back up a bit in iNodes. Still less than 50% of where I was though.

Paul

Offline

 

#15 2019-01-09 09:30:05

eliz82
Member
Romania
2016-04-27
281

Re: Moving a Piwigo install between servers

PolyWogg wrote:

But I still have the following sizes showing under custom:
Custom
Delete e520x360 (Last hit: today)
Delete e1008x756 (Last hit: today)
Delete e120 (Last hit: 2 days ago)
Delete e240 (Last hit: 4 days ago)

Do you think I can delete these? Not sure why they are showing a hit for today?

No idea what is with that custom sizes, they are probably made by some plugins or themes. Not think is a good idea to delete them unless you know what you are doing.
... or maybe delete them and then deactivate and reactivate all your plugins and themes, and they will be probably be regenerated.

PolyWogg wrote:

and others were links to the original files with no -ME or -LA version for some reason).

From I remember when I worked with the Api, when original file resolution is less the the one set in medium, then the Api response will show medium derivative as a link to the original image (instead of building a new image with "-me.jpg" suffix. And this is normal as the images are not upsized to build derivatives, only downsized.

PolyWogg wrote:

Very close to done. Although when I generated the LAs for all of them, it took me back up a bit in iNodes. Still less than 50% of where I was though.

I think Piwigo should not come with all derivatives enabled by default. Like I said in that post I linked, there are many hosts that put a limit on iNodes. Only Piwigo by itself will generate by default 9 derivatives for every original image, if you also install Thumb and Piwigo mobile theme you will end up with 11 derivatives. If your host have a limit of 100.000 iNodes you can upload ~8000 pictures and you hit the limit.

Offline

 

Board footer

Powered by FluxBB

github twitter newsletter Donate Piwigo.org © 2002-2024 · Contact