Announcement

#16 2019-10-11 21:17:31

kc27
Member
2014-02-25
75

Re: "Some checksums are missing." after upgrade to 2.10

erAck wrote:

plg wrote:

(missing "]" after "name=pwg_token".

Note that mistake occurs thrice, line 283, 436 and 486.

jQuery('#applyAction').click(function(e) "sync metadatas or delete photos by blocks", add_md5sum_block() and delete_orphans_block()

I made the change in the three places listed above. My webhost also increased the connection time-out. I was able to get another 5% of the checksums created, but it consumes 100% of my system resources according to the CPanel on my site:

2 GB of 2 GB of physical memory and 2 MB of 2 MB of I/O Usage.

My gallery is still functional, so I can just wait until this gets sorted out. Thanks for the help.

Offline

 

#17 2019-10-12 09:18:16

karolox
Member
2019-10-12
9

Re: "Some checksums are missing." after upgrade to 2.10

Hello
After upgrading to Piwigo 2.10, I am seeing a lot of missing image icons on the thumbnail pages.
For example here: http://hmyzslovenska.info/index.php?/category/798
Before the update I made the transition from menalto gallery.

I am a beginner. Help me please. My site http://www.hmyzslovenska.info
Thank you. Regards Karol

Last edited by karolox (2019-10-12 10:42:09)

Offline

 

#18 2019-10-12 10:56:31

plg
Piwigo Team
Nantes, France, Europe
2002-04-05
13520

Re: "Some checksums are missing." after upgrade to 2.10

@karolox your problem is not related to this topic (but to your photo filename having special chars in them), please open a dedicated topic :-)

Offline

 

#19 2019-10-12 14:06:17

karolox
Member
2019-10-12
9

Re: "Some checksums are missing." after upgrade to 2.10

plg wrote:

@karolox your problem is not related to this topic (but to your photo filename having special chars in them), please open a dedicated topic :-)

Thank you. I adjusted the file config_default.inc ($conf['sync_chars_regex'])...
It was rewritten after the update.
Regards Karol

Offline

 

#20 2019-10-12 14:14:28

erAck
Only trying to help
2015-09-06
1147

Re: "Some checksums are missing." after upgrade to 2.10

karolox wrote:

Thank you. I adjusted the file config_default.inc ($conf['sync_chars_regex'])...
It was rewritten after the update.

You should make config modifications in local/config/config.inc.php instead, which will not get overwritten each update.. either use any editor, or [extension by Piwigo Team] LocalFiles Editor


Running Piwigo at https://erack.net/gallery/

Offline

 

#21 2019-10-12 14:41:58

mphased
Member
2019-10-11
4

Re: "Some checksums are missing." after upgrade to 2.10

Thank you all for looking into this. I now managed to workaround the issue.

Short wrap-up what I did:

1. edited file "admin/themes/default/js/batchManagerGlobal.js" on my server and added the missing "]" on three occurances, described here: https://piwigo.org/forum/viewtopic.php?id=29631#p174580

2. turned off js/css combine (I finally found the setting/switch)

Now I was able to compute the missing checksums. Thanks again for your help!

Sidenote about the admin backend:
At first I didn't find the option to turn off "js/css combine" in the left sidebar (I looked in "Tools" -> "Maintenance" where I was expecting a setting like that). I never bevor clicked or noticed the upper right "Tools"-Link in the top-bar.
To me, the whole Piwigo admin is very left-sidebar-focused and having two "Tools" Links on a admin GUI might not be ideal (the one in the sidebar, the other one in the upper right top bar)
Long story short: maybe you should consider cleaning up functionalities a little bit. Without knowing all the reasons why things are the way they are, I would rename "Tools" in the top bar to "Debug" or something like that. Additionally, it wouldn't hurt, to add the "combine css/js" switch also under "siderbar" -> "Tools" -> "Maintenance" since the option to "purge compiled templates" is also possible in both locations. But that's just my opinion...

EDIT: it seems, that the setting "Tools" -> "combine js/css" in the topbar does not permanently disable the js/css-combination-feature. When I was logged out, my browser network tab showed me, that still the combined js file was loaded. So, to have a permanent effect, I hat do set "$conf['template_combine_files'] = false;" in "local/config/config.inc.php"
This is another good reason to rename that menu item from tools into debug :)

Last edited by mphased (2019-10-12 14:59:26)

Offline

 

#22 2019-10-13 21:30:17

windracer
Member
St. Pete, FL
2014-12-28
342

Re: "Some checksums are missing." after upgrade to 2.10

mphased wrote:

1. edited file "admin/themes/default/js/batchManagerGlobal.js" on my server and added the missing "]" on three occurances, described here: https://piwigo.org/forum/viewtopic.php?id=29631#p174580

2. turned off js/css combine (I finally found the setting/switch)

Now I was able to compute the missing checksums. Thanks again for your help!

Even after doing this, I'm still getting an "Error: 200" when trying to compute 18k checksums. All of my photos are FTP/local sync (no uploads) so I guess this doesn't apply so can I suppress the "some checksums are missing" message in the dashboard?

Offline

 

#23 2019-10-14 08:25:12

aberkvam
Member
2018-03-07
9

Re: "Some checksums are missing." after upgrade to 2.10

kc27 wrote:

My webhost also increased the connection time-out. I was able to get another 5% of the checksums created, but it consumes 100% of my system resources according to the CPanel on my site:

2 GB of 2 GB of physical memory and 2 MB of 2 MB of I/O Usage.

Creating the checksums is very disk and processor intensive, so seeing 100% usage of system resources isn't surprising.  I suspect you would be able to refresh the page and try again, get another 5% done, refresh the page, etc.  Otherwise, maybe there's a Piwigo setting that would limit how many checksums it tries to compute at once?  (Seems to be 5 on my host, if I recall correctly.)

Offline

 

#24 2019-10-14 09:56:08

plg
Piwigo Team
Nantes, France, Europe
2002-04-05
13520

Re: "Some checksums are missing." after upgrade to 2.10

windracer wrote:

Even after doing this, I'm still getting an "Error: 200" when trying to compute 18k checksums.

Error "200" is very unexpected. 200 is the HTTP status code to say "it's fine" :-/

windracer wrote:

All of my photos are FTP/local sync (no uploads) so I guess this doesn't apply so can I suppress the "some checksums are missing" message in the dashboard?

Hum... The "some checksums are missing" is only for users who have FTP/local sync.

Offline

 

#25 2019-10-14 10:12:11

plg
Piwigo Team
Nantes, France, Europe
2002-04-05
13520

Re: "Some checksums are missing." after upgrade to 2.10

aberkvam wrote:

Creating the checksums is very disk and processor intensive, so seeing 100% usage of system resources isn't surprising.

It surprises me. md5 algorithm was designed to be fast at computing a binary signature for your file.

aberkvam wrote:

I suspect you would be able to refresh the page and try again, get another 5% done, refresh the page, etc.

It's exactly why we added a progressive processing : to avoid reloading the page.

aberkvam wrote:

Otherwise, maybe there's a Piwigo setting that would limit how many checksums it tries to compute at once?  (Seems to be 5 on my host, if I recall correctly.)

No, it's currently (Piwigo 2.10.0) hardcoded : the smallest value between "half the set size" or 1000. If you have 15k missing md5, it will process 1k photos at once. If you have 600 missing md5, it will process 300 at once.

You can change this value in file admin/themes/default/js/batchManagerGlobal.js line 423 :

Code:

421  var addBlockSize = Math.min(
422    Number((jQuery('#md5sum_to_add').data('origin') / 2).toFixed()),
423    1000
424  );

Replace "1000" by 10, and see if it works any better.

Offline

 

#26 2019-10-14 11:14:49

plg
Piwigo Team
Nantes, France, Europe
2002-04-05
13520

Re: "Some checksums are missing." after upgrade to 2.10

Actually, there is bug in Piwigo javascript. The blocksize is only used for the first block, then it's 1000 whatever the size of your photo set.

In file ws.php line 505, you can just replace:

Code:

        'block_size' => array('default'=>1000, 'type'=>WS_TYPE_INT|WS_TYPE_POSITIVE),

by

Code:

        'block_size' => array('default'=>10, 'type'=>WS_TYPE_INT|WS_TYPE_POSITIVE),

Offline

 

#27 2019-10-14 11:21:46

plg
Piwigo Team
Nantes, France, Europe
2002-04-05
13520

Re: "Some checksums are missing." after upgrade to 2.10

I just made a test with big photo file (15MB) and computing 1000 md5 signatures takes ~15 seconds, which is quite high actually.

TODO:

* reduce the default block size, to 50
* fix bug to keep using the same block size, whatever the iteration

Offline

 

#28 2019-10-14 23:42:34

windracer
Member
St. Pete, FL
2014-12-28
342

Re: "Some checksums are missing." after upgrade to 2.10

After I upgraded to 2.10.1, the "some checksums are missing" message is gone from my dashboard. Thanks!

Offline

 

#29 2019-10-15 09:25:09

plg
Piwigo Team
Nantes, France, Europe
2002-04-05
13520

Re: "Some checksums are missing." after upgrade to 2.10

windracer wrote:

After I upgraded to 2.10.1, the "some checksums are missing" message is gone from my dashboard. Thanks!

you will find it again on page [Administration > Tools > Synchronize] ;-)

Offline

 

#30 2019-10-16 03:26:03

windracer
Member
St. Pete, FL
2014-12-28
342

Re: "Some checksums are missing." after upgrade to 2.10

plg wrote:

windracer wrote:

After I upgraded to 2.10.1, the "some checksums are missing" message is gone from my dashboard. Thanks!

you will find it again on page [Administration > Tools > Synchronize] ;-)

Yep, actually I just noticed that. So that's expected?

Offline

 

Board footer

Powered by FluxBB

github twitter newsletter Donate Piwigo.org © 2002-2021 · Contact