Piwigo.org

You are not logged in. (Register / Login)

Announcement

Post a reply

Write your message and submit

Click in the dark area of the image to send your post.

Go back

Topic review (newest first)

LucMorizur
2011-05-15 15:04:11

stim wrote:

That 0.0001% of users can probably change the code themselves, optimizing for their back-end.

For any Piwigo user that wants a nice management of case and accents on usernames (for a very large amount of unicode characters), and also who would like to implement a personal version of this management (to be coded "soon" :-/ ), without needing any specific PHP extension, Éric Whiler and me have developed extension:513 ;-) ! It has been tested on a gallery with more than 2000 users registered, and no performance issue was to be noticed for visitors.

(Yet I must admit that the code is... very basic :-/ . But it works, nothing else to say :-)) !)

stim
2011-05-15 13:49:30

With those figures (0.01% * 0.001%) it seems indeed that there is little need to optimize this. However, it still doesn't feel right, we just know it's gonna be an issue some day...

On the other hand, using LOWER eliminates the use of indexes by the database, so that solution will probably also run into performance issues at some point.

Furthermore:

plg wrote:

If you're sure the SQL function "LOWER" is 100% compatible with MySQL/PostgreSQL/SQLite then it's fine, let's go with it, but be sure :-)

I'm not sure, I just noticed that the function is supported in all three systems.
But a little search showed me this: sqlite issues. The same problems that strtolower has.

I guess I should settle with the current commit. That 0.0001% of users can probably change the code themselves, optimizing for their back-end.
(Is it an option to wrap the search_case_username function in if(!is_callable('search_case_username')){[...]} such that it can easily be overridden?)

plg
2011-05-14 22:25:23

As usual, my opinion is to "keep it simple".

By default the authentication is case sensitive, and this can be change only in the configuration file, not in the web administration interface, so let's say only 0.01% of Piwigo users will change it (and it's a big number in my opinion)

Then starting from how many users will the "retrieve all before compare" solution will start to generate performance issues? 10,000 users ? OK, that's 0.001% of Piwigo users (the vast majority of Piwigo installation has between 10 and 100 users I think).

So please try to keep things simple. One day someone will have to maintain this code, he will be grateful if you keep things clear and easy.


If you're sure the SQL function "LOWER" is 100% compatible with MySQL/PostgreSQL/SQLite then it's fine, let's go with it, but be sure :-)

plg
2011-05-14 22:18:34

scalable means that the script won't crash when jumping from 10 to 10,000,000 users in the database.

flop25
2011-05-13 21:52:23

I think it was thinking of scalable "adaptable à tout les cas", not "évolutif". Isn't it ?

LucMorizur
2011-05-13 21:49:39

stim wrote:

I mean to say that I strongly disagree with this solution because it is not scalable at all. Therefore, I don't like to settle with the last commit. Because: if it is not scalable, it's not a actually a feature of the software, since (power)users will be obliged to turn it off eventually, as their traffic grows.

I think everybody agrees that the best code should be prefered :-) .

Just for my personal knowledge: what do you mean by "it is not scalable at all"? (I'm not a programmer, but I'm working on it ;-) !)

flop25
2011-05-13 21:22:30

thx for the doc. I read and didn't see any problem -except one for PostgreSQL but it's 10 years old ...-

I'll do some test with mysql and special character with LOWER.

stim
2011-05-13 20:38:14

Lower in sqlite:
http://www.sqlite.org/lang_corefunc.html
mysql
http://dev.mysql.com/doc/refman/5.0/en/ … tion_lower
postgres:
http://developer.postgresql.org/pgdocs/ … tring.html
General:
http://oreilly.com/catalog/sqlnut/chapt … html#27402

I mean to say that I strongly disagree with this solution because it is not scalable at all. Therefore, I don't like to settle with the last commit. Because: if it is not scalable, it's not a actually a feature of the software, since (power)users will be obliged to turn it off eventually, as their traffic grows(*).


I can understand that it might be undesirable to rely on 'LOWER' (if there exists considerable doubt about its performance). However, that shouldn't be an excuse not to make piwigo scalable.
Therefore I (jokingly) suggested to apply the 'backward' method I shared earlier. (Since, although it is backward in every sense of the word, it is scalable and can solve the issue.)

(*) I don't want to be harsh or anything. And I do see how this might never become an issue for the average user. But it's just bad quality of software to leave it unscalable like this, and I don't believe it's something that can't be solved.
I was very happy to find a community/album that fitted my needs (I've looked for it several times but never found anything good), and I'd like to continue to be of good quality if I'm going to use it. :)

LucMorizur
2011-05-13 19:53:01

stim wrote:

Lower is actually supported in all three.

flop25 wrote:

it's too rare !

You just made sure of that. With this performance issue, I guess no-one who notices it will use this.

I'm almost inclined to go for the backward way, if LOWER is out of the question. :-S
But this commit just takes away all scalability. If it is really that rare that this performance decrease is to be permitted, I don't believe it should be configurable.

-Tim

I don't catch everything you mean, and what you intend to do if the code stays as on last commit.

@both: I'm not skilled enough in DBs to be able to answer about the LOWER() function in the different DB languages. Can you both give links here showing the reason why you say the LOWER() function can/cannot be used?

Anyway what I can say, is that personally speaking, I would trust the code committed by flop25. I don't beleive it would generate issues.

But I'm OK to change my mind, if it is necessary :-) !

stim
2011-05-13 16:28:32

Lower is actually supported in all three.

flop25 wrote:

it's too rare !

You just made sure of that. With this performance issue, I guess no-one who notices it will use this.

I'm almost inclined to go for the backward way, if LOWER is out of the question. :-S
But this commit just takes away all scalability. If it is really that rare that this performance decrease is to be permitted, I don't believe it should be configurable.

-Tim

flop25
2011-05-13 15:39:17

@all : we can't use LOWEr because of the compatibility between the type data base : Postgresql/MySQL/SQLite
Moreover it's a feature which is false by default, and concern gallery with a huge amount of users : it's too rare !

So the first initial commit will stay

LucMorizur
2011-05-13 14:19:32

flop25 wrote:

@luc that's why using LOWER only is better because it's multi-byte safe.

OK, perfect, then :-)) .

flop25
2011-05-13 14:16:39

@luc that's why using LOWER only is better because it's multi-byte safe.

LucMorizur
2011-05-13 14:09:35

stim wrote:

I updated my code in [Bugtracker] bug 1835, there was another thing with count() and pwg_db_fetch_assoc($q) I messed up.

Yes, it seems better this way :-) .

stim wrote:

Yes, I rely on LOWER and str_to_lower to do the same thing. I don't know if they do...
But we can make sure to rely on LOWER only:

Code:

  $q = pwg_query("
    SELECT `".$conf['user_fields']['username']."` AS username
    FROM `".USERS_TABLE."`
    WHERE LOWER(`".$conf['user_fields']['username']."`) = LOWER(\"".pwg_db_real_escape_string($username)."\");
  ");

This one seems fine to me.

stim wrote:

The other option is to do it backwards ;).
Query all the users that match the hashed password, and go look for case ambiguities there... Then you can rely on str_to_lower only. But that would freak every-bodies mind out.

Yes :-D ! I prefer previous solution -- but we'll see what will be chosen at the end; I'm not part of the team.

flop25 wrote:

the problem is for special uppercase

Indeed! That's one of the reason for extension:513 to exist.

flop25 wrote:

the best solution - use mb_strtolower()

It does be quite a solution; yet mb_strtolower() is provided by PHP extension mbstring, thus is there a risk that on some servers this extension is not active. In this case mb_strtolower() generates a notice. For LCAS Whiler proposed a code allowing to get free from mbstring:

Code:

/**
 * Split a multi-byte string in a array of each of its characters.
 * Many thanks to Whiler ;-) in
 * http://fr.piwigo.org/forum/viewtopic.php?pid=162907#p162907
 *
 * @param string s: string to split
 *
 * @return : array of each character of s
*/
function LCAS_mbStringToArray($s) {
  return preg_split('//u', $s, -1, PREG_SPLIT_NO_EMPTY);
}
stim
2011-05-13 14:08:50

So, bringing it together (using LOWER) it would become:

Code:

/**
 * For test on username case sensitivity
 *
 * @param : $username typed in by user for identification
 *
 * @return : $username found in database
 *
 */
function search_case_username($username)
{
  global $conf;

  $q = pwg_query("
    SELECT `".$conf['user_fields']['username']."` AS username
    FROM `".USERS_TABLE."`
    WHERE LOWER(`".$conf['user_fields']['username']."`) = LOWER(\"".pwg_db_real_escape_string($username)."\");
  ");

  if(pwg_db_num_rows($q) == 1)
  {
    $row = pwg_db_fetch_assoc($q);
    return $row['username'];
  }else{
    return $username;
  }
}

Board footer

Powered by FluxBB

About this website · Donate · Contact Piwigo project © 2002-2013