Announcement

  •  » Engine
  •  » Case sensitivity

#1 2010-06-19 11:02:31

LucMorizur
Member
Vienne (Isère) - France
2009-04-30
171

Case sensitivity

Hi dear Piwigo developers ;

as discussed (in french) in [French forum, post 144772 by clemouel in topic 12775] [Plugin] nbc UserAdvManager and [French forum, topic 17946] Sensibilité à la casse dans "le core", since release 2.1 of Piwigo, one can now find the configuration parameter $conf['insensitive_case_logon'] in config_default.inc.php. To my personal opinion, and to the opinion of clemouel who opened the discussion, there is a problem due to the fact that when $conf['insensitive_case_logon'] is set to true, only the creation of usernames is case insensitive, which does not exactly reproduce the old same option of [extension by Eric] UserAdvManager ("NBC UAM"). ie :

Before, with option "Usernames : Case sensitivity" of NBC UAM :
_ option set to "Inactive (default)" :
  _ username creation of "Beatrice" -> OK.
  _ username creation of "beatrice" -> OK.
  _ identification attempt by visitor with username "bEatrice" -> denied, this account doesn't exist.

_ option set to "Active" : (-> makes Piwigo case insensitive) :
  _ username creation of "Beatrice" -> OK.
  _ username creation of "beatrice" -> denied, this account already exists.
  _ identification attempt by visitor with username "bEatrice" -> OK.

Now (no more option "Case insensitivity" in NBC UAM, everything in Piwigo's core) :
_ $conf['insensitive_case_logon'] = false (default) :
  _ username creation of "Beatrice" -> OK.
  _ username creation of "beatrice" -> OK.
  _ identification attempt by visitor with username "bEatrice" -> denied, this account doesn't exist.

_ $conf['insensitive_case_logon'] = true :
  _ username creation of "Beatrice" -> OK.
  _ username creation of "beatrice" -> denied, this account already exists.
  _ identification attempt by visitor with username "bEatrice" -> denied, this account doesn't exist.

---------------------------------------

This last case is a pity we think.

What's more, accents are always sensitive.

I didn't open a bug yet, as I don't know if this way to do is wanted or not.

Thus I have the project to integrate a procedure allowing to make Piwigo insensitive to case and/or accents, up to the webmaster, with four cases of $conf['insensitive_case_logon'] (1, 2, 3, or any other value). This procedure can be tested here : http://lucmorizur.free.fr/test2_en.php . To see what I wrote, you can download the files : http://lucmorizur.free.fr/test2.zip .

Waiting for your opinion :-) .

Last edited by LucMorizur (2010-06-19 11:30:21)


Our gallery : Le Site à Nous (a silly name in french, but here I don't care ;-) ! )
An event, a new gallery ? Plugin Event Cats
My test gallery : Tests Piwigo de Luc
Thanksalot for this beautiful project.

Offline

 

#2 2010-06-19 14:09:15

LucMorizur
Member
Vienne (Isère) - France
2009-04-30
171

Re: Case sensitivity

Just forgot to say that in this procedure I used $lang, so that it can be completed for future languages, and/or overloaded for a different functioning than the one finally chosen.

But I must say that the three arrays are 530 lines long (16 kb), maybe it's not the best way to do :-/ ...


Our gallery : Le Site à Nous (a silly name in french, but here I don't care ;-) ! )
An event, a new gallery ? Plugin Event Cats
My test gallery : Tests Piwigo de Luc
Thanksalot for this beautiful project.

Offline

 

#3 2010-06-19 21:53:52

plg
Piwigo Team
Nantes, France, Europe
2002-04-05
13786

Re: Case sensitivity

OK, so there is a bug at login time. Of course, it needs to be fixed.

Now about extending the case sensitivity to accents, I reply "this is going to be very complicated" because we have to think for all characters supported by UTF-8.

We have already encountered such a difficulty for tag names. A tag has a name and a url_name. the url_name is used in URLs and is a simplified version of the name : without accents. For exemple "béatrice" will become "beatrice". The problem was for russian or chinese tags : url_name was always empty, and it was making the tags feature impossible to use. The solution I took was to say : if the result of the create_url_name($name) was empty then $url_name = $name.

Adding the "accent removal" in Piwigo core doesn't sound like a great idea to me. It's a good idea in a plugin, but not in Piwigo core (not yet at least)

By the way, this configuration parameter should have been name $conf['case_sensitive_username'] (to avoid negative boolean variables... it's much easier to read $conf['show'] = true than $conf['hide'] = false)

Offline

 

#4 2010-08-31 23:21:02

LucMorizur
Member
Vienne (Isère) - France
2009-04-30
171

Re: Case sensitivity

plg wrote:

OK, so there is a bug at login time. Of course, it needs to be fixed.

Sorry... I add this bug quite late :-/ ...

[Bugtracker] ticket 1835

plg wrote:

Now about extending the case sensitivity to accents, I reply "this is going to be very complicated" because we have to think for all characters supported by UTF-8.

We have already encountered such a difficulty for tag names. A tag has a name and a url_name. the url_name is used in URLs and is a simplified version of the name : without accents. For exemple "béatrice" will become "beatrice". The problem was for russian or chinese tags : url_name was always empty, and it was making the tags feature impossible to use. The solution I took was to say : if the result of the create_url_name($name) was empty then $url_name = $name.

Well the track I had started to follow, was to use $lang so that it could be appended by the different users -- of different languages of course.

plg wrote:

Adding the "accent removal" in Piwigo core doesn't sound like a great idea to me. It's a good idea in a plugin, but not in Piwigo core (not yet at least)

I intend to make a plugin for that, as soon as I have finished version 2 of [extension by LucMorizur] Event Cats.

That is, something like in six or twelve months :-/ ...

plg wrote:

By the way, this configuration parameter should have been name $conf['case_sensitive_username'] (to avoid negative boolean variables... it's much easier to read $conf['show'] = true than $conf['hide'] = false)

+1 : [Bugtracker] ticket 1836

! I put "$conf['case_sensitive_username']" in the summary instead of "$conf['insensitive_case_logon']" |-( ... if anyone can edit the summary, many thanks !


Our gallery : Le Site à Nous (a silly name in french, but here I don't care ;-) ! )
An event, a new gallery ? Plugin Event Cats
My test gallery : Tests Piwigo de Luc
Thanksalot for this beautiful project.

Offline

 

#5 2010-12-19 18:45:06

LucMorizur
Member
Vienne (Isère) - France
2009-04-30
171

Re: Case sensitivity

Hi dear Piwigo team ;

LucMorizur wrote:

[Bugtracker] ticket 1835

So to fix that bug, do you think that should be used usual "simple" PHP functions like strtolower() , or that we could use a much more complex, but normally also more complete function, like the one developped by Eric, Whiler and me: http://piwigo.org/dev/browser/extension … v=head#L28 ?


Our gallery : Le Site à Nous (a silly name in french, but here I don't care ;-) ! )
An event, a new gallery ? Plugin Event Cats
My test gallery : Tests Piwigo de Luc
Thanksalot for this beautiful project.

Offline

 

#6 2011-01-05 10:22:48

LucMorizur
Member
Vienne (Isère) - France
2009-04-30
171

Re: Case sensitivity

Hi;

I've submitted a proposal to fix [Bugtracker] ticket 1835 . You can test on my test gallery http://lucmorizur.free.fr that user Test (password: 1234) can be logged in with any spelling : TEST, test, TEst... I added two lines in ./identification.php , and a 30-lines function in ./include/functions_user.inc.php .


Now remains two points:

_ the case when $conf['insensitive_case_logon'] is set to true while there is already both accounts Test2 and TEST2 in the user list. In this case the solution I proposed allows to log both these accounts with their proper spelling, ie test2 or TEst2 will not be logged in, but Test2 and TEST2 will work, as if $conf['insensitive_case_logon'] was set to false. This can be tested on my gallery with these accounts (Test2 and TEST2, pwd: 1234).
Yet, should it be let just this way, or shouldn't be a warning? I think that at least the webmaster should be warned that some accounts will not benefit from $conf['insensitive_case_logon'] = true , but how should it be done? Because $conf['insensitive_case_logon'] can be set to true via FTP, thus an error message in the admin part would not be immediately displayed to the webmaster ; and anyway when should the test be done? This part is not fixed yet.

_ The solution I proposed uses PHP function strtolower() . But it gives me this strange behaviour: it will not convert "Béa" the same way on my local installation (Win XP SP3, WampServer 2) than on my ISP (Free) installation, see images attached. So it might be that accentuated characters might cause a problem, this needs to be cleared.


Our gallery : Le Site à Nous (a silly name in french, but here I don't care ;-) ! )
An event, a new gallery ? Plugin Event Cats
My test gallery : Tests Piwigo de Luc
Thanksalot for this beautiful project.

Offline

 

#7 2011-01-05 11:13:57

Whiler
Member
Clichy
2004-12-24
24

Re: Case sensitivity

LucMorizur wrote:

_ The solution I proposed uses PHP function strtolower() . But it gives me this strange behaviour: it will not convert "Béa" the same way on my local installation (Win XP SP3, WampServer 2) than on my ISP (Free) installation, see images attached. So it might be that accentuated characters might cause a problem, this needs to be cleared.

You have to specify your encoding somewhere...
For example: mb_strtolower($_POST['n_text_entry'], 'UTF-8')

Another way would be to use a source file encoded in ISO.... but not very useful in your context ;o)

Offline

 

#8 2011-01-05 11:21:21

Whiler
Member
Clichy
2004-12-24
24

Re: Case sensitivity

or you can change your internal encoding...

If you test this, I'm pretty sure you are in ISO-8859-1 ;o)

          <tr><td><?php
            echo('mb_strtolower ('.mb_internal_encoding().')&nbsp;:&nbsp;');
          ?></td><td><?php
            echo mb_strtolower($_POST['n_text_entry'], 'UTF-8');
          ?></td></tr>

Offline

 

#9 2011-01-05 11:31:09

LucMorizur
Member
Vienne (Isère) - France
2009-04-30
171

Re: Case sensitivity

Whiler wrote:

or you can change your internal encoding...

If you test this, I'm pretty sure you are in ISO-8859-1 ;o)

          <tr><td><?php
            echo('mb_strtolower ('.mb_internal_encoding().')&nbsp;:&nbsp;');
          ?></td><td><?php
            echo mb_strtolower($_POST['n_text_entry'], 'UTF-8');
          ?></td></tr>

Well, yes in this case mb_internal_encoding() answers ISO-8859-1 cause I didn't precise 'UTF-8' as I should have done, you're right that it should be done this way for mb_strtolower() . But what about strtolower() ? And is it allowed to use mb_strtolower() on all servers ?

EDIT : I precise that in this test page, I added the line <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> , and that FF in the page infos, tells that the page is properly displayed and coded in UTF-8. But that's true, mb_strtolower() should be precised 'UTF-8'.

Last edited by LucMorizur (2011-01-05 11:33:27)


Our gallery : Le Site à Nous (a silly name in french, but here I don't care ;-) ! )
An event, a new gallery ? Plugin Event Cats
My test gallery : Tests Piwigo de Luc
Thanksalot for this beautiful project.

Offline

 

#10 2011-01-05 11:41:04

Whiler
Member
Clichy
2004-12-24
24

Re: Case sensitivity

LucMorizur wrote:

But what about strtolower() ?

It works nicely on my apaches (window and unix...)
So I guess, there's an issue with your server...

LucMorizur wrote:

And is it allowed to use mb_strtolower() on all servers ?

Nope... an extension is needed.. so we can imagine that some providers don't activate it :
extension=php_mbstring.dll|so

Offline

 

#11 2011-01-10 22:00:28

LucMorizur
Member
Vienne (Isère) - France
2009-04-30
171

Re: Case sensitivity

LucMorizur wrote:

Hi;

I've submitted a proposal to fix [Bugtracker] ticket 1835 . You can test on my test gallery http://lucmorizur.free.fr that user Test (password: 1234) can be logged in with any spelling : TEST, test, TEst... I added two lines in ./identification.php , and a 30-lines function in ./include/functions_user.inc.php .


Now remains two points:

(...)

A third point I think : probably this solution does not work for Apache authentication and for external user tables (when Piwigo uses a user table that it doesn't manage by itself). Yet at least, users should be able to authenticate with the proper spelling of their username, just as if $conf['insensitive_case_logon'] was set to false. I didn't test such cases.


Our gallery : Le Site à Nous (a silly name in french, but here I don't care ;-) ! )
An event, a new gallery ? Plugin Event Cats
My test gallery : Tests Piwigo de Luc
Thanksalot for this beautiful project.

Offline

 

#12 2011-01-11 09:17:24

rvelices
Former Piwigo Team
2005-12-29
1960

Re: Case sensitivity

as I said in the bug tracker the easiest and automatic solution would be to change the type of db column #users.username from binary to not binary (case insesitive).

I already did this on my site sometime ago. The only issue if we do that automatically in the next version is if you already have users Test and test -> it will be a problem because username is a unique key and we'll have db errors ...

Offline

 

#13 2011-01-11 13:38:49

LucMorizur
Member
Vienne (Isère) - France
2009-04-30
171

Re: Case sensitivity

rvelices wrote:

as I said in the bug tracker the easiest and automatic solution would be to change the type of db column #users.username from binary to not binary (case insesitive).

I already did this on my site sometime ago. The only issue if we do that automatically in the next version is if you already have users Test and test -> it will be a problem because username is a unique key and we'll have db errors ...

Many thanks rvelices for your contribution.

I have written in the bugtracker that on my local configuration, the table #_users is already of type utf8_general_ci , but as you know well, indeed the table is qualified utf8_general_ci for the whole table, but in fact most of the fields are utf8_general_ci , and the `username` field is utf8_bin . And indeed as you wrote it, once you've changed the `username` field to utf8_general_ci , if you ask for

SELECT * FROM #_users WHERE `username` = 'test'

you will get user Test.

And as you wrote it also, if you have already users "test" and "Test" in this table, the attempt to convert field `username` to utf8_general_ci fails, because an item of the field `username` cannot be duplicate, and "test" and "Test" are equal in a utf8_general_ci field.



Yet if we go on forward, to my opinon the solution you propose would then force to establish at installation time whether usernames shall be case sensitive or not -- which could be better indeed. Because changing `username` field to utf8_general_ci once the gallery is installed , is at least as much complex as my solution (need to arrange the case where conversion fails, propose a button or anything allowing the webmaster to ask for the conversion, and display this command).

Looking forward on your opinion on this :-) .


Our gallery : Le Site à Nous (a silly name in french, but here I don't care ;-) ! )
An event, a new gallery ? Plugin Event Cats
My test gallery : Tests Piwigo de Luc
Thanksalot for this beautiful project.

Offline

 

#14 2011-04-04 03:48:10

justin_ruch1180
Member
2011-04-04
1

Re: Case sensitivity

WHEW.. This is complicated for me...:(

Offline

 

#15 2011-04-04 11:23:18

LucMorizur
Member
Vienne (Isère) - France
2009-04-30
171

Re: Case sensitivity

justin_ruch1180 wrote:

WHEW.. This is complicated for me...:(

For us too ^^ !

The problem is that we want to have one's cake and eat it ("avoir le beurre et l'argent du beurre" in french, or so says my dictionnary :-/ ...): we want to make possible that a gallery is one day "username case sensitive" and the other day "username case insensitive" -- at least is it what parameter $conf[insensitive_case_logon] leads to. So we need to manage the case where some different usernames are in conflict at the instant you turn your gallery "username case insensitive" -- if you have users "Test" and "test" who both registered in your gallery when it was "username case sensitive", and that these two usernames where considered as really different.

Indeed I'm still interested in rvelices' opinion ;-) .


Our gallery : Le Site à Nous (a silly name in french, but here I don't care ;-) ! )
An event, a new gallery ? Plugin Event Cats
My test gallery : Tests Piwigo de Luc
Thanksalot for this beautiful project.

Offline

 
  •  » Engine
  •  » Case sensitivity

Board footer

Powered by FluxBB

github twitter newsletter Donate Piwigo.org © 2002-2024 · Contact