Piwigo.org

You are not logged in. (Register / Login)

Announcement

#1 2012-01-15 06:09:53

hwang00
Member
Registered: 2011-02-18
Posts: 4

support unicode characters in pathname and filename

I try to keep remote files(on server)  synchronize with local files(on pc). But i manage local photos by directory name(as album name) and filename(photo's description) in non-western, local-language-based way.
So you know that, some improvement of core features is devoutly to be wished.

* Support unicode(non west characters) in pathname and filename of data in "galleries" folder.
* and keep the original filename uploaded (contain nonascii characters)

This will bring great benefit to all non-English users,and, it also makes ftp/remote batch photo files management more logical and easier.

Last edited by hwang00 (2012-01-15 06:13:42)

Offline

 

#2 2012-05-11 15:14:57

miblo69
Guest

Re: support unicode characters in pathname and filename

I totally agree with this! The gallery seems to have most features I have been looking for, but by not supporting UTF-8 or international characters - it's a show stopper.
Thanks,
~Mike

 

#3 2012-06-05 14:04:48

miblo69
Member
Location: Stockholm, Sweden
Registered: 2012-06-05
Posts: 16

Re: support unicode characters in pathname and filename

I have gone so far as to try and manually update the code (using 2.4 RC3) and identify where the problems are, to allow UTF-8 characters in filenames and directories when doing a 'Local Sync'.

But it's driving me nuts! There must be some character conversion going on 'behind the scenes', that doesn't allow for UTF-8. I roughly know where the problem lies, but I just can't find a solution. If someone is willing to give a hand, I'll gladly find time to revise the php code. Once finished, I'll supply it to the original developers.

I am working on the Local Sync stuff, and am editing i.php. 

- First I added "$conf['sync_chars_regex'] = '/^[a-zA-Z0-9-_.åäöÅÄÖ ]+$/';" to the config.inc.php in /local/config/  (Yes, Swedish is my native language...)

- Secondly, I have added a urldecode in i.php:
Line 199: changed from '$req = ltrim($req, '/');' to '$req = urldecode(ltrim($req, '/'));

BUT - the query starting on line 449, does NEVER match if $page['src_location'] contains a UTF-8 encoded string. All ASCII characters including space works, but as soon as öäåÖÄÅ are in it - it fails and retruns nothing.

The really strange part is that when I print the value of $page['src_location']  as a HEX string (using error_log and printing to a file), and compare it with the actual field contents (running a manual mysql query) - they are identical!

When the query is run through php/piwigo - no match. Manually with mysql - it matches.

If someone could tell me what the difference is - I'd be able to proceed with the 'Internationalization' of Piwigo.

Running on a Linux host,  3.3.7-1.fc16.i686, PHP 5.3.13, UTF-8 all over, Apache/2.2.22.

Many Thanks for any pointers!
~Mike

Offline

 

#4 2012-06-05 15:27:41

miblo69
Member
Location: Stockholm, Sweden
Registered: 2012-06-05
Posts: 16

Re: support unicode characters in pathname and filename

Jumping Jee-bees! I made it work! :-)

Dont ask me why, but I added the php function:
pwg_db_check_charset();

before this line:
if ( ($row=pwg_db_fetch_assoc(pwg_query($query))) )
in the file i.php.

And then it just worked! In addition to the previously posted urldecode.
Plus I modified DB_COLLATE to contain 'utf8_general_ci' (default it is blank).

~Mike

Offline

 

#5 2012-06-05 16:33:45

flop25
Piwigo Team
Registered: 2006-07-06
Posts: 6430
Website

Re: support unicode characters in pathname and filename

thx a lot I think it will be in 2.4.1
Could you open a entry in our bugtracker ? thx again http://piwigo.org/bugs/main_page.php


To get a better help : Politeness like Hello-A link-Your past actions precisely described
Check my extensions : more than 30 available
who I am and what I do : http://fr.gravatar.com/flop25
My gallery : an illustration of how to integrate Piwigo in your website

Offline

 

#6 2017-03-05 11:07:38

miblo69
Member
Location: Stockholm, Sweden
Registered: 2012-06-05
Posts: 16

Re: support unicode characters in pathname and filename

Long time sine last post.... But I am running latest 2.8.6, and have always (for years) had problems with filenames containing International (Swedish in my case) characters.

Especially if the directory name starts with a Swedish Character such as '/galleries/2017/Åsas Bilder', it will be displayed as 'sas Bilder'. And Quick Local Synchronization will Add it every time.

However, I found a fix and solved it for my purposes. It was a problem with basename() that won't handle MutliByte conversions correctly. The solution was to add a mb_basename() function, and modify the site_update.php in two places.

In functions.inc.php I added:

/**
* Attemot to fix MultiByte filenames
* MB 2017-03-04
* Originally found here:
* http://stackoverflow.com/questions/4451 … e-is-utf-8
*/
if (!function_exists("mb_basename"))
{
    function mb_basename($path)
    {
        $separator = " qq ";
        $path = preg_replace("/[^ ]/u", $separator."\$0".$separator, $path);
        $base = basename($path);
        $base = str_replace($separator, "", $base);
        return $base;
    }
}


Then edit site_update.php on line 216 from $dir = basename($fulldir); to $dir = mb_basename($fulldir);
And on line 502 change $filename = basename($path); to $filename = mb_basename($path);

It works for me. Not excessiveley tested, and don't know if it breaks other things. But I am very happy that I could get Quick Local Synchronization to work correctly with Swedish Characters.


Of course, this requires that you also have edited your $conf['sync_chars_regex']  to allow International/Swedish characters.

Offline

 

#7 2017-03-15 16:38:19

miblo69
Member
Location: Stockholm, Sweden
Registered: 2012-06-05
Posts: 16

Re: support unicode characters in pathname and filename

After some more testing, I found an even simpler solution.

In .../local/config/config.inc.php i added:
setlocale('sv_SE.utf8');

Which matches my system, and which is also installed. You can see which locales are installed by running  'locale -a' from CLI.

EDIT: Spelling.

Last edited by miblo69 (2017-03-15 16:39:39)

Offline

 

Board footer

Powered by FluxBB

github twitter facebook google+ newsletter Donate Piwigo.org © 2002-2017 · Contact