Results 1 to 11 of 11

Thread: ClanLib UTF-8 language support

  1. #1
    ClanLib Developer
    Join Date
    May 2007
    Posts
    1,824

    Default ClanLib UTF-8 language support

    ClanLib 2.2 SVN has improved language support for fonts using UTF-8

    See attached
    Attached Images Attached Images  

  2. #2
    Lesser Knight
    Join Date
    Mar 2009
    Posts
    54

    Default

    Looks great!

    A quick question:
    does that sample works with the tahoma font?

  3. #3
    ClanLib Developer
    Join Date
    May 2007
    Posts
    1,824

    Default

    I have not tried, but it is very unlikely it will work with the tahoma font.

    That example uses the "Simsun" (Chinese Simplified) windows font that contains the relevant glyphs.

    As far as I know, the tahoma font does not contain chinese glyphs.

    I believe that when you use the windows GDI to draw fonts, If the selected font does not contain the glyph, Windows automatically uses a different font closely matching the parameters given to ::CreateFont() to obtain the glyph.

    (In this case "fdwCharSet" in ::CreateFont() would need set to GB2312_CHARSET.)

    ClanLib does not do that.

  4. #4

    Default How can i find the source code?

    Please give me a full tutorial ,How can i get the effect like the attached.

    Quote Originally Posted by rombust View Post
    ClanLib 2.2 SVN has improved language support for fonts using UTF-8

    See attached

  5. #5

    Question

    When I tried to draw text which contains chinese characters, it came out with "????" instead of the right character. Is there any configuration that need to be done? Could you attach a sample code?

  6. #6
    ClanLib Developer
    Join Date
    May 2007
    Posts
    1,824

    Default

    (Copied message from another thread written by myself):

    Okay...

    These do not work...

    Visual Studio 2010 - Encoding: Unicode - Codepage1200

    CL_File file("test_ń.txt");

    Compile Warning: 1>Sources\basic2d.cpp(37): warning C4566: character represented by universal-character-name '\u0144' cannot be represented in the current code page (1252)
    (As expected)

    Visual Studio 2010 - Encoding: Unicode - Codepage1200

    CL_File file(L"test_ń.txt"); <-- widechar

    Compile: Okay
    Run: Cannot open file "test_D.txt"
    (As expected, codepage conversion not performed)

    Visual Studio 2010 - Encoding: Central European (Windows) - Codepage1250

    CL_File file("test_ń.txt");

    Compile: Okay
    Run: Cannot open file "test_?t"
    (As expected, clanlib does not support this codepage)

    Visual Studio 2010 - Encoding: Unicode (UTF-8 with signature) - Codepage 65001

    CL_File file("test_ń.txt");

    Compile Warning: 1>Sources\basic2d.cpp(37): warning C4566: character represented by universal-character-name '\u0144' cannot be represented in the current code page (1252)
    (Not expected, Bug in visual studio editor - in my opinion)

    But these works!

    Visual Studio 2010 - Encoding: Unicode - Codepage1200

    CL_File file(CL_StringHelp::ucs2_to_utf8(L"test_ń.txt"));

    Compile: Okay
    Run: Okay

    Visual Studio 2010 - Encoding: Unicode (UTF-8 without signature) - Codepage 65001

    CL_File file("test_ń.txt");

    Compile: Okay
    Run: Okay


    It appears it works with UTF-8 without signature, but it does not work with the UTF-8 signature.

    This is strange, and is probably a Visual Studio bug (google shows that other people have reported it)

  7. #7
    ClanLib Developer
    Join Date
    May 2007
    Posts
    1,824

    Default

    So... Saving "Encoding: Unicode (UTF-8 without signature) - Codepage 65001"

    (also works in Visual studio 2008)

    CL_Font font(gc, "Simsun", 32);
    ...
    font.draw_text(gc, 32, 32, "你好世界");

    Name:  font_example.png
Views: 16
Size:  10.2 KB

  8. #8

    Default

    Thanks, rombust.
    I was using
    Code:
    font.draw_text(gc, 30, 30, "你好世界");
    but it doesn't work as expected. I found the right one is
    Code:
    font.draw_text(gc, 30, 30, CL_StringHelp::ucs2_to_utf8(L"你好世界"));
    It may have something to do with my VS2010 or OS, since mine is Chinese edition.

    One more question, does clanGUI support UTF-8?

  9. #9
    ClanLib Developer
    Join Date
    Sep 2006
    Location
    Denmark
    Posts
    554

    Default

    There are two articles in MSDN that mention Unicode support in Visual C++:

    Unicode Support in the Compiler and Linker
    Unicode Support

    The first one explains how the compiler itself deals with source files. If the source file is saved as UTF-8 with BOM then the compiler reads it as UTF-8. However, the second article then reveals the problem:

    In this version of the compiler, Unicode characters that cannot be represented in the current system code page are not allowed
    I believe Visual C++ reads the Chinese characters correctly, but then generates an 8 bit string literal in the current system code page. The string passed to CL_String's constructor is therefore not in UTF-8 but instead in your local system code page. This varies on your Windows installation and what country you are in, so if I compiled the source file created by xcc3721 I'd get entirely different characters than on his computer

    The documentation does not precisely describe what happens for wide-character string literals (L"abcabc"), but it is most likely either UCS-2 or UTF-16 that is generated. So that is why CL_StringHelp::ucs2_to_utf8 works. The size of wchar_t is not really standardized (its 16 bit in Windows, 32 bit in Linux afaik), so it may be dangerous to rely on this behavior if you want your program to be easily portable.

    Personally I avoid using any non-ascii characters in my source files simply because I avoid a lot of trouble. If my program needs to be in an other other language than English, I create some file with a string table that maps my strings to other languages. I don't know if this approach is preferable for you or not, but it might be worth considering.

    About clanGUI supporting Unicode: It does partially. All strings are UTF-8 and therefore safe in that way, but it lacks support for IME and right-to-left text handling. I guess the lack of IME support would be important for Chinese, right?

  10. #10
    Lesser Knight
    Join Date
    Jun 2011
    Location
    Shanghai in China
    Posts
    53

    Default

    Quote Originally Posted by Magnus Norddahl View Post
    There are two articles in MSDN that mention Unicode support in Visual C++:

    Unicode Support in the Compiler and Linker
    Unicode Support

    The first one explains how the compiler itself deals with source files. If the source file is saved as UTF-8 with BOM then the compiler reads it as UTF-8. However, the second article then reveals the problem:



    I believe Visual C++ reads the Chinese characters correctly, but then generates an 8 bit string literal in the current system code page. The string passed to CL_String's constructor is therefore not in UTF-8 but instead in your local system code page. This varies on your Windows installation and what country you are in, so if I compiled the source file created by xcc3721 I'd get entirely different characters than on his computer

    The documentation does not precisely describe what happens for wide-character string literals (L"abcabc"), but it is most likely either UCS-2 or UTF-16 that is generated. So that is why CL_StringHelp::ucs2_to_utf8 works. The size of wchar_t is not really standardized (its 16 bit in Windows, 32 bit in Linux afaik), so it may be dangerous to rely on this behavior if you want your program to be easily portable.

    Personally I avoid using any non-ascii characters in my source files simply because I avoid a lot of trouble. If my program needs to be in an other other language than English, I create some file with a string table that maps my strings to other languages. I don't know if this approach is preferable for you or not, but it might be worth considering.

    About clanGUI supporting Unicode: It does partially. All strings are UTF-8 and therefore safe in that way, but it lacks support for IME and right-to-left text handling. I guess the lack of IME support would be important for Chinese, right?
    "I guess the lack of IME support would be important for Chinese, right?" yes ,this is a very importang thing. How to solve the problem? Now it's a big thing for me.
    Make excellent games,make happly life.

  11. #11

Similar Threads

  1. Dose the GUI support language besides english?
    By westpointer in forum Novashell Game Creation System
    Replies: 1
    Last Post: 05-24-2010, 12:30 PM
  2. ClanLib 0.8 PhysFS support
    By Kez in forum Official ClanLib SDK Forums
    Replies: 1
    Last Post: 04-24-2009, 11:21 PM
  3. Does ClanLib support the Linux framebuffer or DirectFB?
    By stodge in forum Official ClanLib SDK Forums
    Replies: 1
    Last Post: 11-03-2007, 07:47 PM
  4. how does clanlib support other language?
    By gison in forum Official ClanLib SDK Forums
    Replies: 2
    Last Post: 06-06-2007, 02:32 AM

Bookmarks

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •