Comments on: Big5 to UTF-8 on Chinese Windows http://randomstringofwords.com/big5-to-utf-8-on-chinese-windows/ RandomStringOfWords.com || RSoW.com Wed, 01 Feb 2012 01:03:09 +0000 hourly 1 By: Shawn http://randomstringofwords.com/big5-to-utf-8-on-chinese-windows/comment-page-1/#comment-325 Shawn Thu, 10 May 2007 02:06:43 +0000 http://info.rsow.com/big5-to-utf-8-on-chinese-windows/#comment-325 The problem basically lies in SQL Server only supporting UCS-2, and not UTF-8. Among Microsoft's suggestions for storing UTF-8 in SQL Server: use binary (ie. non-searchable, non-usable) fields to store all of your strings, or convert your entire application to our way instead of UTF-8. As far as I know, this stays this way until the next version of SQL Server, due for release sometime in the future (2008, I think). Unless of course, they drop that feature or delay the product. The real solution: don't use SQL Server if you need to store UTF-8 bytes. It does amuse me that Microsoft's site sends Content-Type: text/html; charset=utf-8, since I have no clue what a browser would do if it sent Content-Type: text/html; charset=ucs-2. But, then, since they use IIS, ASP, and Windows as their server, they have the only platform supported for automatic conversion to UTF-8 for output when using SQL Server. Also see: "What is the difference between UCS-2 and UTF-16?" http://unicode.org/faq/basic_q.html#25 The problem basically lies in SQL Server only supporting UCS-2, and not UTF-8. Among Microsoft’s suggestions for storing UTF-8 in SQL Server: use binary (ie. non-searchable, non-usable) fields to store all of your strings, or convert your entire application to our way instead of UTF-8. As far as I know, this stays this way until the next version of SQL Server, due for release sometime in the future (2008, I think). Unless of course, they drop that feature or delay the product.

The real solution: don’t use SQL Server if you need to store UTF-8 bytes.

It does amuse me that Microsoft’s site sends Content-Type: text/html; charset=utf-8, since I have no clue what a browser would do if it sent Content-Type: text/html; charset=ucs-2. But, then, since they use IIS, ASP, and Windows as their server, they have the only platform supported for automatic conversion to UTF-8 for output when using SQL Server.

Also see: “What is the difference between UCS-2 and UTF-16?” http://unicode.org/faq/basic_q.html#25

]]> By: Jason http://randomstringofwords.com/big5-to-utf-8-on-chinese-windows/comment-page-1/#comment-324 Jason Wed, 09 May 2007 14:48:08 +0000 http://info.rsow.com/big5-to-utf-8-on-chinese-windows/#comment-324 Shawn could you sum up what the problem/solution was to this? Shawn could you sum up what the problem/solution was to this?

]]> By: Shawn http://randomstringofwords.com/big5-to-utf-8-on-chinese-windows/comment-page-1/#comment-323 Shawn Wed, 02 May 2007 03:34:07 +0000 http://info.rsow.com/big5-to-utf-8-on-chinese-windows/#comment-323 I actually meant how SQL Server doesn't actually support UTF-8...at all (http://support.microsoft.com/kb/232580). Not to mention how your queries have to have the SQL in ASCII, with the data in UCS-2. Yeah, it would kinda shock me if it didn't run DB2, but you never know... I actually meant how SQL Server doesn’t actually support UTF-8…at all (http://support.microsoft.com/kb/232580). Not to mention how your queries have to have the SQL in ASCII, with the data in UCS-2.

Yeah, it would kinda shock me if it didn’t run DB2, but you never know…

]]> By: Jason http://randomstringofwords.com/big5-to-utf-8-on-chinese-windows/comment-page-1/#comment-322 Jason Wed, 02 May 2007 00:42:08 +0000 http://info.rsow.com/big5-to-utf-8-on-chinese-windows/#comment-322 Well the DB in question is DB2.. I assumed the bug tracking tools db was also DB2.. would be a little surprised if it wasn't.. but.. that'd be funny.. Well the DB in question is DB2.. I assumed the bug tracking tools db was also DB2.. would be a little surprised if it wasn’t.. but.. that’d be funny..

]]> By: Shawn http://randomstringofwords.com/big5-to-utf-8-on-chinese-windows/comment-page-1/#comment-321 Shawn Tue, 01 May 2007 23:16:33 +0000 http://info.rsow.com/big5-to-utf-8-on-chinese-windows/#comment-321 *cough* SQL Server *cough* That bug-tracking train wreck has temped me to find security holes in various places just so I could see if tracking a XSS or SQL injection bug would break it. :-) *cough* SQL Server *cough*

That bug-tracking train wreck has temped me to find security holes in various places just so I could see if tracking a XSS or SQL injection bug would break it. :-)

]]>