Featured post

new redirect for blender.org bpy docs.

http://www.blender.org/api/blender_python_api_current/ As of 10/11 november 2015 we can now link to the current api docs and not be worr...

December 22, 2012

json, csv, and extended character sets

For those of you interested in json and csv loading, not all files can be read without first stating their encoding. In cases where the file has a different encoding than the default utf-8, you can expect an error message when the interpreter encounters the first special character. An error like UnicodeDecodeError: 'utf8' codec can't decode byte (...), is mitigated by including the encoding standard this way: open(filename, 'r', encoding='ISO-8859-15').

ISO-8859-15 is commonly used and worth trying first, but is not the only possibility. If you continue to get errors like UnicodeDecodeError then try other types of character encoding, read about them at wikipedia. Also here a pycon talk and direct explanation by Ned Batchelder

This snippet combines the complimentary data from two files. The csv contains group, type and named type data (ie. metal/non-metal and sub type) and combines it with the content of a more elaborate .json. Full script with data files here