Commit Graph

2480 Commits

Author SHA1 Message Date
Paul Sokolovsky 9e215fa4c2 py: Make unichar_charlen() accept/return machine_uint_t. 2014-06-28 23:15:29 +03:00
Paul Sokolovsky a62da515af Merge pull request #729 from stinos/fix-include-order
unix: Fix mpconfig.h not being included before misc.h
2014-06-28 23:12:39 +03:00
stijn 5478ed18ea unix: Fix mpconfig.h not being included before misc.h
This fixes count_lead_ones in misc.h not compiling due to unknown types
2014-06-28 20:49:39 +02:00
Damien George b1b840554d Merge branch 'unicode' 2014-06-28 10:30:53 +01:00
Damien George 635b60e299 unix, stmhal: Add option for STR_UNICODE to mpconfigport.h.
Unicode is disabled by default for now, since FileIO.read(n) is
currently not implemented for text-mode files, and this is an
often function.
2014-06-28 10:29:52 +01:00
Damien George 8546ce1e28 py: Add missing #endif. 2014-06-28 10:29:22 +01:00
Damien George 41736f8201 tests: Write output in byte mode, not text mode.
This enables testing unicode and non-unicode implementations.
2014-06-28 10:29:12 +01:00
Damien George e04a44e2f6 py: Small comments, name changes, use of machine_int_t. 2014-06-28 10:27:23 +01:00
Damien George b3a50f0f3e Merge branch 'master' into unicode
Conflicts:
	py/mpconfig.h
2014-06-28 10:27:15 +01:00
Paul Sokolovsky 8993fb6cf0 py: Add protection against printing too nested or recursive data structures.
With a test which cannot be automatically validated so far.
2014-06-28 02:25:04 +03:00
Paul Sokolovsky 7e4ec3bf4f bare-arm: Hint of setting MICROPY_ERROR_REPORTING to REPORTING_TERSE.
Commented out so far, as enabled leads to dozen more bytes used actually
(due to string pooling effects).
2014-06-27 21:02:04 +03:00
Paul Sokolovsky 81df1e6c98 bare-arm: Disable array module and even bytearray type.
To squeeze few more hundreds of bytes.
2014-06-27 21:02:04 +03:00
Paul Sokolovsky cb78f862cb py: Allow to disable array module and bytearray type.
array.array and bytearray share big deal of code, so to get real savings,
both need to be disabled.
2014-06-27 21:02:04 +03:00
Paul Sokolovsky 0a1ea40273 bare-arm: Enable link map file.
This port supposed to be demo of uPy minimality, so let people behold it in
details.
2014-06-27 21:02:04 +03:00
Paul Sokolovsky 8a96ebea75 py: Move stack_ctrl_init() to mp_init().
As stack checking is enabled by default, ports which don't call
stack_ctrl_init() are broken now (report RuntimeError on startup). Save
them trouble and just init stack control framework in interpreter init.
2014-06-27 21:02:04 +03:00
Paul Sokolovsky 64c58403ef Merge pull request #720 from iabdalkader/mcu_name
Change MCU name config micro
2014-06-27 18:24:32 +03:00
mux a75e382a9b Change MCU name config micro 2014-06-27 00:35:53 +02:00
Paul Sokolovsky 3c8ce38d20 Merge pull request #717 from stinos/dead-code
unix: Remove unused CTRL-D definition
2014-06-27 01:23:18 +03:00
Paul Sokolovsky 3659af97c5 Merge pull request #703 from iabdalkader/micro_names
Add MICROPY_HW_MICRO_NAME to boards config
2014-06-27 01:19:17 +03:00
Paul Sokolovsky ed07d035d5 tests: Add basic test for unicode file i/o. 2014-06-27 00:04:20 +03:00
Paul Sokolovsky f5f6c3b792 streams: Reading by char count from unicode text streams is not implemented. 2014-06-27 00:04:20 +03:00
Paul Sokolovsky ce81312d8a misc: Add count_lead_ones() function, useful for UTF-8 handling. 2014-06-27 00:04:20 +03:00
Paul Sokolovsky 63143c94ce tests: Test for explicit start/end args to str methods for unicode. 2014-06-27 00:04:20 +03:00
Paul Sokolovsky ea2c936c7e objstrunicode: Refactor str_index_to_ptr() following objstr. 2014-06-27 00:04:20 +03:00
Paul Sokolovsky 26fda6dc8e objstr: 64-bit issues. 2014-06-27 00:04:19 +03:00
Paul Sokolovsky 00c904b47a objstrunicode: Signedness issues. 2014-06-27 00:04:19 +03:00
Paul Sokolovsky 1044c3dfe6 unicode: Make get_char()/next_char()/charlen() be 8-bit compatible.
Based on config define.
2014-06-27 00:04:19 +03:00
Paul Sokolovsky b1949e4c09 tests: Add tests for unicode find()/rfind()/index(). 2014-06-27 00:04:19 +03:00
Paul Sokolovsky 5048df0b7c objstr: find(), rfind(), index(): Make return value be unicode-aware. 2014-06-27 00:04:19 +03:00
Paul Sokolovsky 46d31e9ca9 unicode: Add utf8_ptr_to_index().
Useful when we have pointer to char inside string, but need to return char
index. (E.g. str.find()).
2014-06-27 00:04:19 +03:00
Paul Sokolovsky ded0fc77f7 py: Add dedicated unicode header. 2014-06-27 00:04:19 +03:00
Paul Sokolovsky 17994d1bd3 tests: Add test for unicode string iteration. 2014-06-27 00:04:19 +03:00
Paul Sokolovsky 79b7fe2ee5 objstrunicode: Implement iterator. 2014-06-27 00:04:19 +03:00
Paul Sokolovsky cdc020da4b objstrunicode: Re-add buffer protocol back for now, required for io.StringIO. 2014-06-27 00:04:18 +03:00
Paul Sokolovsky e7f2b4c875 objstrunicode: Revamp len() handling for unicode, and optimize bool(). 2014-06-27 00:04:18 +03:00
Paul Sokolovsky 86d3898e70 objstrunicode: Get rid of bytes checking, it's separate type. 2014-06-27 00:04:18 +03:00
Paul Sokolovsky d215ee1dc1 py: Make MICROPY_PY_BUILTINS_STR_UNICODE=1 buildable. 2014-06-27 00:04:18 +03:00
Paul Sokolovsky 9731912ccb py: Prune unneeded code from objstrunicode, reuse code in objstr. 2014-06-27 00:04:18 +03:00
Paul Sokolovsky 165eb69b86 vstr: Restore bytestr compatibility. 2014-06-27 00:04:18 +03:00
Paul Sokolovsky 42a52516fe builtin: Restore bytestr compatibility. 2014-06-27 00:04:18 +03:00
Chris Angelico 2ba2299d28 lexer, vstr: Add unicode support. 2014-06-27 00:04:18 +03:00
Chris Angelico 1e3781bc35 tests: Add unicode test. 2014-06-27 00:04:17 +03:00
Chris Angelico 9a1a4beb56 builtin: ord, chr: Unicode support. 2014-06-27 00:04:17 +03:00
Chris Angelico 64b468d873 objstrunicode: Basic implementation of unicode handling.
Squashed commit of the following:

commit 99dc21b67a
Author: Chris Angelico <rosuav@gmail.com>
Date:   Thu Jun 12 02:18:54 2014 +1000

    Optimize as per TODO (thanks Damien!)

commit 5bf0153eca
Author: Chris Angelico <rosuav@gmail.com>
Date:   Tue Jun 10 08:42:06 2014 +1000

    Test a default (= UTF-8) encode and decode

commit c962057ac3
Merge: e2c9782 195de32
Author: Chris Angelico <rosuav@gmail.com>
Date:   Tue Jun 10 05:23:03 2014 +1000

    Merge branch 'master' into unicode, resolving conflict on py/obj.h

commit e2c9782a65
Author: Chris Angelico <rosuav@gmail.com>
Date:   Tue Jun 10 05:05:57 2014 +1000

    More whitespace fixups

commit 086a2a0f57
Author: Chris Angelico <rosuav@gmail.com>
Date:   Tue Jun 10 05:04:20 2014 +1000

    Properly implement string slicing

commit 0d339a143e
Author: Chris Angelico <rosuav@gmail.com>
Date:   Tue Jun 10 02:24:11 2014 +1000

    Support slicing in str_index_to_ptr, and fix a bounds error

commit 24371c7267
Author: Chris Angelico <rosuav@gmail.com>
Date:   Tue Jun 10 02:10:22 2014 +1000

    Break out index-to-pointer calculation into a function

commit 616c24ac01
Author: Chris Angelico <rosuav@gmail.com>
Date:   Tue Jun 10 02:03:11 2014 +1000

    Add tests of string slicing, which currently fail

commit a24d19f676
Author: Chris Angelico <rosuav@gmail.com>
Date:   Tue Jun 10 01:56:53 2014 +1000

    Change string indexing to not precalculate the charlen, and add test for neg indexing

commit 0bcc7ab89e
Author: Chris Angelico <rosuav@gmail.com>
Date:   Sun Jun 8 22:09:17 2014 +1000

    Clean up constant qstr declarations now that charlen isn't needed

commit 5473e1a1db
Author: Chris Angelico <rosuav@gmail.com>
Date:   Sun Jun 8 07:18:42 2014 +1000

    Remove the charlen field from strings, calculating it when required

commit 5c1658ec71
Author: Chris Angelico <rosuav@gmail.com>
Date:   Sun Jun 8 07:11:27 2014 +1000

    Get rid of mp_obj_str_get_data_len() which was used in only one place

commit a019ba968b
Author: Chris Angelico <rosuav@gmail.com>
Date:   Sun Jun 8 06:58:26 2014 +1000

    Add a unichar_charlen() function to calculate length-in-characters from length-in-bytes

commit 44b0d5cff8
Author: Chris Angelico <rosuav@gmail.com>
Date:   Sun Jun 8 06:32:44 2014 +1000

    Use utf8_get/next_char in building up a string's repr

commit 30d1bad33f
Author: Chris Angelico <rosuav@gmail.com>
Date:   Sun Jun 8 06:10:45 2014 +1000

    Make utf8_get_char() and utf8_next_char() actually do what their names say

commit bc990dad9a
Author: Chris Angelico <rosuav@gmail.com>
Date:   Sun Jun 8 02:10:59 2014 +1000

    Revert "Add PEP 393-flags to strings and stub usage."

    This reverts commit c239f50952.

commit f9bebb28ad
Author: Chris Angelico <rosuav@gmail.com>
Date:   Sat Jun 7 15:41:48 2014 +1000

    Whitespace fixes

commit 279de0c8eb
Author: Chris Angelico <rosuav@gmail.com>
Date:   Sat Jun 7 15:28:35 2014 +1000

    Formatting/layout improvements - introduce macros for UTF-8 byte detection, add braces. No functional changes.

commit f1911f53d5
Author: Chris Angelico <rosuav@gmail.com>
Date:   Sat Jun 7 11:56:02 2014 +1000

    Make chr() Unicode-aware

commit f51ad737b4
Author: Chris Angelico <rosuav@gmail.com>
Date:   Sat Jun 7 11:44:07 2014 +1000

    Make a string's repr Unicode-aware

commit 01bd686846
Author: Chris Angelico <rosuav@gmail.com>
Date:   Sat Jun 7 11:33:43 2014 +1000

    Expand the Unicode tests

commit 7bc91904f8
Author: Chris Angelico <rosuav@gmail.com>
Date:   Sat Jun 7 11:27:30 2014 +1000

    Record byte lengths for byte strings

commit bb13212071
Author: Chris Angelico <rosuav@gmail.com>
Date:   Sat Jun 7 11:25:06 2014 +1000

    Make ord() Unicode-aware

commit 03f0cbe905
Author: Chris Angelico <rosuav@gmail.com>
Date:   Sat Jun 7 10:24:35 2014 +1000

    Retain characters as UTF-8 encoded Unicode

commit e924659b85
Author: Chris Angelico <rosuav@gmail.com>
Date:   Sat Jun 7 08:37:27 2014 +1000

    Add support for \u and \U escapes, but not \N (with explanatory comment)

commit 231031ac5f
Author: Chris Angelico <rosuav@gmail.com>
Date:   Sat Jun 7 05:09:35 2014 +1000

    Add character length to qstr

commit 6df1b946fb
Author: Chris Angelico <rosuav@gmail.com>
Date:   Fri Jun 6 13:48:36 2014 +1000

    Add test of UTF-8 encoded source file resulting in properly formed string

commit 16429b81a8
Author: Chris Angelico <rosuav@gmail.com>
Date:   Fri Jun 6 13:44:15 2014 +1000

    Make len(s) return character length (even though creation's still buggy)

commit cd2cf6663c
Author: Chris Angelico <rosuav@gmail.com>
Date:   Fri Jun 6 13:15:36 2014 +1000

    HACK - When indexing a qstr, count its charlen. Stupidly inefficient but POC.

    All tests pass now, though string creation is still buggy.

commit 47c234584d
Author: Chris Angelico <rosuav@gmail.com>
Date:   Fri Jun 6 13:15:32 2014 +1000

    objstr: Record character length separately from byte length

    CAUTION: Buggy, may crash stuff - qstr needs equivalent functionality too

commit b0f41c72af
Author: Chris Angelico <rosuav@gmail.com>
Date:   Fri Jun 6 05:37:36 2014 +1000

    Beginnings of UTF-8 support - construct strings from that many UTF-8-encoded chars, and subscript bytes the same way

commit 89452be641
Author: Chris Angelico <rosuav@gmail.com>
Date:   Fri Jun 6 05:28:47 2014 +1000

    Update comments - now aiming for UTF-8 rather than PEP 393 strings

commit c239f50952
Author: Chris Angelico <rosuav@gmail.com>
Date:   Wed Jun 4 05:28:12 2014 +1000

    Add PEP 393-flags to strings and stub usage.

    The test suite all passes, but nothing has actually been changed.
2014-06-27 00:04:17 +03:00
Paul Sokolovsky 83865347db objstrunicode: Complete copy of objstr, to be patched for unicode support. 2014-06-27 00:04:17 +03:00
Chris Angelico c88987c1af py: Implement basic unicode functions. 2014-06-27 00:04:17 +03:00
Paul Sokolovsky 12bc13eeb8 mpconfig.h: Add MICROPY_PY_BUILTINS_STR_UNICODE. 2014-06-27 00:04:17 +03:00
Paul Sokolovsky 16ac4962ae tests: Add test for catching infinite function recursion.
Put into misc/ to not complicate life for builds with check disabled.
2014-06-27 00:03:56 +03:00
Paul Sokolovsky 7a8ab5a730 stmhal: Use stackctrl framework. 2014-06-27 00:03:55 +03:00
Paul Sokolovsky 23668698cb py: Add portable framework to query/check C stack usage.
Such mechanism is important to get stable Python functioning, because Python
function calling is handled with C stack. The idea is to sprinkle
STACK_CHECK() calls in places where there can be C recursion.

TODO: Add more STACK_CHECK()'s.
2014-06-27 00:03:55 +03:00