Commit Graph

2527 Commits

Author SHA1 Message Date
Damien George d7e3b36a09 py/compile: Remove unnecessary label in compilation of for statement. 2015-10-14 15:51:12 +01:00
Damien George fcce1483fa py: Fix build of ARM native emitter due to recent viper changes.
Addresses #1510.
2015-10-14 12:40:54 +01:00
Damien George 59a41e8fcd py/qstr: Fix calc of qstr memory usage, due to new qstr chunk allocation. 2015-10-13 15:52:06 +01:00
Damien George b8f9ac5411 py: Implement ptr32 load and store in viper emitter. 2015-10-13 00:50:17 +01:00
Damien George 3c9c3687d6 py: Add support to call __init__ from a builtin module on first import. 2015-10-12 13:46:01 +01:00
Paul Sokolovsky 408b74d74c py: Allow to to build MicroPython as a static library.
The whole current port gets slurped into a static lib named
"libmicropython.a". Maybe that's not ideal, but at least something
to start with.
2015-10-12 15:32:06 +03:00
Damien George fdfcee7b1e py/parse: Make parser error handling cleaner, less spaghetti-like. 2015-10-12 12:59:18 +01:00
Damien George 64f2b213bb py: Move constant folding from compiler to parser.
It makes much more sense to do constant folding in the parser while the
parse tree is being built.  This eliminates the need to create parse
nodes that will just be folded away.  The code is slightly simpler and a
bit smaller as well.

Constant folding now has a configuration option,
MICROPY_COMP_CONST_FOLDING, which is enabled by default.
2015-10-12 12:58:45 +01:00
Paul Sokolovsky 91fc075a33 py/objarray: Allow to create array of void pointers, as extension to CPython.
Using 'P' format specifier (matches struct module). This is another shortcut
for FFI, just as previously introduced "array of objects" ('O').
2015-10-12 10:13:51 +03:00
Paul Sokolovsky 1b586f3a73 py: Rename MP_BOOL() to mp_obj_new_bool() for consistency in naming. 2015-10-11 15:18:15 +03:00
Paul Sokolovsky 53ca6ae1f3 py/makeqstrdata.py: Catch and report case of empty input file.
The usual cause would be that a cross-compiler for a port is not in PATH.
2015-10-11 11:09:57 +03:00
Damien George 366239b8b9 py/parse: Factor logic when creating parse node from and-rule. 2015-10-08 23:13:18 +01:00
Damien George b948de36fb py: Don't generate unnecessary parse nodes for assignment or kwargs.
This patch eliminates the need for a nested parse node for assignments
and keyword arguments.  It saves a little bit of RAM when parsing.
2015-10-08 14:26:01 +01:00
Damien George 9f5f156b9d py/emitnative: Raise ViperTypeError for unsupported unary ops. 2015-10-08 13:08:59 +01:00
Damien George 7e12a601b8 py/compile: Fix edge case when constant-folding negation of integer.
Also adds tests specifically for testing constant folding.
2015-10-08 13:02:00 +01:00
Paul Sokolovsky aaa8867d4a modussl: SSL socket wrapper module based on axTLS. 2015-10-06 18:10:39 +03:00
Damien George 0496de26d3 py: Allow to enable inline assembler without native emitter. 2015-10-03 17:07:54 +01:00
Damien George 58e0f4ac50 py: Allocate parse nodes in chunks to reduce fragmentation and RAM use.
With this patch parse nodes are allocated sequentially in chunks.  This
reduces fragmentation of the heap and prevents waste at the end of
individually allocated parse nodes.

Saves roughly 20% of RAM during parse stage.
2015-10-02 00:11:11 +01:00
Damien George e5635f4ab3 py: Catch all cases of integer (big and small) division by zero. 2015-10-01 22:48:48 +01:00
Damien George 2065373f67 py/mpz: Fix bignum anding of large negative with smaller positive int. 2015-10-01 22:35:06 +01:00
Damien George 2f4e8511cd py/mpz: Force rhs of mpz_shl_inpl/mpz_shr_inpl to be unsigned.
Python semantics are that rhs of shift must be non-negative, so there's
no need to handle negative values in the underlying mpz implementation.
2015-10-01 18:01:37 +01:00
Damien George 4c02e54298 py/mpz: Raise NotImplError instead of failing assertion. 2015-10-01 17:57:36 +01:00
Damien George 5f3c3ec5e6 py/parsenum: Provide detailed error for int parsing with escaped bytes.
This patch adds more fine grained error message control for errors when
parsing integers (now has terse, normal and detailed).  When detailed is
enabled, the error now escapes bytes when printing them so they can be
more easily seen.
2015-10-01 17:18:12 +01:00
Tom Soulanille f1a9923308 py/objrange: Bugfix for range_subscr() when index is a slice object. 2015-09-28 14:01:28 +00:00
Damien George 9d5e5c08ab py/compile: Put compiler state on the C stack.
It's relatively small (between 44 and 56 bytes) and helps to reduce heap
pressure and fragmentation during compilation.
2015-09-24 13:15:57 +01:00
Damien George fbcaf0ea18 py: Slightly simplify compile and emit of star/double-star arguments.
Saves a few bytes of code space and eliminates need for rot_two
bytecode (hence saving RAM and execution time, by a tiny bit).
2015-09-23 11:47:01 +01:00
Delio Brignoli e6978a4e26 py: Fix call args when a stararg is followed by keyword args. 2015-09-23 11:37:00 +01:00
Alex March bfb272b9e0 py/repl: Treat escaped quotes correctly in REPL continuation.
Escaped quotes are now recognised correctly in REPL when used
inside normal quotes.

Fixes: #1419
2015-09-19 14:06:23 +01:00
Tom Soulanille 661d9d1901 py/objslice: Fix indent. 2015-09-15 22:46:30 +01:00
Tom Soulanille aeb62f9ae3 py/objslice: Make slice attributes (start/stop/step) readable.
Configurable with MICROPY_PY_BUILTINS_SLICE_ATTRS.  Disabled by default.
2015-09-15 21:59:20 +01:00
Damien George 8b4fb4fe14 py/mpz: Fix calculation of max digit storage for mpz; fix sys.maxsize.
When creating constant mpz's, the length of the mpz must be exactly how
many digits are used (not allocated) otherwise these numbers are not
compatible with dynamically allocated numbers.

Addresses issue #1448.
2015-09-15 16:15:57 +01:00
Damien George 0af73014cc lib/mp-readline: Add auto-indent support.
4 spaces are added at start of line to match previous indent, and if
previous line ended in colon.

Backspace deletes 4 space if only spaces begin a line.

Configurable via MICROPY_REPL_AUTO_INDENT.  Disabled by default.
2015-09-12 22:07:23 +01:00
Dave Hylands 9d6128acdc stmhal: fix single precision float printing error
Fixes #1435.
2015-09-11 23:09:50 +03:00
Damien George 2b000474d9 py/lexer: Properly classify floats that look like hex numbers.
Eg 0e0 almost looks like a hex number but in fact is a float.
2015-09-07 17:33:44 +01:00
Damien George 0be3c70cd8 py/lexer: Raise SyntaxError when unicode char point out of range. 2015-09-07 17:19:17 +01:00
Damien George 081f9325f5 py/lexer: Raise NotImplError for unicode name escape, instead of assert. 2015-09-07 17:08:49 +01:00
Damien George 558a016e2c py/compile: Refine SyntaxError for repeated use of global/nonlocal. 2015-09-07 16:55:02 +01:00
Damien George 3a2171e406 py: Eliminate some cases which trigger unused parameter warnings. 2015-09-04 16:53:46 +01:00
Damien George 42cec5c893 py/objstr: Check for keyword args before checking for no posn args.
Otherwise something like bytes(abc=123) will succeed.
2015-09-04 16:51:55 +01:00
Damien George 55b11e6d38 py/objstr: For str.endswith(s, start) raise NotImpl instead of assert. 2015-09-04 16:49:56 +01:00
Damien George 0b7a66ab97 py/objbool: Simplify dispatch of bool binary op.
This optimises (in speed and code size) for the common case where the
binary op for the bool object is supported.  Unsupported binary ops
still behave the same.
2015-09-04 16:46:15 +01:00
Damien George ea5b59bfe6 py/compile: Only compile function annotations if really needed.
Function annotations are only needed when the native emitter is enabled
and when the current scope is emitted in viper mode.  All other times
the annotations can be skipped completely.
2015-09-04 16:44:14 +01:00
Tony Abboud 8d8fdcb4be stmhal: add option to query for the current usb mode
Fetch the current usb mode and return a string representation when
pyb.usb_mode() is called with no args. The possible string values are interned
as qstr's. None will be returned if an incorrect mode is set.
2015-09-03 23:30:43 +01:00
Damien George 821b7f22fe py: Use mp_not_implemented consistently for not implemented features. 2015-09-03 23:14:06 +01:00
Damien George e2aa117798 py/objstr: Simplify printing of bytes objects when unicode enabled. 2015-09-03 23:03:57 +01:00
Damien George 516982242d py: Inline single use of mp_obj_str_get_len in mp_obj_len_maybe.
Gets rid of redundant double check for string type.

Also remove obsolete declaration of mp_obj_str_get_hash.
2015-09-03 23:01:07 +01:00
Paul Sokolovsky 8bf00084b6 py: Make "enumerate" qstr be conditional on MICROPY_PY_BUILTINS_ENUMERATE. 2015-09-03 19:35:52 +03:00
Damien George 81794fcd31 py/binary: Add support for array('q') and array('Q'). 2015-09-01 16:31:48 +01:00
Damien George 22602cc37b py/objstr: Make str.rsplit(None,n) raise NotImpl instead of assert(0). 2015-09-01 15:35:31 +01:00
Paul Sokolovsky 2ff2ea5f3b vm: Handle "raise X from Y" statements the best way we can.
By issuing a warning that exception chaining is not supported, and ignoring
"from Y" argument.
2015-09-01 10:39:04 +03:00
Paul Sokolovsky 21ffa7c4ba modbuiltins: Consistently use indentation for #if. 2015-08-31 00:22:11 +03:00
Damien George 000730ecaa py/objstr: Simplify error handling for bad conversion specifier. 2015-08-30 12:43:21 +01:00
Tom Soulanille 6433f71e8f py/objgetitemiter: Make it_iternext() recognize IndexError. 2015-08-30 11:49:49 +01:00
Damien George b648e98ad0 py/objstr: Fix error reporting for unexpected end of modulo format str. 2015-08-29 23:13:51 +01:00
Damien George 7ef75f9f75 py/objstr: Fix error type for badly formatted format specifier.
Was KeyError, should be ValueError.
2015-08-29 23:13:51 +01:00
Damien George 51b9a0d0c4 py/objstr: Make string formatting 8-bit clean. 2015-08-29 23:13:51 +01:00
Paul Sokolovsky 18c22faf4d py: Treat -m32 flag as part of CC, LD, etc.
Indeed, this flag efectively selects architecture target, and must
consistently apply to all compiles and links, including 3rd-party
libraries, unlike CFLAGS, which have MicroPython-specific setting.
2015-08-29 21:18:10 +03:00
Paul Sokolovsky 8b3b2d04a8 objset: frozensets are hashable. 2015-08-28 22:31:52 +03:00
Paul Sokolovsky 22ff397fb1 py: Add MICROPY_PY_BUILTINS_FILTER, disable for minimal ports.
Saves 320 bytes on x86.
2015-08-20 01:05:11 +03:00
Damien George 7f70b60f4d py: Remove unused compile scope flags, and irrelevant flag compute code. 2015-08-17 22:39:03 +01:00
Damien George 65dc960e3b unix-cpy: Remove unix-cpy. It's no longer needed.
unix-cpy was originally written to get semantic equivalent with CPython
without writing functional tests.  When writing the initial
implementation of uPy it was a long way between lexer and functional
tests, so the half-way test was to make sure that the bytecode was
correct.  The idea was that if the uPy bytecode matched CPython 1-1 then
uPy would be proper Python if the bytecodes acted correctly.  And having
matching bytecode meant that it was less likely to miss some deep
subtlety in the Python semantics that would require an architectural
change later on.

But that is all history and it no longer makes sense to retain the
ability to output CPython bytecode, because:

1. It outputs CPython 3.3 compatible bytecode.  CPython's bytecode
changes from version to version, and seems to have changed quite a bit
in 3.5.  There's no point in changing the bytecode output to match
CPython anymore.

2. uPy and CPy do different optimisations to the bytecode which makes it
harder to match.

3. The bytecode tests are not run.  They were never part of Travis and
are not run locally anymore.

4. The EMIT_CPYTHON option needs a lot of extra source code which adds
heaps of noise, especially in compile.c.

5. Now that there is an extensive test suite (which tests functionality)
there is no need to match the bytecode.  Some very subtle behaviour is
tested with the test suite and passing these tests is a much better
way to stay Python-language compliant, rather than trying to match
CPy bytecode.
2015-08-17 12:51:26 +01:00
blmorris bdd78c31b6 py: Add stream_tell method, and use for unix and stmhal file tell. 2015-08-13 22:56:32 +01:00
Damien George c39093d801 py: In native ARM emitter, load r7 with table earlier in func prelude.
r7 may be needed to set up code state, so it must be loaded before the
set-up function is called.
2015-08-12 23:31:19 +01:00
Damien George 94ef8879cd py/makeversionhdr.py: Use returncode attr to be Python2.7 compat. 2015-08-12 23:28:16 +01:00
Damien George 0d5d16074f py/makeversionhdr.py: Fallback to using docs version if no git repo.
Addresses issue #1420.
2015-08-11 12:27:38 +01:00
Damien George 9a2913ed1c py/objlist: Make list += accept all arguments and add test. 2015-08-02 20:53:54 +01:00
Paul Sokolovsky 3a2fb201a5 makeqstrdata.py: Typo fix in comment. 2015-07-31 14:58:14 +03:00
Damien George cfc4c33801 py/compile: Give more precise line number for compile errors.
Previous to this patch there were some cases where line numbers for
errors were 0 (unknown).  Now the compiler attempts to give a better
line number where possible, in some cases giving the line number of the
closest statement, and other cases the line number of the inner-most
scope of the error (eg the line number of the start of the function).
This helps to give good (and sometimes exact) line numbers for
ViperTypeError exceptions.

This patch also makes sure that the first compile error (eg SyntaxError)
that is encountered is reported (previously it was the last one that was
reported).
2015-07-29 22:16:01 +00:00
Damien George 28596edf07 py/repl: Don't look inside strings for unmatched brackets/quotes.
When looking to see if the REPL input needs to be continued on the next
line, don't look inside strings for unmatched ()[]{} ''' or """.

Addresses issue #1387.
2015-07-29 15:21:42 +00:00
Damien George 84d59c2873 py: For viper compile errors, add traceback with function and filename.
ViperTypeError now includes filename and function name where the error
occurred.  The line number is the line number of the start of the
function definition, which is the best that can be done without a lot
more work.

Partially addresses issue #1381.
2015-07-27 22:20:00 +01:00
Damien George d8a7f8bff2 py: Disable REPL EMACS key bindings by default. 2015-07-26 15:49:13 +01:00
Tom Soulanille 7d588b0c7c lib/mp-readline: Add emacs-style control characters for cursor movement.
Disabled by default.  Adds 108 bytes to Thumb2 arch when enabled.
2015-07-26 15:22:13 +01:00
Damien George 96f0dd3cbc py/parse: Fix handling of empty input so it raises an exception. 2015-07-24 15:05:56 +00:00
Damien George fa7c61dfab py/parse: De-duplicate and simplify code for parser "or" rule. 2015-07-24 14:35:57 +00:00
Damien George d241c2a592 py/lexer: Raise SyntaxError when str hex escape sequence is malformed.
Addresses issue #1390.
2015-07-23 23:20:37 +01:00
Damien George f17e663493 py: Issue an error when compiling Viper functions with more than 4 args.
Otherwise it can be very hard to track down bugs.
2015-07-23 14:30:37 +01:00
Damien George e45c1dbd6f py: Allow viper functions to take up to 4 arguments.
Addresses issue #1380.
2015-07-23 14:11:29 +01:00
Delio Brignoli 6a388aaa7c py: reduce array slice assignment code size 2015-06-06 22:17:24 +02:00
Damien George 4915c2b871 py: Small code space optimisations for memoryview slice assigment.
Also adds #if guards to allow uPy core to compile without memoryview
enabled, but with slice assignment enabled.
2015-07-20 16:12:26 +01:00
Delio Brignoli 32aba40830 py: Implement memoryview slice assignment.
Adds ability to do "memcpy" with memoryview objects, such as:
m1[0:3] = m2[2:5].
2015-07-20 15:53:22 +01:00
Damien George c3bd9415cc py: Make qstr hash size configurable, defaults to 2 bytes.
This patch makes configurable, via MICROPY_QSTR_BYTES_IN_HASH, the
number of bytes used for a qstr hash.  It was originally fixed at 2
bytes, and now defaults to 2 bytes.  Setting it to 1 byte will save
ROM and RAM at a small expense of hash collisions.
2015-07-20 11:03:13 +00:00
Sebastian Plamauer 1e8ca3a3cf modbuiltins: Implement round() to precision. 2015-07-19 21:49:44 +03:00
Damien George ade9a05236 py: Improve allocation policy of qstr data.
Previous to this patch all interned strings lived in their own malloc'd
chunk.  On average this wastes N/2 bytes per interned string, where N is
the number-of-bytes for a quanta of the memory allocator (16 bytes on 32
bit archs).

With this patch interned strings are concatenated into the same malloc'd
chunk when possible.  Such chunks are enlarged inplace when possible,
and shrunk to fit when a new chunk is needed.

RAM savings with this patch are highly varied, but should always show an
improvement (unless only 3 or 4 strings are interned).  New version
typically uses about 70% of previous memory for the qstr data, and can
lead to savings of around 10% of total memory footprint of a running
script.

Costs about 120 bytes code size on Thumb2 archs (depends on how many
calls to gc_realloc are made).
2015-07-14 22:56:32 +01:00
Dave Hylands 9f76dcd682 py: Prevent many extra vstr allocations.
I checked the entire codebase, and every place that vstr_init_len
was called, there was a call to mp_obj_new_str_from_vstr after it.

mp_obj_new_str_from_vstr always tries to reallocate a new buffer
1 byte larger than the original to store the terminating null
character.

In many cases, if we allocated the initial buffer to be 1 byte
longer, we can prevent this extra allocation, and just reuse
the originally allocated buffer.

Asking to read 256 bytes and only getting 100 will still cause
the extra allocation, but if you ask to read 256 and get 256
then the extra allocation will be optimized away.

Yes - the reallocation is optimized in the heap to try and reuse
the buffer if it can, but it takes quite a few cycles to figure
this out.

Note by Damien: vstr_init_len should now be considered as a
string-init convenience function and used only when creating
null-terminated objects.
2015-07-06 17:29:27 +01:00
Damien George ef7dd8db2d py/repl: Fix case where shorter names are shadowed by longer names.
Previous to this patch, if "abcd" and "ab" were possible completions
to tab-completing "a", then tab would expand to "abcd" straight away
if this identifier appeared first in the dict.
2015-07-06 14:00:09 +01:00
Paul Sokolovsky 7e66b859b2 modstruct: Raise NotImplementedError for unsupported repeat specification. 2015-07-05 22:44:14 +03:00
Galen Hazelwood 616986a5f3 extmod: Add a2b_base64 and b2a_base64 functions to ubinascii. 2015-07-04 12:26:52 +03:00
Damien George 035deae1c6 py/objarray.c: Allow to build with debugging and bytearray but no array. 2015-07-02 16:26:57 +01:00
Daniel Campora 077812b2ab py: Add TimeoutError exception subclassed from OSError.
The TimeoutError is useful for some modules, specially the the
socket module. TimeoutError can then be alised to socket.timeout
and then Python code can differentiate between socket.error and
socket.timeout.
2015-07-02 11:53:08 +02:00
Paul Sokolovsky 9780e55274 builtinimport: Fix running package submodule with -m.
When "micropython -m pkg.mod" command was used, relative imports in pkg.mod
didn't work, because pkg.mod.__name__ was set to __main__, and the fact that
it's a package submodule was missed. This is an original workaround to this
issue. TODO: investigate and compare how CPython deals with this issue.
2015-06-29 00:26:45 +03:00
Paul Sokolovsky c4045f57e3 builtinimport: Catch case when relative import happens without active package.
CPython raises SystemError in this case, but we don't have that enabled, so
raise ImportError.
2015-06-27 00:40:21 +03:00
Paul Sokolovsky 6557a096d6 runtime: Improve mp_import_name() debug logging. 2015-06-27 00:40:21 +03:00
Paul Sokolovsky f44cc517a2 objstr: Add note that replace() is nicely optimized.
Doesn't allocate memory and returns original string if no replacements are
to be made.
2015-06-26 17:35:12 +03:00
Damien George 59fba2d6ea py: Remove mp_load_const_bytes and instead load precreated bytes object.
Previous to this patch each time a bytes object was referenced a new
instance (with the same data) was created.  With this patch a single
bytes object is created in the compiler and is loaded directly at execute
time as a true constant (similar to loading bignum and float objects).
This saves on allocating RAM and means that bytes objects can now be
used when the memory manager is locked (eg in interrupts).

The MP_BC_LOAD_CONST_BYTES bytecode was removed as part of this.

Generated bytecode is slightly larger due to storing a pointer to the
bytes object instead of the qstr identifier.

Code size is reduced by about 60 bytes on Thumb2 architectures.
2015-06-25 14:42:13 +00:00
Damien George ed570e4b2a py: Remove mp_load_const_str and replace uses with inlined version. 2015-06-25 13:58:41 +00:00
Damien George 2a1090a637 py: Clarify comment in parsenum.c about ValueError vs SyntaxError. 2015-06-23 16:08:51 +00:00
Daniel Campora 228c68a9cd py: Change exception type to ValueError when error reporting is terse.
Addresses issue #1347
2015-06-23 15:30:49 +02:00
Damien George 7f19a39a3b py: Cast argument for printf to int, to be compatible with more ports.
This allows stmhal to be compiled with MICROPY_DEBUG_PRINTERS.
2015-06-22 17:40:12 +01:00
Damien George 06593fb0f2 py: Use a wrapper to explicitly check self argument of builtin methods.
Previous to this patch a call such as list.append(1, 2) would lead to a
seg fault.  This is because list.append is a builtin method and the first
argument to such methods is always assumed to have the correct type.

Now, when a builtin method is extracted like this it is wrapped in a
checker object which checks the the type of the first argument before
calling the builtin function.

This feature is contrelled by MICROPY_BUILTIN_METHOD_CHECK_SELF_ARG and
is enabled by default.

See issue #1216.
2015-06-20 16:39:39 +01:00
Damien George c8870b7c69 py: Make showbc decode UNPACK_EX, and use correct range for unop/binop. 2015-06-18 15:12:17 +00:00
Damien George e9ce00d874 py: Implement divmod for mpz bignum. 2015-06-13 23:38:28 +01:00
Damien George c5029bcbf3 py: Add MP_BINARY_OP_DIVMOD to simplify and consolidate divmod builtin. 2015-06-13 23:36:30 +01:00
Damien George 6f49520042 py: Implement second arg for math.log (optional value for base). 2015-06-13 22:35:25 +01:00
Paul Sokolovsky a19ba5fea0 py: Rebuild port if mpconfigport.mk changed (if any).
mpconfigport.mk contains configuration options which affect the way
MicroPython is linked. In this regard, it's "stronger" configuration
dependency than even mpconfigport.h, so if we rebuild everything on
mpconfigport.h change, we certianly should of that on mpconfigport.mk
change too.
2015-06-11 11:08:39 +03:00
Damien George 7ed58cb663 py: Support unicode (utf-8 encoded) identifiers in Python source.
Enabled simply by making the identifier lexing code 8-bit clean.
2015-06-09 10:58:07 +00:00
Damien George 6e56bb623c py: Fallback to stack alloca for Python-stack if heap alloc fails.
If heap allocation for the Python-stack of a function fails then we may
as well allocate the Python-stack on the C stack.  This will allow to
run more code without using the heap.
2015-06-08 22:07:27 +01:00
Paul Sokolovsky 07408cbd1f unix: Make micropython -m <module> work for frozen modules.
This requires some special handling, which was previosuly applied only to
the main code path.
2015-06-06 00:10:58 +03:00
Damien George d7192fe68c py: Expose KeyboardInterrupt in builtins module. 2015-06-05 10:46:22 +01:00
Damien George 031278f661 unix: Allow to cat a script into stdin from the command line.
See issue #1306.
2015-06-04 23:42:45 +01:00
Damien George 567b349c2b py: Implement native multiply operation in viper emitter. 2015-06-04 14:00:29 +00:00
Damien George 4d9cad180d py: Implement implicit cast to obj for viper load/store index/value.
This allows to do "ar[i]" and "ar[i] = val" in viper when ar is a Python
object and i and/or val are native viper types (eg ints).

Patch also includes tests for this feature.
2015-06-04 11:52:16 +01:00
Damien George 953c23b1bc py: Add stack check to mp_iternext, since it can be called recursively.
Eg, builtin map can map over a map, etc, and call iternext deeply.

Addresses issue #1294.
2015-06-03 22:19:41 +01:00
Paul Sokolovsky 18fda7b42f frozenmod: Include header with function prototypes. 2015-05-31 23:02:04 +03:00
Damien George db52fd8e4d py: Wrap qstr defs in quotes to protect from C preprocessor.
This patch converts Q(abc) to "Q(abc)" to protect the abc from the
C preprocessor, then converts back after the preprocessor is finished.
So now we can safely put includes in mpconfig(port).h, and also
preprocess qstrdefsport.h (latter is now done also in this patch).

Addresses issue #1252.
2015-05-30 23:14:34 +01:00
Damien George 0ec8cf8e80 py/parsenum.c: Rename "raise" func to "raise_exc" to avoid name clash.
"raise" is a common word that was found to exist in a vendor's stdlib.
2015-05-30 23:13:16 +01:00
Damien George 4e4772bb5b py: Add further autodetection of endianess in mpconfig.h.
This patch was needed for gcc 4.4.
2015-05-30 23:12:30 +01:00
Damien George 26b512ea1b py: Get makeqstrdata.py and makeversionhdr.py running under Python 2.6.
These scripts should run under as wide a range of Python versions as
possible.
2015-05-30 23:11:16 +01:00
Damien George 79474c6b16 py: Remove unnecessary extra handling of padding of nan/inf.
C's printf will pad nan/inf differently to CPython.  Our implementation
originally conformed to C, now it conforms to CPython's way.

Tests for this are also added in this patch.
2015-05-28 14:22:12 +00:00
Damien George 2cae0f6290 py: Reduce size of mp_printf by eliminating unnecessary code.
Saves around 120 bytes on Thumb2 archs.
2015-05-28 13:54:56 +00:00
Damien George 1a97f6721f py: Make makeversionhdr.py extract version from docs/conf.py if no git.
Addresses issue #1285.
2015-05-25 13:26:47 +01:00
Damien George 3c4b5d4281 stmhal: Implement sys.std{in,out,err}.buffer, for raw byte mode.
It's configurable and only enabled for stmhal port.
2015-05-24 14:31:33 +01:00
Daniel Campora 6f1cffeb28 py: Remove hexdigest QSTR since the method has been removed as well. 2015-05-21 18:52:39 +02:00
Dave Hylands afaa66b657 py: Minor improvement to unichar_isxdigit
This drops the size of unicode_isxdigit from 0x1e + 0x02 filler to
0x14 bytes (so net code reduction of 12 bytes) and will make
unicode_is_xdigit perform slightly faster.
2015-05-20 09:31:22 +01:00
Dave Hylands 3ad94d6072 extmod: Add ubinascii.unhexlify
This also pulls out hex_digit from py/lexer.c and makes unichar_hex_digit
2015-05-20 09:29:22 +01:00
stijn 861670ba2a py: Implement mp_format_float for doubles and use where appropriate
This allows using (almost) the same code for printing floats everywhere,
removes the dependency on sprintf and uses just snprintf and
applies an msvc-specific fix for snprintf in a single place so
nan/inf are now printed correctly.
2015-05-17 21:47:11 +01:00
Kaspar Schleiser f5dd6f7f37 py/binary: Make return type of mp_binary_get_size size_t instead of int.
Fixes sign-compare warning.
2015-05-17 18:12:19 +01:00
Kaspar Schleiser b5cef5c7ea py/objobject: Don't make locals_dict if there's nothing to go in it. 2015-05-17 17:48:47 +01:00
Kaspar Schleiser cf5112b26f py: Change _mp_obj_fun_builtin_t.fun to function pointer.
ISO C forbids conversion between function pointers and void*, gcc
-pedantic triggers a warning.
2015-05-17 17:42:04 +01:00
Damien George 44e7cbf019 py: Clean up declarations of str type/funcs that are also in unicode.
Background: trying to make an amalgamation of all the code gave some
errors with redefined types and inconsistent use of static.
2015-05-17 16:44:24 +01:00
stijn 709955b601 py: Fix printing of complex number when imaginary part is nan 2015-05-13 23:10:15 +01:00
Damien George c50772d19f py: Add mp_obj_get_int_truncated and use it where appropriate.
mp_obj_get_int_truncated will raise a TypeError if the argument is not
an integral type.  Use mp_obj_int_get_truncated only when you know the
argument is a small or big int.
2015-05-12 23:05:53 +01:00
Damien George c2a4e4effc py: Convert hash API to use MP_UNARY_OP_HASH instead of ad-hoc function.
Hashing is now done using mp_unary_op function with MP_UNARY_OP_HASH as
the operator argument.  Hashing for int, str and bytes still go via
fast-path in mp_unary_op since they are the most common objects which
need to be hashed.

This lead to quite a bit of code cleanup, and should be more efficient
if anything.  It saves 176 bytes code space on Thumb2, and 360 bytes on
x86.

The only loss is that the error message "unhashable type" is now the
more generic "unsupported type for __hash__".
2015-05-12 22:46:02 +01:00
Paul Sokolovsky 6738c1dded vm: Properly handle StopIteration raised in user instance iterator.
I.e. in bytecode Python functions.
2015-05-11 23:57:42 +01:00
Paul Sokolovsky d5e629ad0e objgenerator: Can optimize StopIteration to STOP_ITERATION only if arg is None.
Unfortunately, MP_OBJ_STOP_ITERATION doesn't have means to pass an associated
value, so we can't optimize StopIteration exception with (non-None) argument
to MP_OBJ_STOP_ITERATION.
2015-05-11 23:57:42 +01:00
Paul Sokolovsky aa9dbb1b03 objgenerator: If generator yielded STOP_ITERATION value, it's stopped.
MP_OBJ_STOP_ITERATION is equivalent of raising StopIteration, except
mp_vm_return_kind_t for it is "yield".
2015-05-11 23:57:42 +01:00
Paul Sokolovsky a7c02c4538 vm: Null pointer test when checking for StopIteration optimizations.
When generator raises exception, it is automatically terminated (by setting
its code_state.ip to 0), which interferes with this check.

Triggered in particular by CPython's test_pep380.py.
2015-05-10 17:20:46 +03:00
Paul Sokolovsky 4a60cac916 runtime: Add TODO for mp_resume() on handling .close().
Exceptions in .close() should be ignored (dumped to sys.stderr, not
propagated), but in uPy, they are propagated. Fix would require
nlr-wrapping .close() call, which is expensive. Bu on the other hand,
.close() is not called often, so maybe that's not too bad (depends,
if it's finally called and that causes stack overflow, there's nothing
good in that). And yet on another hand, .close() can be implemented to
catch exceptions on its side, and that should be the right choice.
2015-05-10 02:39:45 +03:00
Paul Sokolovsky 4ed7b7f751 py: iternext() may not return MP_OBJ_NULL, only MP_OBJ_STOP_ITERATION.
Testing for incorrect value led to premature termination of generator
containing yield from for such iterator (e.g. "yield from [1, 2]").
2015-05-10 00:41:49 +03:00
Paul Sokolovsky 351424e719 emitnative: Revamp ARM codegen compile after full-arg support refactors.
The code was apparently broken after 9988618e0e
"py: Implement full func arg passing for native emitter.". This attempts to
propagate those changes to ARM emitter.
2015-05-08 22:35:34 +01:00
Damien George 57ebe1b27d unix-cpy: Fix adjustment of stack size when leaving exception handler.
Also remove __debug__ from one of the bytecode tests.
2015-05-06 16:46:21 +01:00
Damien George 9a42eb541e py: Fix naming of function arguments when function is a closure.
Addresses issue #1226.
2015-05-06 13:55:33 +01:00
Paul Sokolovsky cd87d20f46 mkrules.mk: Add comment why dependency parsing regex was tweaked.
(Windows compatibility.)
2015-05-06 15:34:33 +03:00
Ari Suutari c097ea5dd2 Adjust sed regex that processes dependency file from compiler
so that resulting file is correct also on windows systems (ie.
with file names containing drive letter).
2015-05-06 15:32:13 +03:00
Damien George 8872abcbc4 py: Remove LOAD_CONST_ELLIPSIS bytecode, use LOAD_CONST_OBJ instead.
Ellipsis constant is rarely used so no point having an extra bytecode
for it.
2015-05-05 22:15:42 +01:00
Paul Sokolovsky 37c6555b44 obj: Handle user instance hash based on Python adhoc rules.
User instances are hashable by default (using __hash__ inherited from
"object"). But if __eq__ is defined and __hash__ not defined in particular
class, instance is not hashable.
2015-05-05 22:48:19 +03:00
Paul Sokolovsky 7667727021 objsingleton: New home for Ellipsis and NotImplemented.
Having NotImplemented as MP_OBJ_SENTINEL turned out to be problematic
(it needs to be checked for in a lot of places, otherwise it'll crash
as would pass MP_OBJ_IS_OBJ()), so made a proper singleton value like
Ellipsis, both of them sharing the same type.
2015-05-05 22:18:07 +03:00
Paul Sokolovsky 5ab5ac5448 modbuiltins: Add NotImplemented builtin constant.
From https://docs.python.org/3/library/constants.html#NotImplemented :
"Special value which should be returned by the binary special methods
(e.g. __eq__(), __lt__(), __add__(), __rsub__(), etc.) to indicate
that the operation is not implemented with respect to the other type;
may be returned by the in-place binary special methods (e.g. __imul__(),
__iand__(), etc.) for the same purpose. Its truth value is true."

Some people however appear to abuse it to mean "no value" when None is
a legitimate value (don't do that).
2015-05-04 19:45:53 +03:00
Paul Sokolovsky 3d3ef36e97 modstruct: Rename module to "ustruct", to allow full Python-level impl. 2015-05-04 16:53:52 +03:00
Paul Sokolovsky 1829d86ef5 modstruct: Group module qstr's together. 2015-05-04 15:51:08 +03:00
Damien George 47b9809d23 py: Check that arg to object.__new__ is a user-defined type.
Addresses issue #1203.
2015-05-04 11:08:40 +01:00
Paul Sokolovsky 0116218fa8 modmachine: Add new module to access hardware, starting with physical memory.
Refactored from "stm" module, provides mem8, mem16, mem32 objects with
array subscript syntax.
2015-05-04 13:05:12 +03:00
Damien George f27aa27a0c py/repl.c: Fix shadowing of local variable "i". 2015-04-29 01:01:48 +01:00
Damien George a1a2c411b2 py, readline: Add tab autocompletion for REPL.
Can complete names in the global namespace, as well as a chain of
attributes, eg pyb.Pin.board.<tab> will give a list of all board pins.

Costs 700 bytes ROM on Thumb2 arch, but greatly increases usability of
REPL prompt.
2015-04-29 00:46:26 +01:00
Damien George ad9daadf8a py: Fix attrtuple array length in print and creation. 2015-04-29 00:17:48 +01:00
Damien George 95f53461c2 py: Replace py-version.sh with makeversionhdr.py, written in Python.
Also rename py-version.h to mpversion.h for consistency with mpconfig.h.
2015-04-28 23:52:36 +01:00
Daniel Campora 94792dd88f py: In attrtuple use the correct length value and index for 'fields'. 2015-04-29 00:04:00 +02:00
Damien George 9472907ae1 py: Fix handling of negative numbers in struct.pack of q/Q. 2015-04-25 23:51:14 +01:00
Paul Sokolovsky ae2c81ff38 vm: On exiting except block, clear sys.exc_info() value.
This doesn't handle case fo enclosed except blocks, but once again,
sys.exc_info() support is a workaround for software which uses it
instead of properly catching exceptions via variable in except clause.
2015-04-26 01:40:37 +03:00
Damien George 0f553fe10b py: Implement power op for long-long implementation of bignum. 2015-04-25 23:28:10 +01:00
Damien George 271d18eb08 py: Support conversion of bignum to bytes.
This gets int.to_bytes working for bignum, and also struct.pack with 'q'
and 'Q' args on 32-bit machines.

Addresses issue #1155.
2015-04-25 23:16:39 +01:00
Paul Sokolovsky 8b85d14b92 modsys: Add basic sys.exc_info() implementation.
The implementation is very basic and non-compliant and provided solely for
CPython compatibility. The function itself is bad Python2 heritage, its
usage is discouraged.
2015-04-25 03:49:23 +03:00
Paul Sokolovsky cf5b6f6974 objfun: Fix to stackless mode after recent refactor. 2015-04-25 01:43:45 +03:00
Damien George 8c1d23a0e2 py: Modify bytecode "with" behaviour so it doesn't use any heap.
Before this patch a "with" block needed to create a bound method object
on the heap for the __exit__ call.  Now it doesn't because we use
load_method instead of load_attr, and save the method+self on the stack.
2015-04-24 01:52:28 +01:00
Damien George ede0f3ab3d py: Add optional code to check bytes constructor values are in range.
Compiled in only if MICROPY_CPYTHON_COMPAT is set.

Addresses issue #1093.
2015-04-23 15:28:18 +01:00
Damien George 5e9810396f py/objint_mpz.c: Make int_from_uint actually return uint. 2015-04-22 23:17:34 +01:00
Damien George f66ee4dfd7 py/mpz.c: Fix bug with shl not truncating zero digits correctly. 2015-04-22 23:16:49 +01:00
Damien George 8f6aad2f48 py/emitnative.c: Fix stack adjustment when erroring on binary op. 2015-04-22 23:16:03 +01:00
Damien George 956d765786 py: Fix printing of "inf" and "nan" floating point values. 2015-04-22 16:51:29 +01:00
Damien George 4735c45c51 py: Clean up some bits and pieces in parser, grammar. 2015-04-21 16:43:18 +00:00
Damien George fa90ab1407 py: Simplify grammar for stmt rule (this is also how CPython has it). 2015-04-21 16:35:50 +00:00
Damien George d8837cea6f stmhal: Implement os.uname() to get details of OS and hardware. 2015-04-21 14:51:49 +00:00
Damien George c3184aea63 py: Add sys.implementation, containing uPy name and version number.
Uses attrtuple if it's enabled, otherwise just a normal tuple.
2015-04-21 14:45:04 +00:00
Damien George 5aa311d330 py: Add attrtuple object, for space-efficient tuples with attr access.
If you need the functionality of a namedtuple but will only make 1 or a
few instances, then use an attrtuple instead.
2015-04-21 14:14:24 +00:00
Damien George c8b60f013b py: Make viper codegen raise proper exception (ViperTypeError) on error.
This fixes a long standing problem that viper code generation gave
terrible error messages, and actually no errors on pyboard where
assertions are disabled.

Now all compile-time errors are raised as proper Python exceptions, and
are of type ViperTypeError.

Addresses issue #940.
2015-04-20 13:29:31 +00:00
= 5008972fef py/inlinethumb: Support for core floating point instructions.
Adds support for the following Thumb2 VFP instructions, via the option
MICROPY_EMIT_INLINE_THUMB_FLOAT:

vcmp
vsqrt
vneg
vcvt_f32_to_s32
vcvt_s32_to_f32
vmrs
vmov
vldr
vstr
vadd
vsub
vmul
vdiv
2015-04-19 15:47:05 +01:00
Damien George d8cbbcaa9d py: Fix builtin ord so that it can handle bytes values >= 0x80.
Addresses issue #1188.
2015-04-19 12:26:46 +01:00
Damien George 1c9a499135 py/vstr.c: Allow vstr_printf to print correctly to a fixed buffer.
This patch allows vstr_printf to use all the available space of a fixed
vstr buffer.  vstr_printf is a good alternative to snprintf.
2015-04-18 14:23:13 +01:00
Damien George 044c473de2 py: Add %q format support to mp_[v]printf, and use it. 2015-04-16 14:30:16 +00:00
Damien George e72cda99fd py: Convert occurrences of non-debug printf to mp_printf. 2015-04-16 14:30:16 +00:00
Damien George 5ae5ec986e py: Make mp_sys_stdout_print object, wrapping sys.stdout for mp_print*.
So now all printing should go via either mp_plat_print or
mp_sys_stdout_print.
2015-04-16 14:30:16 +00:00
Damien George 7f9d1d6ab9 py: Overhaul and simplify printf/pfenv mechanism.
Previous to this patch the printing mechanism was a bit of a tangled
mess.  This patch attempts to consolidate printing into one interface.

All (non-debug) printing now uses the mp_print* family of functions,
mainly mp_printf.  All these functions take an mp_print_t structure as
their first argument, and this structure defines the printing backend
through the "print_strn" function of said structure.

Printing from the uPy core can reach the platform-defined print code via
two paths: either through mp_sys_stdout_obj (defined pert port) in
conjunction with mp_stream_write; or through the mp_plat_print structure
which uses the MP_PLAT_PRINT_STRN macro to define how string are printed
on the platform.  The former is only used when MICROPY_PY_IO is defined.

With this new scheme printing is generally more efficient (less layers
to go through, less arguments to pass), and, given an mp_print_t*
structure, one can call mp_print_str for efficiency instead of
mp_printf("%s", ...).  Code size is also reduced by around 200 bytes on
Thumb2 archs.
2015-04-16 14:30:16 +00:00
Paul Sokolovsky 56beb01724 objarray: Support assignment of bytes to bytearray slice. 2015-04-16 00:51:12 +03:00
Dave Hylands b21786947f py/formatfloat.c: Fix format of floating point numbers near 1.0.
In particular, numbers which are less than 1.0 but which
round up to 1.0.

This also makes those numbers which round up to 1.0 to
print with e+00 rather than e-00 for those formats which
print exponents.

Addresses issue #1178.
2015-04-12 13:06:20 +01:00
Paul Sokolovsky 8b7faa31e1 objstr: split(None): Fix whitespace properly. 2015-04-12 00:17:57 +03:00
Damien George 6eb7530083 py: In emitinlinethumb, use qstr_data instead of qstr_str and strlen. 2015-04-11 21:53:39 +01:00
Damien George 55fe92bb8f py: Remove old debugging printf's in compile.c. 2015-04-11 21:52:27 +01:00
Damien George b1bbe966c4 py: Combine load_attr and store_attr type methods into one (attr).
This simplifies the API for objects and reduces code size (by around 400
bytes on Thumb2, and around 2k on x86).  Performance impact was measured
with Pystone score, but change was barely noticeable.
2015-04-11 16:54:37 +01:00
Damien George 91bc32dc16 py: Provide typedefs for function types instead of writing them inline. 2015-04-09 15:31:53 +00:00
Damien George 4dea922610 py: Adjust some spaces in code style/format, purely for consistency. 2015-04-09 15:29:54 +00:00
stijn e50cff69bb py: Use a dummy type for referring to extern structs
Fixes msvc linker warnings about mismatching sizes between the mp_obj_fdfile_t
struct defined in file.c and the mp_uint_t declarations found in modsys.c and modbuiltins.c
2015-04-09 15:03:22 +02:00
stijn 36cc84a2a9 py: Fix msvc warning '*/ found outside of comment'
Also prevents some of the weaker syntax parsers out there treating the whole
'*/*const*/' part as a comment
2015-04-09 12:40:38 +02:00
Damien George 9988618e0e py: Implement full func arg passing for native emitter.
This patch gets full function argument passing working with native
emitter.  Includes named args, keyword args, default args, var args
and var keyword args.  Fully Python compliant.

It reuses the bytecode mp_setup_code_state function to do all the hard
work.  This function is slightly adjusted to accommodate native calls,
and the native emitter is forced a bit to emit similar prelude and
code-info as bytecode.
2015-04-07 22:43:28 +01:00
Damien George c9aa1883ed py: Simplify bytecode prelude when encoding closed over variables. 2015-04-07 00:08:17 +01:00
Damien George 78772ada0d py: Implement calling functions with *args in native emitter. 2015-04-06 22:48:21 +01:00
Paul Sokolovsky 282ca09f8e py: Add MICROPY_PY_BUILTINS_REVERSED, disable for minimal ports. 2015-04-07 00:17:11 +03:00
Paul Sokolovsky e2d44e30c7 py: Add MICROPY_PY_BUILTINS_ENUMERATE, disable for minimal ports. 2015-04-06 23:51:29 +03:00
Damien George 56606f3475 py: Implement delete for property and descriptors.
Without this patch deleting a property, or class with descriptor, will
call the setter with a NULL value and lead to a crash.
2015-04-04 20:15:31 +01:00
Damien George 0528c5a22a py: In str unicode, str_subscr will never be passed a bytes object. 2015-04-04 19:42:03 +01:00
Damien George 2801e6fad8 py: Some trivial cosmetic changes, for code style consistency. 2015-04-04 15:53:11 +01:00
Paul Sokolovsky 7f59b4b2ca objstr: Fix bugs introduced by inability to have shadow variables.
Warnings lead to programming errors - as expected.
2015-04-04 01:55:40 +03:00
Paul Sokolovsky acf6aec71c objstr: Avoid variable shadowing. 2015-04-04 01:24:59 +03:00
Paul Sokolovsky ac2f7a7f6a objstr: Add .splitlines() method.
splitlines() occurs ~179 times in CPython3 standard library, so was
deemed worthy to implement. The method has subtle semantic differences
from just .split("\n"). It is also defined as working for any end-of-line
combination, but this is currently not implemented - it works only with
LF line-endings (which should be OK for text strings on any platforms,
but not OK for bytes).
2015-04-04 00:09:48 +03:00
Damien George fa5950eb00 py: Fix bug in native emitter when closing over an argument. 2015-04-03 15:03:24 +00:00
Damien George 99957384ea py: Get native emitter working again with x86 (now supports closures). 2015-04-03 14:38:41 +00:00
Damien George 4cd9ced8dc py: Implement closures in native code generator.
Currently supports only x64 and Thumb2 archs.
2015-04-03 15:05:53 +01:00
Damien George 2cc5473021 py: Implement (non-compliant) support for delete_fast in native emitter.
This implementation is smaller (in code size) than #1024.
2015-04-03 14:29:30 +01:00
Damien George 12ab9eda8d py: Make heap printing compatible with 16-bit word size. 2015-04-03 14:11:13 +01:00
Damien George 3f327cc4c6 py: Allow MPZ_DIG_SIZE to be optionally configured by a port. 2015-04-03 14:11:13 +01:00
Damien George 567184e21e py: Allow configurable object representation, with 2 different options. 2015-04-03 14:11:13 +01:00
Damien George 12a5e17afb py: Add finer configuration of static funcs when not in stackless mode.
Also rename call_args_t to mp_call_args_t.
2015-04-02 22:56:58 +01:00
Paul Sokolovsky 7f1c98177b vm: Support strict stackless mode, with proper exception reporting.
I.e. in this mode, C stack will never be used to call a Python function,
but if there's no free heap for a call, it will be reported as
RuntimeError (as expected), not MemoryError.
2015-04-03 00:26:47 +03:00
Paul Sokolovsky f0a8f21190 vm: Implement stackless for CALL_FUNCTION_VAR_KW & CALL_METHOD_VAR_KW. 2015-04-03 00:03:07 +03:00
Paul Sokolovsky e6c6fe3275 runtime: Split mp_call_prepare_args_n_kw_var() from mp_call_method_n_kw_var().
Allow for reuse for stackless design, where preparing args is separate from
calling.
2015-04-03 00:03:07 +03:00
Paul Sokolovsky 390e92688c vm: Stackless support for MP_BC_CALL_METHOD. 2015-04-03 00:03:07 +03:00
Paul Sokolovsky 332a909d44 vm: If there's no heap to call function in stackless manner, call via C stack. 2015-04-03 00:03:07 +03:00
Paul Sokolovsky 2039757b85 vm: Initial support for calling bytecode functions w/o C stack ("stackless"). 2015-04-03 00:03:07 +03:00
Paul Sokolovsky f88eec0de2 makeqstrdata.py: Add support for strings with backslash escapes. 2015-04-02 01:10:11 +03:00
Damien George 2686f9b3e8 py: Fix emitnative's creation of small ints so it uses the macro. 2015-04-01 00:12:43 +01:00
Paul Sokolovsky 1bc534247c objtype: Add special unary methods __pos__, __neg__, __invert__.
Conditional on MICROPY_PY_ALL_SPECIAL_METHODS.
2015-03-31 01:05:03 +03:00
Paul Sokolovsky fdaac1dbf8 modbuiltins: round(): Accept second arg, and at least support it to be 0.
Per https://docs.python.org/3/library/functions.html#round, 2-args format
guaranteedly returns float.
2015-03-31 01:02:44 +03:00
stijn 28fa84b445 py: Add optional support for descriptors' __get__ and __set__ methods.
Disabled by default.  Enabled on unix and windows ports.
2015-03-26 23:55:14 +00:00
Damien George 51229afbde py: Increase fixed size of stack-info in native emitter.
This is a temporary fix.
2015-03-26 17:54:12 +00:00
Damien George 4112590a60 py, compiler: When just bytecode, make explicit calls instead of table.
When just the bytecode emitter is needed there is no need to have a
dynamic method table for the emitter back-end, and we can instead
directly call the mp_emit_bc_XXX functions.  This gives a significant
reduction in code size and a very slight performance boost for the
compiler.

This patch saves 1160 bytes code on Thumb2 and 972 bytes on x86, when
native emitters are disabled.

Overall savings in code over the last 3 commits are:

bare-arm: 1664 bytes.
minimal:  2136 bytes.
stmhal:    584 bytes (it has native emitter enabled).
cc3200:   1736 bytes.
2015-03-26 16:52:45 +00:00
Damien George a210c774f9 py, compiler: Remove emit_pass1 code, using emit_bc to do its job.
First pass for the compiler is computing the scope (eg if an identifier
is local or not) and originally had an entire table of methods dedicated
to this, most of which did nothing.  With changes from previous commit,
this set of methods can be removed and the methods from the bytecode
emitter used instead, with very little modification -- this is what is
done in this commit.

This factoring has little to no impact on the speed of the compiler
(tested by compiling 3763 Python scripts and timing it).

This factoring reduces code size by about 270-300 bytes on Thumb2 archs,
and 400 bytes on x86.
2015-03-26 16:52:45 +00:00
Damien George 542bd6b4a1 py, compiler: Refactor load/store/delete_id logic to reduce code size.
Saves around 230 bytes on Thumb2 and 750 bytes on x86.
2015-03-26 16:52:45 +00:00
Paul Sokolovsky 0090c714ba objdict: Cast mp_obj_t to concrete types explicitly.
Continuation of refactoring applied previously to objlist.
2015-03-26 12:28:56 +02:00
Damien George 44f65c0e2f py: Fix bug in compiler which allowed through illegal augmented assign.
It allowed such things as (a, b) += c.
2015-03-25 23:06:48 +00:00
Damien George 5e1d993f54 py: Clean up some logic in VM to remove assert(0)'s.
Saves around 30 bytes code on Thumb2 archs.
2015-03-25 22:20:37 +00:00
Damien George aedf583af2 py: Simplify some logic in compiler; add comments about CPython compat. 2015-03-25 22:06:47 +00:00
Paul Sokolovsky 3d598256df py: Cast mp_obj_t to concrete types explicitly.
mp_obj_t internal representation doesn't have to be a pointer to object,
it can be anything.

There's also a support for back-conversion in the form of MP_OBJ_UNCAST.
This is kind of optimization/status quo preserver to minimize patching the
existing code and avoid doing potentially expensive MP_OBJ_CAST over and
over. But then one may imagine implementations where MP_OBJ_UNCAST is very
expensive. But such implementations are unlikely interesting in practice.
2015-03-25 09:25:41 +02:00
Paul Sokolovsky 44cd46a7e4 objnamedtuple: Accept field list as a string.
This change is required to unbreak some CPython stdlib modules (as included
into micropython-lib).
2015-03-23 22:45:14 +02:00
Paul Sokolovsky 8705171233 objstr: Expose mp_obj_str_split() for reuse in other modules. 2015-03-23 22:43:37 +02:00
Paul Sokolovsky e38b892144 objnamedtuple: Check that 2nd arg to namedtuple() is a list. 2015-03-22 23:08:19 +02:00
Damien George 55b74d1ff5 py: Combine duplicated code that converts members from a lookup.
Despite initial guess, this code factoring does not hamper performance.
In fact it seems to improve speed by a little: running pystone(1.2) on
pyboard (which gives a very stable result) this patch takes pystones
from 1729.51 up to 1742.16.  Also, pystones on x64 increase by around
the same proportion (but it's much noisier).

Taking a look at the generated machine code, stack usage with this patch
is unchanged, and call is tail-optimised with all arguments in
registers.  Code size decreases by about 50 bytes on Thumb2 archs.
2015-03-21 14:21:54 +00:00
stijn 3cc17c69ff py: Allow retrieving a function's __name__.
Disabled by default.  Enabled on unix and stmhal ports.
2015-03-20 23:13:32 +00:00
Paul Sokolovsky 07b8dc68d6 runtime: mp_load_method_maybe(): Don't use confusing "base" term.
"Base" should rather refer to "base type"."Base object for attribute
lookup" should rather be just "object".

Also, a case of common subexpression elimination.
2015-03-21 00:59:39 +02:00
Paul Sokolovsky 69922c602c objlist: list_reverse(): Fix typesafety error. 2015-03-20 23:35:21 +02:00
Damien George d1cee02783 py: Clarify API for map/set lookup when removing&adding at once.
Addresses issue #1160.
2015-03-20 17:41:37 +00:00
Paul Sokolovsky 0ef01d0a75 py: Implement core of OrderedDict type.
Given that there's already support for "fixed table" maps, which are
essentially ordered maps, the implementation of OrderedDict just extends
"fixed table" maps by adding an "is ordered" flag and add/remove
operations, and reuses 95% of objdict code, just making methods tolerant
to both dict and OrderedDict.

Some things are missing so far, like CPython-compatible repr and comparison.

OrderedDict is Disabled by default; enabled on unix and stmhal ports.
2015-03-20 17:26:10 +00:00
Damien George 8e9a71257d py: Implement DELETE_GLOBAL in showbc.c. 2015-03-20 17:12:09 +00:00
Paul Sokolovsky 3425431370 objtype: More comment clarification for attribute lookup. 2015-03-20 00:51:55 +02:00
Damien George 2e2e404ff7 py: Allow to compile with extra warnings (sign-compare, unused-param). 2015-03-19 00:25:33 +00:00
Paul Sokolovsky 1954d8021f objtype: Clarify comment why we call mp_load_method_maybe() for native sub-obj. 2015-03-17 02:08:08 +02:00
Paul Sokolovsky 3cb766344d objtype: Refactor dealing with native sub-objects for clarity. 2015-03-16 14:00:01 +02:00
Paul Sokolovsky f0dc0d50e3 objtype: mp_obj_class_lookup: Remove implausible condition.
We already have branch for lookup->is_type == true, so here it's guaranteed
to be false.
2015-03-16 13:41:57 +02:00
Paul Sokolovsky 2b67a40fdb objtype: Clarify comment for mp_obj_class_lookup(). 2015-03-16 13:36:58 +02:00
Paul Sokolovsky 66c11ec581 objtype: Clarify code by consistently using common subexpression. 2015-03-16 13:36:58 +02:00
Damien George d4bd37a561 py: Fix printing of error message when parsing malformed integer. 2015-03-16 10:42:50 +00:00
Damien George 1ef26b35c1 py, extmod: Remove include of unnecessary system headers. 2015-03-14 23:11:25 +00:00
Damien George 836e46976f py: In pfenv_vprintf, adjust type from mp_uint_t to unsigned int. 2015-03-14 23:09:57 +00:00
Damien George fa1edff006 py: Remove unnecessary and unused sgn argument from pfenv_print_mp_int. 2015-03-14 22:32:40 +00:00
Damien George 6837d46c1d py: Fix builtin abs so it works for bools and bignum. 2015-03-14 22:07:30 +00:00
Damien George 42e0c59308 py: Add MICROPY_COMP_{DOUBLE,TRIPLE}_TUPLE_ASSIGN config options.
These allow to fine-tune the compiler to select whether it optimises
tuple assignments of the form a, b = c, d and a, b, c = d, e, f.
Sensible defaults are provided.
2015-03-14 13:11:35 +00:00
Damien George a77ffe66b2 py: In compiler, put macro guard around potentially unused asm vars. 2015-03-14 12:59:31 +00:00
Damien George 848dd0e762 py: Make some mpz functions static and remove unused ones. 2015-03-12 22:48:45 +00:00
Peter D. Gray b2a237d337 py: Add support for start/stop/step attributes of builtin range object. 2015-03-11 20:02:06 +00:00
Paul Sokolovsky cb0fc063ed objmemoryview: Introduce mp_obj_new_memoryview().
This follows existing pattern for object constructor API and allows to
create memoryview objects e.g. in external modules.
2015-03-06 21:37:28 +02:00
Paul Sokolovsky 24c1000741 objarray: Support array('O'), array of objects, as extension to CPython.
Might be useful at least for memoryview hacks.
2015-03-05 22:58:31 +02:00
Paul Sokolovsky 16b1f5e842 objarray: Fix typo in null TYPECODE_MASK. 2015-03-04 23:03:31 +02:00
Damien George d891452a73 py: Add MICROPY_MALLOC_USES_ALLOCATED_SIZE to allow simpler malloc API. 2015-03-03 21:23:13 +00:00
Paul Sokolovsky e104acdb8c runtime: Typo fixes in comments. 2015-03-03 21:37:50 +02:00
Damien George dc790977d4 py: In inline assembler, reset labels on code-size pass. 2015-03-03 17:34:49 +00:00
Damien George 3665d0b2ff py: Simplify some inline-assembler error messages, but retain meaning.
Just to reduce code size.  Messages are still to the point and
unambiguous.
2015-03-03 17:11:18 +00:00
Damien George 9c5cabb502 py: Give error for duplicate label in inline assembler. 2015-03-03 17:08:02 +00:00
stijn 803264bb17 py: Guard against redef of nlr_push with DEBUG + MICROPY_NLR_SETJMP. 2015-03-03 14:34:40 +00:00
Damien George 67c5f89af5 py: In inline assembler, fix branch out-of-range error reporting.
Should only give an error on the last pass of the assembler, since
that's when we are certain about the branch size.
2015-03-02 17:51:32 +00:00
Damien George db1e10d5ea py: Use SMALL_INT creation macro in builtin sum. 2015-03-02 17:19:44 +00:00
Damien George 9f142f0c84 py: For inline assembler, add bcc_n and bcc_w ops.
Addresses issue #1143.
2015-03-02 14:29:52 +00:00
Damien George a2e383820d py: Clean up and comment out unused functions in mpz. 2015-03-02 12:58:06 +00:00
Damien George 62a3a287d9 py: Set compiler scope before folding constants so error messages work.
Addresses issue #1140.
2015-03-01 12:04:05 +00:00
Damien George 7711afbb4a py: Combine complie functions for or_test/and_test to reduce code size.
Saves around 60 bytes code on Thumb2 archs.
2015-02-28 15:10:18 +00:00
Damien George 63f3832e81 py: Combine emit functions for jump true/false to reduce code size.
Saves 116 bytes for stmhal and 56 bytes for cc3200 port.
2015-02-28 15:04:06 +00:00
Damien George 0b2fd91890 py: Combine logic for compiling and/or tests, to reduce code size.
Reduces code size by 72 bytes on Thumb2 archs.
2015-02-28 14:37:54 +00:00
Paul Sokolovsky cefcbb22b2 objarray: Implement array slice assignment.
This is rarely used feature which takes enough code to implement, so is
controlled by MICROPY_PY_ARRAY_SLICE_ASSIGN config setting, default off.
But otherwise it may be useful, as allows to update arbitrary-sized data
buffers in-place.

Slice is yet to implement, and actually, slice assignment implemented in
such a way that RHS of assignment should be array of the exact same item
typecode as LHS. CPython has it more relaxed, where RHS can be any sequence
of compatible types (e.g. it's possible to assign list of int's to a
bytearray slice).

Overall, when all "slice write" features are implemented, it may cost ~1KB
of code.
2015-02-27 22:17:15 +02:00
Damien George 0bb971370b py: Transform assert logic in compiler to save code space.
Saves about 250 code bytes for Thumb2 archs.
2015-02-27 14:25:47 +00:00
Damien George 4d77e1a034 py: Use m_{new,renew,del} consistently.
This is so all memory requests go through the same interface.
2015-02-27 09:34:51 +00:00
Damien George 4852e09c79 py: Fix adding of traceback so that it appends to existing info.
This makes exception traceback info self contained (ie doesn't rely on
list object, which was a bit of a hack), reduces code size, and reduces
RAM footprint of exception by eliminating the list object.

Addresses part of issue #1126.
2015-02-27 00:36:39 +00:00
Damien George 12d6d777e1 py: Small optimisation of logic flow in BC_WITH_CLEANUP bytecode.
Slightly smaller code, and does not need to use C stack to save
temporaries.
2015-02-26 17:54:50 +00:00
Damien George 534574348e py: Make inline assembler raise exception when branch not in range.
Addresses issue #1132.
2015-02-25 15:45:55 +00:00
Damien George 993f067fa2 py: In inline assembler, add return statement to fix flow logic. 2015-02-24 22:43:01 +00:00
Damien George e5315f7ffd py: Factor some code in inline thumb assembler to reduce code size. 2015-02-24 16:35:37 +00:00
Damien George e41b21c01e py: Make more asmthumb functions inline to reduce code size. 2015-02-24 16:32:52 +00:00
Damien George 8f7976ba0d py: Reduce code size of inline thumb assembler by using static tables.
Reduces stmhal by about 300 bytes ROM.
2015-02-24 16:10:58 +00:00
nhtshot 5d323defe4 py: Update parse.c&mpconfig.h to reflect rename of mp_lexer_show_token.
This function is only used when DEBUG_PRINTERS and USE_RULE_NAME are
enabled.
2015-02-23 21:36:05 +00:00
Paul Sokolovsky 71ebd4b7f0 py: Implement UnicodeError.
Still too shy to implement UnicodeEncodeError which was really needed for
micropython-lib case.
2015-02-23 23:20:16 +02:00
Damien George 5cbeacebdb py: Make math special functions configurable and disabled by default.
The implementation of these functions is very large (order 4k) and they
are rarely used, so we don't enable them by default.

They are however enabled in stmhal and unix, since we have the room.
2015-02-22 14:48:18 +00:00
Paul Sokolovsky e06cf89f04 py: Add few more special methods. 2015-02-22 01:21:08 +02:00
Damien George eff359e114 py: Expose mp_obj_list_remove as a public function. 2015-02-21 14:47:02 +00:00
Paul Sokolovsky 8c705233f3 py: Fix mp_obj_print() to work when Python streams are not used. 2015-02-17 00:32:18 +02:00
Paul Sokolovsky 1f91e92cc6 py: Revamp mp_obj_print() to use Python streams.
Most of printing infrastructure now uses streams, but mp_obj_print() used
libc's printf(), which led to weird buffering issues in output. So, switch
mp_obj_print() to streams too, even though it may make sense to move it to
a separate file, as it is purely a debugging function now.
2015-02-17 00:13:01 +02:00
Damien George eff10f66a6 py: Implement bl/bx instructions for inline Thumb assembler. 2015-02-16 18:17:07 +00:00
Damien George 42495392da py: Implement "it" instruction for inline Thumb assembler. 2015-02-16 17:46:49 +00:00
Damien George 3d7bf5d4b1 py: More robust checking in inline assembler compiler. 2015-02-16 17:46:28 +00:00
Paul Sokolovsky 9e6c82960e builtinimport: Revamp&refactor handling of relative imports.
Relative imports are based of a package, so we're currently at a module
within a package, we should get to package first.

Also, factor out path travsering operation, but this broke testing for
boundary errors with relative imports. TODO: reintroduce them, together
with proper tests.
2015-02-16 12:11:34 +02:00
Paul Sokolovsky 078172dcab builtinimport: Improve debugging output. 2015-02-16 12:11:03 +02:00
Paul Sokolovsky e8432b3c72 stackctrl: Encode "recursion depth exceeded" message as qstr.
So corresponding exception can be thrown even under tight memory conditions.
2015-02-15 22:41:14 +03:00
Paul Sokolovsky fa3b895145 objexcept: Optimize traceback allocation for exception.
Traceback allocation for exception will now never lead to recursive
MemoryError exception - if there's no memory for traceback, it simply
won't be created.
2015-02-15 22:41:14 +03:00
Paul Sokolovsky 29c4f92e13 objexcept: Optimize using messages without formatting substitutions.
They are directly cast to str object, skipping allocation of formatting
buffer.
2015-02-15 22:41:14 +03:00
Paul Sokolovsky 3077fbff26 nlr: Add even more optional debugging logging.
Has to be enabled by manual editing, but at least it's there, as debugging
NLR issues may be weird.
2015-02-15 20:28:18 +03:00
Paul Sokolovsky e89cc13e5c nlr: If DEBUG, guard against recursive nlr_push().
Pushing same NLR record twice would lead to "infinite loop" in nlr_jump
(but more realistically, it will crash as soon as NLR record on stack is
overwritten).
2015-02-15 20:23:52 +03:00
Paul Sokolovsky 53e5e0fa28 py: Make old_globals part of mp_code_state structure.
Conceptually it is part of code state, so let it be allocated in the same way
as the rest of state.
2015-02-15 19:24:15 +03:00
Damien George e5039c6ff8 py: Use TextIOWrapper only if PY_IO_FILEIO def'd; cast size_t for print. 2015-02-15 13:17:11 +00:00
Damien George f6532bb9e0 py: Simplify and remove redundant code for __iter__ method lookup. 2015-02-15 01:10:13 +00:00
Damien George d1c3788375 py: Fix loading of immediate pointer in Thumb assembler.
Addresses issue #1117.
2015-02-15 00:45:28 +00:00
Damien George 0868942e77 py: Check for valid file when creating lexer for execfile.
Addresses issue #1119.
2015-02-15 00:02:27 +00:00
stijn 1b8e76b8e6 py: Cleanup duplication in instance_is_callable/instance_call. 2015-02-14 23:49:04 +00:00
stijn c1832fd206 py: Add setattr builtin. 2015-02-14 23:35:00 +00:00
Paul Sokolovsky 089c3f321e py/asm*.c: Typo fixes in comments. 2015-02-14 02:20:35 +08:00
Damien George 192d536fe4 py: Implement clz and rbit for inline Thumb assembler. 2015-02-13 11:06:23 +00:00
Damien George 32f0b7942c py: Implement sdiv/udiv for inline Thumb assembler. 2015-02-13 10:43:05 +00:00
Damien George 0d967b8ae4 py: Implement push/pop for inline Thumb assembler. 2015-02-13 02:30:35 +00:00
Damien George dfe944c3e5 py: Expose compile.c:list_get as mp_parse_node_extract_list. 2015-02-13 02:29:46 +00:00
Damien George 8dfbd2d589 py: Make inline assembler raise proper SyntaxError exception on error.
Also gives line number of location of error.  Very useful!
2015-02-13 01:00:51 +00:00
Damien George 1bf5a022fe py: Add ldrex and strex to thumb2 inline assembler.
These are useful for implementing atomic lock operations.
2015-02-12 22:52:42 +00:00
Damien George 596f41da04 py: Reuse value stack in VM WITH_CLEANUP opcode to reduce C-stack size.
Saves 8 bytes C-stack on stmhal and 16 bytes on unix x86.
2015-02-10 13:21:42 +00:00
Damien George ea0461dcd3 py: Add option to micropython.qstr_info() to dump actual qstrs. 2015-02-10 11:02:28 +00:00
Damien George d0df10b2c6 py: Don't unnecessarily create a bound method. 2015-02-09 16:59:15 +00:00
Damien George 48244044a2 py: Allow subclass of native object to delegate to the native buffer_p.
Addresses issue #1109.
2015-02-09 15:08:00 +00:00
Damien George 1a6721fefd py: Remove obsolete MP_F_LOAD_CONST_{INT,DEC} from emitnative.c. 2015-02-09 14:53:16 +00:00
Paul Sokolovsky 194117a066 objstr: Fix bytes creation from array of long ints. 2015-02-09 12:11:49 +08:00
Damien George 28631537bd py: Add MICROPY_OBJ_BASE_ALIGNMENT to help with 16-bit ports. 2015-02-08 13:42:00 +00:00
Damien George f804833a97 py: Initialise variables in mp_parse correctly, to satisfy gcc warning. 2015-02-08 13:40:20 +00:00
Damien George 7d414a1b52 py: Parse big-int/float/imag constants directly in parser.
Previous to this patch, a big-int, float or imag constant was interned
(made into a qstr) and then parsed at runtime to create an object each
time it was needed.  This is wasteful in RAM and not efficient.  Now,
these constants are parsed straight away in the parser and turned into
objects.  This allows constants with large numbers of digits (so
addresses issue #1103) and takes us a step closer to #722.
2015-02-08 01:57:40 +00:00
Damien George 5f97aaeca4 py: Fix instance lookup, since object is not a real type. 2015-02-08 00:42:13 +00:00
Damien George 0bfc7638ba py: Protect mp_parse and mp_compile with nlr push/pop block.
To enable parsing constants more efficiently, mp_parse should be allowed
to raise an exception, and mp_compile can already raise a MemoryError.
So these functions need to be protected by an nlr push/pop block.

This patch adds that feature in all places.  This allows to simplify how
mp_parse and mp_compile are called: they now raise an exception if they
have an error and so explicit checking is not needed anymore.
2015-02-07 18:33:58 +00:00
Damien George e1e359ff59 py: Put mp_sys_path, mp_sys_argv and gc_collected in mp_state_ctx_t.
Without mp_sys_path and mp_sys_argv in the root pointer section of the
state, their memory was being incorrectly collected by GC.
2015-02-07 17:24:10 +00:00
Damien George 744e767458 py: Make list.sort keep stack usage within O(log(N)) bound.
Also fix list.sort so it works with user-defined types, and parse the
keyword arguments properly.

Addresses issue #338.
2015-02-02 15:14:22 +00:00
Damien George f49782f005 py: Fix cmath.log10; fix printing of complex number with negative imag. 2015-02-02 12:52:14 +00:00
Paul Sokolovsky 98c4bc3fac py: Add MICROPY_PY_ALL_SPECIAL_METHODS and __iadd__ special method under it. 2015-01-31 00:35:56 +02:00
Damien George 32bade19d9 py: Convert CR to LF and CR LF to LF in lexer.
Only noticeable difference is how newlines are encoded in triple-quoted
strings.  The behaviour now matches CPython3.
2015-01-30 00:27:46 +00:00
Damien George 3da677e658 py: Implement Ellipsis object in native emitter. 2015-01-29 15:13:40 +00:00
Damien George 827b0f747b py: Change vstr_null_terminate -> vstr_null_terminated_str, returns str. 2015-01-29 13:57:23 +00:00
Damien George 81e70a88a7 py: Make REPL printing function print repr of object, not str.
Addresses issue #1014.
2015-01-28 23:53:13 +00:00
Damien George 0d3cb6726d py: Change vstr so that it doesn't null terminate buffer by default.
This cleans up vstr so that it's a pure "variable buffer", and the user
can decide whether they need to add a terminating null byte.  In most
places where vstr is used, the vstr did not need to be null terminated
and so this patch saves code size, a tiny bit of RAM, and makes vstr
usage more efficient.  When null termination is needed it must be
done explicitly using vstr_null_terminate.
2015-01-28 23:43:01 +00:00
Paul Sokolovsky bbd9251bac py: bytes(): Make sure we add values as bytes, not as chars. 2015-01-28 22:29:07 +02:00
Damien George 98e3a64694 py: Remove duplicated mp_obj_str_make_new function from objstrunicode.c. 2015-01-28 14:14:57 +00:00
Damien George 16677ce311 py: Be more precise about unicode type and disabled unicode behaviour. 2015-01-28 14:07:11 +00:00
David Steinberg 0b3014ce3a py: Add support for floats in mp_binary_{get,set}_val()
- This then provides support for floats in the struct package
2015-01-27 22:49:01 +02:00
Damien George a5efcd4745 py: Specify unary/binary op name in TypeError error message.
Eg, "() + 1" now tells you that __add__ is not supported for tuple and
int types (before it just said the generic "binary operator").  We reuse
the table of names for slot lookup because it would be a waste of code
space to store the pretty name for each operator.
2015-01-27 18:02:25 +00:00
Damien George a9dc9b8f6d py: Fix comparison of minus-zero long int. 2015-01-27 17:47:38 +00:00
Damien George 8788b131ec py: Check for NDEBUG using #ifdef rather than #if.
Defining NDEBUG (to any value, even 0) disables debugging.  Otherwise,
if it's not defined, debugging is enabled.
2015-01-25 18:35:54 +00:00
Damien George 32444b759a py: Don't use anonymous unions, name them instead.
This makes the code (more) compatible with the C99 standard.
2015-01-24 23:14:12 +00:00
Damien George 5c670acb1f py: Be more machine-portable with size of bit fields. 2015-01-24 23:12:58 +00:00
David Steinberg 0fb17f6ef4 py: Use float-to-int classifications for mp_obj_new_int_from_float() functions 2015-01-24 20:54:28 +00:00
David Steinberg ca377b10de py: Add float-to-int classification function 2015-01-24 20:54:28 +00:00
David Steinberg 8d427b7ab7 py: Fix issue in mpz_set_from_float() when mp_int_t is larger than float 2015-01-24 20:54:28 +00:00
David Steinberg c585ad1020 py: Move mp_float_t related defines to misc.h 2015-01-24 20:54:28 +00:00
stijn 6b636738b2 py: Fix segfault in namedtuple when name is a non-interned string
- namedtuple was wrongly using MP_OBJ_QSTR_VALUE instead of mp_obj_str_get_qstr,
so when passed a non-interned string it would segfault; fix this by using mp_obj_str_get_qstr
- store the namedtuple field names as qstrs so it is not needed to use mp_obj_str_get_qstr
everytime the field name has to be accessed. This also slighty increases performance when
fetching attributes
2015-01-24 20:21:09 +00:00
Paul Sokolovsky 91232d3850 binary: Rework array accessors. They work with native, not stdint types. 2015-01-24 03:18:33 +02:00
Paul Sokolovsky ca3dbb8d8b stream: readall(): Make sure there's a trailing NUL char. 2015-01-24 00:22:47 +02:00
Paul Sokolovsky 425f952a1e stream: Fix readall() implementation in respect to NUL terminator bytes.
After vstr refactor. Fixes #1084.
2015-01-23 17:59:37 +02:00
Paul Sokolovsky 344e15b1ae objstr: Remove code duplication and unbreak Windows build.
There was really weird warning (promoted to error) when building Windows
port. Exact cause is still unknown, but it uncovered another issue:
8-bit and unicode str_make_new implementations should be mutually exclusive,
and not built at the same time. What we had is that bytes_decode() pulled
8-bit str_make_new() even for unicode build.
2015-01-23 02:15:56 +02:00
Paul Sokolovsky 6113eb2f33 objstr*: Use separate names for locals_dict of 8-bit and unicode str's.
To somewhat unbreak -DSTATIC="" compile.
2015-01-23 02:05:58 +02:00
Damien George e5bcbcdebd py: Allow asmx64 to compile with -Wsign-compare.
See issue #699.
2015-01-22 14:08:58 +00:00
Damien George 8b77e3dd2f stmhal: Put mod_network_nic_list in global root-pointer state.
It needs to be scanned by GC.  Thanks to Daniel Campora.
2015-01-22 00:16:41 +00:00
Damien George 77089bebd4 py: Add comments for vstr_init and mp_obj_new_str. 2015-01-21 23:18:02 +00:00
Damien George 05005f679e py: Remove mp_obj_str_builder and use vstr instead.
With this patch str/bytes construction is streamlined.  Always use a
vstr to build a str/bytes object.  If the size is known beforehand then
use vstr_init_len to allocate only required memory.  Otherwise use
vstr_init and the vstr will grow as needed.  Then use
mp_obj_new_str_from_vstr to create a str/bytes object using the vstr
memory.

Saves code ROM: 68 bytes on stmhal, 108 bytes on bare-arm, and 336 bytes
on unix x64.
2015-01-21 23:18:02 +00:00
Damien George 0b9ee86133 py: Add mp_obj_new_str_from_vstr, and use it where relevant.
This patch allows to reuse vstr memory when creating str/bytes object.
This improves memory usage.

Also saves code ROM: 128 bytes on stmhal, 92 bytes on bare-arm, and 88
bytes on unix x64.
2015-01-21 23:17:27 +00:00
Paul Sokolovsky 8064892c9b builtinimport: Make sure that qstr is used properly to load frozen modules. 2015-01-21 23:14:46 +02:00
Damien George b6e6b5277f py: Implement proper re-raising in native codegen's finally handler.
This allows an exception to propagate correctly through a finally
handler.
2015-01-21 17:00:01 +00:00
Damien George 962a5d50c9 py: Implement __reversed__ slot.
Addresses issue #1073.
2015-01-21 00:19:42 +00:00
stijn bf19541f46 py: Prevent segfault for operations on closed StringIO.
Addresses issue #1067.
2015-01-20 23:50:43 +00:00
Damien George 50149a5730 py: Use mp_arg_check_num in some _make_new functions.
Reduces stmhal code size by about 250 bytes.
2015-01-20 14:11:27 +00:00
Damien George ff8dd3f486 py, unix: Allow to compile with -Wunused-parameter.
See issue #699.
2015-01-20 12:47:20 +00:00
Damien George 50912e7f5d py, unix, stmhal: Allow to compile with -Wshadow.
See issue #699.
2015-01-20 11:55:10 +00:00
Paul Sokolovsky 640e0b221e py: Implement very simple frozen modules support.
Only modules (not packages) supported now. Source modules can be converted
to frozen module structures using tools/make-frozen.py script.
2015-01-20 11:52:12 +02:00
Damien George 963a5a3e82 py, unix: Allow to compile with -Wsign-compare.
See issue #699.
2015-01-16 17:47:07 +00:00
Damien George 0abb5609b0 py: Remove unnecessary id_flags argument from emitter's load_fast.
Saves 24 bytes in bare-arm.
2015-01-16 12:24:49 +00:00
Paul Sokolovsky 87bc8e2b3d pyexec: Add event-driven variant pyexec_friendly_repl().
pyexec_friendly_repl_process_char() and friends, useful for ports which
integrate into existing cooperative multitasking system.

Unlike readline() refactor before, this was implemented in less formal,
trial&error process, minor functionality regressions are still known
(like soft&hard reset support). So, original loop-based pyexec_friendly_repl()
is left intact, specific implementation selectable by config setting.
2015-01-16 01:30:42 +02:00
Damien George d2d64f00fb py: Add "default" to switches to allow better code flow analysis.
This helps compiler produce smaller code.  Saves 124 bytes on stmhal and
bare-arm.
2015-01-14 21:32:42 +00:00
Damien George 65ef6b768c py: Only allocate strings/bytes once for load_const_obj. 2015-01-14 21:17:27 +00:00
Damien George d9dc6fff21 py: Allocate memory for assembled code at start of PASS_EMIT.
Previously was allocating at end of PASS_COMPUTE, and this pass was
being run twice, so memory was being allocated twice.
2015-01-14 00:38:33 +00:00
Damien George 1e1779eacf py: Reluctantly add an extra pass to bytecode compiler.
Bytecode also needs a pass to compute the stack size.  This is because
the state size of the bytecode function is encoded as a variable uint,
so we must know the value of this uint before we encode it (otherwise
the size of the generated code changes from one pass to the next).

Having an entire pass for this seems wasteful (in time).  Alternative is
to allocate fixed space for the state size (would need 3-4 bytes to be
general, when 1 byte is usually sufficient) which uses a bit of extra
RAM per bytecode function, and makes the code less elegant in places
where this uint is encoded/decoded.

So, for now, opt for an extra pass.
2015-01-14 00:20:28 +00:00
Damien George 2127e9a844 py, unix: Trace root pointers with native emitter under unix port.
Native code has GC-heap pointers in it so it must be scanned.  But on
unix port memory for native functions is mmap'd, and so it must have
explicit code to scan it for root pointers.
2015-01-14 00:11:09 +00:00
Damien George c935d69f74 py: Make compiler not crash when default except is not last. 2015-01-13 23:33:16 +00:00
Damien George d6ed6702f7 py/showbc.c: Handle new LOAD_CONST_OBJ opcode, and opcodes with cache. 2015-01-13 23:08:47 +00:00
Damien George 4c81ba8015 py: Never intern data of large string/bytes object; add relevant tests.
Previously to this patch all constant string/bytes objects were
interned by the compiler, and this lead to crashes when the qstr was too
long (noticeable now that qstr length storage defaults to 1 byte).

With this patch, long string/bytes objects are never interned, and are
referenced directly as constant objects within generated code using
load_const_obj.
2015-01-13 16:21:23 +00:00
Damien George dab1385177 py: Add load_const_obj to emitter, add LOAD_CONST_OBJ to bytecode.
This allows to directly load a Python object to the Python stack.  See
issue #722 for background.
2015-01-13 15:55:54 +00:00
Damien George bbf5cd01e3 py: Allow to compile with -Wstrict-prototypes. 2015-01-12 22:45:35 +00:00
Damien George 30d8a82220 py: Allow to compile with -Wredundant-decls. 2015-01-12 22:41:55 +00:00
Damien George abc1959e2c py, unix, lib: Allow to compile with -Wold-style-definition. 2015-01-12 22:34:38 +00:00
Damien George cd34207409 py: Can compile with -Wmissing-declarations and -Wmissing-prototypes. 2015-01-12 22:30:49 +00:00
Damien George 3dd1c0a88a py: Make a function static and comment out those not used. 2015-01-12 22:22:46 +00:00
Damien George 0178aa9a11 py, unix: Allow to compile with -Wdouble-promotion.
Ref issue #699.
2015-01-12 21:56:35 +00:00
Damien George 99ab64ffd4 py/makeqstrdata.py: Make it work again with both Python2 and Python3. 2015-01-11 22:40:38 +00:00
Damien George 95836f8439 py: Add MICROPY_QSTR_BYTES_IN_LEN config option, defaulting to 1.
This new config option sets how many fixed-number-of-bytes to use to
store the length of each qstr.  Previously this was hard coded to 2,
but, as per issue #1056, this is considered overkill since no-one
needs identifiers longer than 255 bytes.

With this patch the number of bytes for the length is configurable, and
defaults to 1 byte.  The configuration option filters through to the
makeqstrdata.py script.

Code size savings going from 2 to 1 byte:
- unix x64 down by 592 bytes
- stmhal down by 1148 bytes
- bare-arm down by 284 bytes

Also has RAM savings, and will be slightly more efficient in execution.
2015-01-11 22:27:30 +00:00
Damien George 6942f80a8f py: Add qstr cfg capability; generate QSTR_NULL and QSTR_ from script. 2015-01-11 22:06:53 +00:00
Damien George e233a55a29 py: Remove unnecessary BINARY_OP_EQUAL code that just checks pointers.
Previous patch c38dc3ccc7 allowed any
object to be compared with any other, using pointer comparison for a
fallback.  As such, existing code which checked for this case is no
longer needed.
2015-01-11 21:07:15 +00:00
Damien George c38dc3ccc7 py: Implement fallback for equality check for all types.
Return "not equal" for objects that don't implement equality check.
This is as per Python specs.
2015-01-11 15:13:18 +00:00
Damien George ec21405821 py: Add (commented out) code to gc_dump_alloc_table for qstr info. 2015-01-11 14:37:06 +00:00
Damien George 56e1f99ca1 py/makeqstrdata.py: Add more allowed qstr characters; escape quot. 2015-01-11 14:16:24 +00:00
Damien George 01418e9690 py: Fix hard-coded hash for empty qstr (was 0x0000 now 0x1505). 2015-01-11 14:15:45 +00:00
Damien George ddd1e18801 py: Add config option MICROPY_COMP_MODULE_CONST for module consts.
Compiler optimises lookup of module.CONST when enabled (an existing
feature).  Disabled by default; enabled for unix, windows, stmhal.
Costs about 100 bytes ROM on stmhal.
2015-01-10 14:07:24 +00:00
Damien George 58056b0f43 py: Fix handling of "0" mpz in some functions. 2015-01-09 20:58:58 +00:00
Damien George 5d48f234d2 py: Make mem_info print correct remaining stack bytes. 2015-01-09 20:37:49 +00:00
Damien George 89deec0bab py: Add MICROPY_PY_MICROPYTHON_MEM_INFO to enable mem-info funcs.
This allows to enable mem-info functions in micropython module, even if
MICROPY_MEM_STATS is not enabled.  In this case, you get mem_info and
qstr_info but not mem_{total,current,peak}.
2015-01-09 20:12:54 +00:00
Damien George 4a5895c4eb py: Disable stack checking by default; enable on most ports. 2015-01-09 00:10:55 +00:00
Damien George c33ecb83ba tests: Add test for when instance member overrides class member. 2015-01-08 17:48:44 +00:00
Damien George 5b7aa294e0 py: Fix nlr mp_state_ctx symbol error for Mac. 2015-01-08 16:24:44 +00:00
stijn afd6c8e1d2 Remove obsolete bss-related code/build features
GC for unix/windows builds doesn't make use of the bss section anymore,
so we do not need the (sometimes complicated) build features and code related to it
2015-01-08 15:29:44 +01:00
Damien George 7ee91cf861 py: Add option to cache map lookup results in bytecode.
This is a simple optimisation inspired by JITing technology: we cache in
the bytecode (using 1 byte) the offset of the last successful lookup in
a map. This allows us next time round to check in that location in the
hash table (mp_map_t) for the desired entry, and if it's there use that
entry straight away.  Otherwise fallback to a normal map lookup.

Works for LOAD_NAME, LOAD_GLOBAL, LOAD_ATTR and STORE_ATTR opcodes.

On a few tests it gives >90% cache hit and greatly improves speed of
code.

Disabled by default.  Enabled for unix and stmhal ports.
2015-01-07 21:07:23 +00:00
Damien George b4b10fd350 py: Put all global state together in state structures.
This patch consolidates all global variables in py/ core into one place,
in a global structure.  Root pointers are all located together to make
GC tracing easier and more efficient.
2015-01-07 20:33:00 +00:00
Damien George ad2307c92c py: Temporary fix for conversion of float to int when fits in small int.
Addresses issue #1044 (see also #1040).  Could do with a better fix.
2015-01-07 12:10:47 +00:00
Paul Sokolovsky d8bfd77ad5 showbc: Show conditional jump destination as unsigned value.
This is consistent with how BC_JUMP was handled before. We never show jumps
destinations relative to jump instrucion itself, only relative to beginning
of function. Another useful way to show them as absolute (real memory
address), and this change makes result expected and consistent with how
BC_JUMP is shown.
2015-01-07 00:29:15 +02:00
Paul Sokolovsky 343ca1e63a objarray: Make sure that longint works as bytearray size. 2015-01-04 17:19:16 +02:00
Paul Sokolovsky ed3b20aae8 modbuiltins.c: Fix NULL vs MP_OBJ_NULL usage. 2015-01-04 13:26:43 +02:00
Paul Sokolovsky ff8e35b42e objstr: Common subexpression elimination for vstr_str(field_name). 2015-01-04 13:23:44 +02:00
Paul Sokolovsky c114496641 objstr: Implement kwargs support for str.format(). 2015-01-04 00:26:31 +02:00
Damien George 6fd4b36bc5 py: Raise exception if trying to convert inf/nan to int. 2015-01-02 23:04:09 +00:00
David Steinberg 6e0b6d02db py: Fix float to int conversion for large exponents. 2015-01-02 22:31:41 +00:00
stijn fbfd3554fa msvc: Fix unresolved mp_arg_error_terse_mismatch since 7f23384
The compiler treats `if (MICROPY_ERROR_REPORTING == MICROPY_ERROR_REPORTING_TERSE)` as
a normal statement and generates assembly for it in degug mode as if MICROPY_ERROR_REPORTING
is an actual symbol instead of a preprocessor definition.
As such linking fails because mp_arg_error_terse_mismatch is not defined when
MICROPY_ERROR_REPORTING_TERSE is detailed or normal.
2015-01-02 16:53:54 +01:00
Damien George fd40a9c38e py: Make GC's STACK_SIZE definition a proper MICROPY_ config variable. 2015-01-01 22:04:46 +00:00
Damien George 872a82970d py: Fix windows external name error for nlr_top. 2015-01-01 22:03:44 +00:00
Damien George 8a2347723e py: Move global variable nlr_top to one place, in a .c file.
This reduces dependency on assembler, and allows to consolidate global
variables in the future.
2015-01-01 21:47:58 +00:00
Damien George 51dfcb4bb7 py: Move to guarded includes, everywhere in py/ core.
Addresses issue #1022.
2015-01-01 20:32:09 +00:00
Paul Sokolovsky db1ac360c3 emitnative: Disable warning in delete_fast for now (breaks test). 2015-01-01 22:09:18 +02:00
Paul Sokolovsky 8a8c1fc82f py: Add basic framework for issuing compile/runtime warnings. 2015-01-01 22:09:18 +02:00
Damien George ebde3c694f py: Add guarded includes for asm-based headers. 2015-01-01 18:07:43 +00:00
Damien George 84e0cf0d21 py: Change namedtuple error messages to reduce code size.
We are not word-for-word compatible with CPython exceptions, so we are
free to make them short but informative in order to reduce code size.
Also, try to make messages the same as existing ones where possible.
2015-01-01 15:43:25 +00:00
Damien George 7f23384d49 py: Make terse_arg_mismatch a global function and use it elsewhere.
Reduces code size when MICROPY_ERROR_REPORTING_TERSE is selected.
2015-01-01 15:33:50 +00:00
Paul Sokolovsky 276159e5dd objnamedtuple: Make sure to initialize type structure completely. 2015-01-01 15:31:51 +02:00
stijn 021dc44009 py: Allow keyword arguments for namedtuple 2015-01-01 14:53:23 +02:00
stijn 12340147b0 py: Use sequence of strings for named tuple initialization
- remove single string initialization style
- take list of strings instead
- store list in the type for fast lookup
2015-01-01 14:53:23 +02:00
Damien George e0ac194f4f py: Fix rshift and not of zero/one edge cases in mpz.
Addresses issue #1027.
2014-12-31 19:35:01 +00:00
Paul Sokolovsky 2c75665445 objstr: Fix %d-formatting of floats. 2014-12-31 02:21:19 +02:00
Paul Sokolovsky 5f68094e10 py: mp_obj_new_int_from_float() supported only for MICROPY_PY_BUILTINS_FLOAT. 2014-12-30 00:34:54 +02:00
Paul Sokolovsky f79cd6a233 py: Implement mp_obj_new_int_from_float() for MICROPY_LONGINT_IMPL_NONE. 2014-12-30 00:33:32 +02:00
Paul Sokolovsky 12033df511 py: Partially fix float to int conversion.
This fixes conversion when float type has more mantissa bits than small int,
and float value has small exponent. This is for example the case of 32-bit
platform using doubles, and converting value of time.time(). Conversion of
floats with larg exponnet is still not handled correctly.
2014-12-30 00:22:50 +02:00
Damien George 9ddbe291c4 py: Add include guards to mpconfig,misc,qstr,obj,runtime,parsehelper. 2014-12-29 01:02:19 +00:00
Damien George f89d659e3b py: In VM, for selective ip saving, store 1 byte past last opcode.
This is for efficiency, so we don't need to subtract 1 from the ip
before storing it to code_state->ip.  It saves a lot of ROM bytes on
unix and stmhal.
2014-12-29 00:29:59 +00:00
Paul Sokolovsky 23f1b5ff66 py: Add note about -fno-crossjumping when compiling vm.c. 2014-12-29 00:07:47 +00:00
Paul Sokolovsky 361909e3ca py: Add MP_LIKELY(), MP_UNLIKELY() macros to help branch prediction. 2014-12-29 00:51:24 +02:00
Paul Sokolovsky 1ee1785bed showbc: Print operation mnemonic in BINARY_OP. 2014-12-28 21:43:44 +02:00
Paul Sokolovsky df103462dc showbc: Make code object start pointer semi-public.
This allows to pring either absolute addresses or relative offsets in jumps
and code references.
2014-12-28 21:37:17 +02:00
Paul Sokolovsky 749575097f vm: Record exception ip only for instructions where exceptions may happen.
Mirroring ip to a volatile memory variable for each opcode is an expensive
operation. For quite a lot of often executed opcodes like stack manipulation
or jumps, exceptions cannot actually happen. So, record ip only for opcode
where that's possible.
2014-12-28 07:37:04 +02:00
Damien George 83204f3406 py: Allow to properly disable builtin slice operation.
This patch makes the MICROPY_PY_BUILTINS_SLICE compile-time option
fully disable the builtin slice operation (when set to 0).  This
includes removing the slice sytanx from the grammar.  Now, enabling
slice costs 4228 bytes on unix x64, and 1816 bytes on stmhal.
2014-12-27 17:33:30 +00:00
Damien George e37dcaafb4 py: Allow to properly disable builtin "set" object.
This patch makes MICROPY_PY_BUILTINS_SET compile-time option fully
disable the builtin set object (when set to 0).  This includes removing
set constructor/comprehension from the grammar, the compiler and the
emitters.  Now, enabling set costs 8168 bytes on unix x64, and 3576
bytes on stmhal.
2014-12-27 17:33:30 +00:00
Paul Sokolovsky 8ab6f90674 py: Move to guarded includes for compile.h and related headers. 2014-12-27 16:12:17 +02:00
Paul Sokolovsky 343266ea51 showbc: Refactor to allow inline instruction printing. 2014-12-27 05:01:21 +02:00
Damien George c55a4d82cf py: Make bytes objs work with more str methods; add tests. 2014-12-24 20:28:30 +00:00
Damien George 74eb44c392 py: Reduce size of VM exception stack element by 1 machine word.
This optimisation reduces the VM exception stack element (mp_exc_stack_t)
by 1 word, by using bit 1 of a pointer to store whether the opcode was a
FINALLY or WITH opcode.  This optimisation was pending, waiting for
maturity of the exception handling code, which has now proven itself.

Saves 1 machine word RAM for each exception (4->3 words per exception).
Increases stmhal code by 4 bytes, and decreases unix x64 code by 32
bytes.
2014-12-22 12:49:57 +00:00
Damien George 81836c28b3 py: Use str_to_int function in more places to reduce code size. 2014-12-21 21:07:03 +00:00
Damien George 01039b5bd8 py: Remove last uses of printf from compile; use proper SyntaxError. 2014-12-21 17:44:27 +00:00
Damien George 584ba6762f py: Move global/nonlocal decl code to compiler for proper SyntaxError.
This patch gives proper SyntaxError exceptions for bad global/nonlocal
declarations.  It also reduces code size: 304 bytes on unix x64, 132
bytes on stmhal.
2014-12-21 17:26:45 +00:00
Damien George b063b9b36d py: Fix iteration over map in 2 places. 2014-12-21 16:24:09 +00:00
Damien George 6efa66f125 py: Remove unnecessary RULE_none and PN_none from parser. 2014-12-20 18:41:59 +00:00
Damien George b47ea4eadd py: Add blank and ident flags to grammar rules to simplify parser.
This saves around 100 bytes code space on stmhal, more on unix.
2014-12-20 18:37:50 +00:00
Damien George 2870d85a11 py: Save a few code bytes in parser; make vars local where possible. 2014-12-20 18:06:08 +00:00
Damien George 2a3e2b9033 py: Add execfile function (from Python 2); enable in stmhal port.
Adds just 60 bytes to stmhal binary.  Addresses issue #362.
2014-12-19 13:36:17 +00:00
Paul Sokolovsky c0bc3bd736 asmarm: Fix bug with encoding small negative ints using MVN instruction. 2014-12-14 03:24:17 +02:00