Commit Graph

2425 Commits

Author SHA1 Message Date
Damien George e36ff98c80 py/parse: Add uerrno to list of modules to look for constants in. 2016-05-10 23:30:39 +01:00
Damien George 596a3feb8f py: Add uerrno module, with errno constants and dict. 2016-05-10 23:30:39 +01:00
Damien George 3f56fd64b8 py: Add mperrno.h file with uPy defined errno constants. 2016-05-10 23:30:39 +01:00
Paul Sokolovsky 6f34e138f1 py/vstr: Change allocation policy, +16 to requested size, instead of *2.
Effect measured on esp8266 port:

Before:
>>> pystone_lowmem.main(10000)
Pystone(1.2) time for 10000 passes = 44214 ms
This machine benchmarks at 226 pystones/second
>>> pystone_lowmem.main(10000)
Pystone(1.2) time for 10000 passes = 44246 ms
This machine benchmarks at 226 pystones/second

After:
>>> pystone_lowmem.main(10000)
Pystone(1.2) time for 10000 passes = 44343ms
This machine benchmarks at 225 pystones/second
>>> pystone_lowmem.main(10000)
Pystone(1.2) time for 10000 passes = 44376ms
This machine benchmarks at 225 pystones/second
2016-05-10 00:56:51 +03:00
Paul Sokolovsky 40f0096ee7 Revert "py/objstr: .format(): Avoid call to vstr_null_terminated_str()."
This reverts commit 6de8dbb488. The change
was incorrect (correct change would require comparing with end pointer in
each if statement in the block).
2016-05-09 23:42:42 +03:00
Paul Sokolovsky a1f2245a81 py/vstr: vstr_null_terminated_str(): Extend string by at most one byte.
vstr_null_terminated_str is almost certainly a vstr finalization operation,
so it should add the requested NUL byte, and not try to pre-allocate more.
The previous implementation could actually allocate double of the buffer
size.
2016-05-09 22:39:57 +03:00
Paul Sokolovsky 6de8dbb488 py/objstr: .format(): Avoid call to vstr_null_terminated_str().
By comparing with string end pointer instead of checking for NUL byte.
Should alleviate reallocations and fragmentation a tiny bit.
2016-05-09 21:55:09 +03:00
Damien George 460b086333 py/mpz: Fix mpn_div so that it doesn't modify memory of denominator.
Previous to this patch bignum division and modulo would temporarily
modify the RHS argument to the operation (eg x/y would modify y), but on
return the RHS would be restored to its original value.  This is not
allowed because arguments to binary operations are const, and in
particular might live in ROM.  The modification was to normalise the arg
(and then unnormalise before returning), and this patch makes it so the
normalisation is done on the fly and the arg is now accessed as read-only.

This change doesn't increase the order complexity of the operation, and
actually reduces code size.
2016-05-09 17:21:42 +01:00
Damien George 65402ab1ec py/mpz: Do Python style division/modulo within bignum divmod routine.
This patch consolidates the Python logic for division/modulo to one place
within the bignum code.
2016-05-08 22:21:21 +01:00
Damien George dc3faea040 py/mpz: Fix bug with overflowing C-shift in division routine.
When DIG_SIZE=32, a uint32_t is used to store limbs, and no normalisation
is needed because the MSB is already set, then there will be left and
right shifts (in C) by 32 of a 32-bit variable, leading to undefined
behaviour.  This patch fixes this bug.
2016-05-08 21:38:43 +01:00
Paul Sokolovsky d59c2e5e45 py/repl: If there're no better alternatives, try to complete "import".
Also do that only for the first word in a line. The idea is that when you
start up interpreter, high chance that you want to do an import. With this
patch, this can be achieved with "i<tab>".
2016-05-08 20:40:47 +03:00
Damien George 470c429ee1 py/runtime: Properly handle passing user mappings to ** keyword args. 2016-05-07 22:02:46 +01:00
Damien George 12dd8df375 py/objstr: Binary type of str/bytes for buffer protocol is 'B'.
The type is an unsigned 8-bit value, since bytes objects are exactly
that.  And it's also sensible for unicode strings to return unsigned
values when accessed in a byte-wise manner (CPython does not allow this).
2016-05-07 21:18:17 +01:00
Damien George eb54e4d065 py/obj: Add warning note about get_array return value and GC blocks. 2016-05-04 10:19:08 +01:00
Paul Sokolovsky 9549590fc6 py/modcollections: Rename module name have "u" prefix for consistency. 2016-05-02 13:57:46 +03:00
Paul Sokolovsky ddb9dba2f7 py/modio: Rename module name to "uio" for consistency with other modules. 2016-05-02 13:56:33 +03:00
Paul Sokolovsky 25d0f7d59d extmod/modwebrepl: Module to handle WebREPL protocol.
While just a websocket is enough for handling terminal part of WebREPL,
handling file transfer operations requires demultiplexing and acting
upon, which is encapsulated in _webrepl class provided by this module,
which wraps a websocket object.
2016-04-29 00:52:52 +03:00
Paul Sokolovsky eff85bb1dc py/vm: "yield from" didn't handle MP_OBJ_STOP_ITERATION optimization.
E.g. crashed when yielding from already stopped generators.
2016-04-28 02:08:43 +03:00
Paul Sokolovsky 51cee4495e py/mkrules.mk: Typo fixes in comments. 2016-04-26 12:39:28 +03:00
Damien George 23df4b08fb py/emitnative: Use MP_OBJ_NEW_SMALL_INT instead of manual bit shifting. 2016-04-26 10:02:32 +01:00
Damien George 2bddfd4922 py/obj.h: When constructing a small-int cast to mp_uint_t for bit-shift.
The C standard says that left-shifting a signed value (on the LHS of the
operator) is undefined.  So we cast to an unsigned integer before the
shift.  gcc does not issue a warning about this, but clang does.
2016-04-26 09:51:37 +01:00
stijn 9264d42e2a py/makeqstrdefs.py: Windows compatibility.
- msvc preprocessor output contains full paths with backslashes so the
  ':' and '\' characters needs to be erased from the paths as well
- use a regex for extraction of filenames from preprocessor output so it
  can handle both gcc and msvc preprocessor output, and spaces in paths
  (also thanks to a PR from @travnicekivo for part of that regex)
- os.rename will fail on windows if the destination file already exists,
  so simply attempt to delete that file first
2016-04-25 22:34:22 +01:00
stijn b2b771ca02 py/makeqstrdefs.py: Remove unused function/variable/import. 2016-04-25 22:34:20 +01:00
Paul Sokolovsky bababce6de py/runtime_utils: Fix nanbox build. 2016-04-25 20:03:14 +03:00
Paul Sokolovsky 6d103b6548 py: Move call_function_*_protected() functions to py/ for reuse.
They almost certainly needed by any C code which calls Python callbacks.
2016-04-25 19:31:17 +03:00
Damien George 51dca54cd0 py/mkrules.mk: Remove obsolete rules for auto qstr generation. 2016-04-22 11:36:19 +01:00
Damien George 109990fc32 py/mkenv.mk: Remove -s and -S args from PYTHON variable.
Qstr auto-generation is now much faster so this optimisation for start-up
time is no longer needed.  And passing "-s -S" breaks some things, like
stmhal's "make deploy".
2016-04-21 22:25:35 +01:00
Damien George fea40ad468 py: Fix bug passing a string as a keyword arg in a dict.
Addresses issue #1998.
2016-04-21 16:51:36 +01:00
Paul Sokolovsky 1b60a6dc4e py: Divide "split" and "cat" phases of qstr extraction for better efficiency.
E.g. for stmhal, accumulated preprocessed output may grow large due to
bloated vendor headers, and then reprocessing tens of megabytes on each
build make take couple of seconds on fast hardware (=> potentially dozens
of seconds on slow hardware). So instead, split once after each change,
and only cat repetitively (guaranteed to be fast, as there're thousands
of lines involved at most).
2016-04-19 14:39:08 +03:00
Paul Sokolovsky 8dd704b019 py/makeqstrdefs.py: Process only CPP line-numbering info.
Not stuff like "#pragma", etc.
2016-04-19 12:52:57 +03:00
Paul Sokolovsky 4494b521ea py/mkrules.mk: Fix Bashism. 2016-04-19 12:32:23 +03:00
Paul Sokolovsky 21ab304c41 py/mkrules.mk: Cleanup command passed to shell. 2016-04-19 12:28:30 +03:00
Paul Sokolovsky 0dc85c9f86 py/mkrules.mk: Try to detect and emulate make -B behavior for qstr extraction.
If make -B is run, the rule is run with $? empty. Extract fron all file in
this case. But this gets fragile, really "make clean" should be used instead
with such build complexity.
2016-04-19 12:18:46 +03:00
Paul Sokolovsky c618f91e22 py: Rework QSTR extraction to work in simple and obvious way.
When there're C files to be (re)compiled, they're all passed first to
preprocessor. QSTR references are extracted from preprocessed output and
split per original C file. Then all available qstr files (including those
generated previously) are catenated together. Only if the resulting content
has changed, the output file is written (causing almost global rebuild
to pick up potentially renumbered qstr's). Otherwise, it's not updated
to not cause spurious rebuilds. Related make rules are split to minimize
amount of commands executed in the interim case (when some C files were
updated, but no qstrs were changed).
2016-04-19 11:37:56 +03:00
Paul Sokolovsky 50e4fa5e19 py/mkenv.mk: Optimize Python startup type during make process.
By skipping loading site.py, etc.
2016-04-17 16:11:44 +03:00
Damien George 31480fb91b py/frozenmod: Pass the source name of the frozen module to the lexer.
This allows for better error messages, since the name of the file (sans
.py) can now be printed when an exception occurs within a frozen script.
2016-04-17 12:37:00 +01:00
Jan Čapek 53e3770b15 py/mkrules.mk: Suppress line-no output from CPP for qstr auto-gen. 2016-04-16 13:19:35 +01:00
Jan Čapek 000eae121c py/py.mk: Add makefile variable for qstr autogeneration control.
- any architecture may explicitely build with qstring make
  QSTR_AUTOGEN_DISABLE=1 autogeneration disabled and provide its
  own list of qstrings by the standard
  mechanisms (qstrdefsport.h).
2016-04-16 13:19:23 +01:00
Jan Čapek 61b560f63f py/mkrules.mk: Add mpconfig[port].h dependency to qstr generating rule. 2016-04-16 13:18:51 +01:00
Jan Čapek d76c65f599 py: Add rules for automated extraction of qstrs from sources.
- add template rule that converts a specified source file into a qstring file

- add special rule for generating a central header that contains all
  extracted/autogenerated strings - defined by QSTR_DEFS_COLLECTED
  variable. Each platform appends a list of sources that may contain
  qstrings into a new build variable: SRC_QSTR. Any autogenerated
  prerequisities are should be appened to SRC_QSTR_AUTO_DEPS variable.

- remove most qstrings from py/qstrdefs, keep only qstrings that
  contain special characters - these cannot be easily detected in the
  sources without additional annotations

- remove most manual qstrdefs, use qstrdef autogen for: py, cc3200,
  stmhal, teensy, unix, windows, pic16bit:

   - remove all micropython generic qstrdefs except for the special strings that contain special characters (e.g. /,+,<,> etc.)
   - remove all port specific qstrdefs except for special strings
   - append sources for qstr generation in platform makefiles (SRC_QSTR)
2016-04-16 13:18:09 +01:00
Pavel Moravec dbbf082786 py/makeqstrdefs: Add script to automate extraction of qstr from sources.
This script will search for patterns of the form Q(...) and generate a
list of them.

The original code by Pavel Moravec has been significantly simplified to
remove the part that searched for C preprocessor directives (eg #if).
This is because all source is now run through CPP before being fed into
this script.
2016-04-16 13:13:52 +01:00
Damien George 00137b8c11 py/map: Change hash-table allocation policy to be less aggressive.
Small hash tables (eg those used in user class instances that only have a
few members) now only use the minimum amount of memory necessary to hold
the key/value pairs.  This can reduce performance for instances that have
many members (because then there are many reallocations/rehashings of the
table), but helps to conserve memory.

See issue #1760.
2016-04-15 16:24:46 +01:00
Paul Sokolovsky 9b0714b24c py: Declare help, input, open builtins in core.
These are *defined* per-port, but why redeclare them again and again.
2016-04-15 00:07:56 +03:00
Damien George a649d72606 py/makeqstrdata: Add special case to handle \n qstr. 2016-04-14 15:22:36 +01:00
Damien George 2243d68345 py/makeqstrdata: Reinstate Python2 compatibility. 2016-04-14 14:37:04 +01:00
Damien George 49bb04ee64 py/makeqstrdata: Fix rendering of qstrs that have non-printable ASCII.
The qstr data needs to be turned into a proper C string so non-ASCII
chars must be properly escaped according to C rules.
2016-04-14 14:20:25 +01:00
Damien George 0c1de1cdee py: Simplify "and" action within parser by making ident-rules explicit.
Most grammar rules can optimise to the identity if they only have a single
argument, saving a lot of RAM building the parse tree.  Previous to this
patch, whether a given grammar rule could be optimised was defined (mostly
implicitly) by a complicated set of logic rules.  With this patch the
definition is always specified explicitly by using "and_ident" in the rule
definition in the grammar.  This simplifies the logic of the parser,
making it a bit smaller and faster.  RAM usage in unaffected.
2016-04-14 13:49:23 +01:00
Damien George f30b6f0af5 py/makeqstrdata: Add more names for escaped chars and esc non-printable.
Non-printable characters are escaped as 0xXX, where XX are the hex
digits of the character value.
2016-04-13 22:12:39 +01:00
Damien George 0a2e9650f5 py: Add ability to have frozen persistent bytecode from .mpy files.
The config variable MICROPY_MODULE_FROZEN is now made of two separate
parts: MICROPY_MODULE_FROZEN_STR and MICROPY_MODULE_FROZEN_MPY.  This
allows to have none, either or both of frozen strings and frozen mpy
files (aka frozen bytecode).
2016-04-13 16:07:47 +01:00
Damien George 594fa73411 py/makeqstrdata: Factor out some code to functions that can be reused. 2016-04-13 16:05:43 +01:00