Commit Graph

790 Commits

Author SHA1 Message Date
Damien George 824f5c5a32 py/vstr: Combine vstr_new_size with vstr_new since they are rarely used.
Now there is just one function to allocate a new vstr, namely vstr_new
(in addition to vstr_init etc).  The caller of this function should know
what initial size to allocate for the buffer, or at least have some policy
or config option, instead of leaving it to a default (as it was before).
2016-10-14 16:46:34 +11:00
Damien George 016dba0e98 unix: Use common RAISE_ERRNO macro from mphalport.h. 2016-10-07 14:09:59 +11:00
Damien George 503089ea9d unix: Use mp_raise_OSError helper function. 2016-10-07 14:05:15 +11:00
Paul Sokolovsky 9cc8ec843e py/py.mk: Add support for building modussl_mbedtls. 2016-09-23 14:30:46 +03:00
Damien George c4a69c75a5 unix: Enable btree module for coverage build. 2016-09-22 11:10:11 +10:00
Damien George 93c4a6a3f7 all: Remove 'name' member from mp_obj_module_t struct.
One can instead lookup __name__ in the modules dict to get the value.
2016-09-22 00:23:16 +10:00
Paul Sokolovsky 5bf1b4e9d9 unix/modjni: array(): Support creation of object arrays. 2016-09-18 13:37:40 +03:00
Paul Sokolovsky ee324c501e unix/modjni: Add array() top-level function to create Java array.
Takes element primitive type encoded as a char per standard JNI encoding,
and array size. TODO: Support object arrays.
2016-09-17 16:14:02 +03:00
Paul Sokolovsky 3fea1f014c unix/modjni: Implement subscription for object arrays. 2016-09-16 00:59:48 +03:00
Renato Aguiar 081c0648ec unix: Fix build for when MICROPY_PY_SOCKET=0. 2016-09-12 16:43:56 +10:00
Damien George 9526e24234 unix,stmhal,esp8266: When find'ing frozen files follow symbolic links.
It's useful to be able to use symbolic links to add files and directories
to the set of scripts to be frozen.
2016-09-05 12:35:05 +10:00
Delio Brignoli e2ac8bb3f1 py: Add MICROPY_USE_INTERNAL_PRINTF option, defaults to enabled.
This new config option allows to control whether MicroPython uses its own
internal printf or not (if not, an external one should be linked in).
Accompanying this new option is the inclusion of lib/utils/printf.c in the
core list of source files, so that ports no longer need to include it
themselves.
2016-09-05 12:18:53 +10:00
Damien George 2d8740a4d1 tests/extmod: Add a test for framebuf module, tested by coverage build. 2016-09-04 16:40:40 +10:00
Damien George 58f3861358 tests/unix/extra_coverage: Add test for str/bytes with invalid hash. 2016-09-02 15:07:42 +10:00
Delio Brignoli f98bb2ddcb py/mpprint: Fail an assertion with unsupported format specifiers.
Arguments of an unknown type cannot be skipped and continuing to parse a
format string after encountering an unknown format specifier leads to
undefined behaviour.  This patch helps to find use of unsupported formats.
2016-09-01 18:09:44 +10:00
Damien George 9c04ef2a67 unix,stmhal,esp8266: When find'ing frozen files don't use extra slash.
This extra forward slash for the starting-point directory is unnecessary
and leads to additional slashes on Max OS X which mean that the frozen
files cannot be imported.

Fixes #2374.
2016-08-31 15:12:57 +10:00
Paul Sokolovsky c428367543 extmod/modubinascii: Make crc32() support configurable.
Disable by default, enable in unix port.
2016-08-24 18:28:43 +03:00
Damien George 675d1c9c60 ports: Remove typedef of machine_ptr_t, it's no longer needed.
This type was used only for the typedef of mp_obj_t, which is now defined
by the object representation.  So we can now remove this unused typedef,
to simplify the mpconfigport.h file.
2016-08-15 11:02:59 +10:00
ilovezfs 4e36dd570b unix/mpconfigport.h: Don't include stdio.h on MacOS.
Fixes build errors such as
"../lib/utils/printf.c:43:5: error: expected parameter declarator"
2016-08-10 21:43:34 +03:00
Paul Sokolovsky 56eb25f049 py/objstr: Make .partition()/.rpartition() methods configurable.
Default is disabled, enabled for unix port. Saves 600 bytes on x86.
2016-08-07 06:46:55 +03:00
Paul Sokolovsky a4aaf82421 unix/moduselect: Allow poll.register(), etc. accept fd-like objects.
This includes file and socket objects, backed by Unix file descriptor.
This improves compatibility with stmhal's uselect (and convenience of
use), though not completely: return value from poll.poll() is still
raw file descriptor.
2016-08-07 01:25:59 +03:00
Paul Sokolovsky 5b9e7e29f6 unix: Enable websocket module. 2016-08-06 15:15:53 +03:00
Paul Sokolovsky aac9e8cfa3 unix/Makefile: And note why btree module is disabled for coverage build. 2016-07-31 02:27:13 +03:00
Paul Sokolovsky 61e77a4e88 py/mpconfig.h: Add MICROPY_STREAMS_POSIX_API setting.
To filter out even prototypes of mp_stream_posix_*() functions, which
require POSIX types like ssize_t & off_t, which may be not available in
some ports.
2016-07-30 20:05:56 +03:00
Paul Sokolovsky 64da62ec2e unix/file: Use generic stream flush() method. 2016-07-28 00:28:27 +03:00
Paul Sokolovsky 01816068c8 unix/file: fdfile_ioctl(): Fix argument to check_fd_is_open(). 2016-07-27 02:50:11 +03:00
Paul Sokolovsky ade36806c8 unix/file: ioctl(): Check that file is open before operations. 2016-07-27 00:30:09 +03:00
Paul Sokolovsky 8fac939889 unix/file: Implement MP_STREAM_FLUSH ioctl. 2016-07-27 00:24:09 +03:00
Paul Sokolovsky 243f8988be unix/mpconfigport.h: Include stdio.h by default.
This allows to use printf() in a any source file with unix port, for quick
debugging.
2016-07-26 02:51:25 +03:00
Paul Sokolovsky 918851e836 unix: Enable MICROPY_PY_STR_BYTES_CMP_WARN.
Also, fix a warning text (remove "duplicate" BytesWarning).
2016-07-22 00:52:07 +03:00
Paul Sokolovsky 68815901d4 unix: Disable MICROPY_GC_ALLOC_THRESHOLD for minimal build. 2016-07-21 00:37:53 +03:00
Paul Sokolovsky bd6622abe8 unix: Cache libaxtls.a in local build dir.
Allows to build the library variant for other ports in parallel.
2016-07-16 04:56:54 +03:00
Paul Sokolovsky 20283aec10 extmod/modussl_axtls: Further changes to allow alternative SSL modules.
Make variable MICROPY_SSL_AXTLS=1 should be defined to activate modussl_axtls
and link with -laxtls.
2016-07-13 01:49:38 +03:00
Damien George ee622cc1ed unix/mpthreadport: Adjust minimum thread stack, and stack limit check.
The minimum thread stack size is set by pthreads (16k bytes) so we must
use that value for our minimum.  The stack limit check is also adjusted
to work correctly for 32-bit builds.
2016-07-11 14:59:47 +00:00
Damien George 3096928d5a unix: Disable the GIL to improve performance of non-thread code.
Threading support is still very new so stay conservative at this point
and enable threading without the GIL.  This requires users to protect
concurrent access of mutatable Python objects (eg lists) with locks at
the Python level (something you should probably do anyway).  The
advantage is that there is less of a performance hit for non-threaded
code, because the VM does not need to constantly release/acquire the GIL.

In the future the GIL will be made more efficient.  There is also room to
improve the efficiency of non-GIL code by not using mutex's if there is
only one thread active.
2016-07-09 16:48:06 +01:00
Paul Sokolovsky 520f35632d unix/main: When preparing sys.path, allocate exact strings on uPy heap.
Due to the way modern compilers work (allocating space for stack vars once
at tha start of function, and deallocating once on exit from), using
intermediate stack buffer of big size caused blockage of 4K (PATH_MAX)
on stack for the entire duration of MicroPython execution.
2016-07-09 01:48:07 +03:00
Paul Sokolovsky 649b69a1db unix/main: Improve help for -X options a bit. 2016-07-08 20:53:08 +03:00
Paul Sokolovsky 0e5e14fe7c unix/main: Error out on unknown value of suffix in -X heapsize= option.
E.g. -X heapsize=16Kfoo, -X heapsize=1G will lead to error.
2016-07-08 20:49:41 +03:00
Paul Sokolovsky d83177b248 unix/mpconfigport_minimal.h: Allow to print a string within 1KB of heap.
By adjusting parser allocation policy.
2016-07-03 00:12:46 +03:00
Paul Sokolovsky e965d2b82a unix/Makefile: Make "minimal" build be minimal again. 2016-07-03 00:10:58 +03:00
Paul Sokolovsky 43241ceaac unix: Enable btree module.
But disable it for coverage build, as its extra warninsg aren't compatible
with K&R C BerkeleyDB uses.
2016-07-02 15:26:07 +03:00
Damien George df95f52583 py/modthread: Allow to properly set the stack limit of a thread.
We rely on the port setting and adjusting the stack size so there is
enough room to recover from hitting the stack limit.
2016-06-28 11:28:51 +01:00
Damien George 94d2919b77 unix/modtime: Release the GIL when sleeping. 2016-06-28 11:28:50 +01:00
Damien George f443504f86 unix/mpthreadport: Suppress compiler warning about unused arguments. 2016-06-28 11:28:50 +01:00
Damien George f1dbd2909e unix/gccollect: Provide declaration of exported function. 2016-06-28 11:28:50 +01:00
Damien George c2508ac8bd unix/mpthreadport: Use SA_SIGINFO for GC signal handler.
SA_SIGINFO allows the signal handler to access more information about
the signal, especially useful in a threaded environment.  The extra
information is not currently used but it may prove useful in the future.
2016-06-28 11:28:49 +01:00
Damien George dbd54e0b5b unix: Implement garbage collection with threading.
This patch allows any given thread to do a proper garbage collection and
scan all the pointers of all active threads.
2016-06-28 11:28:49 +01:00
Damien George f1b6db2218 unix/file: If write syscall returns because of EINTR then try again.
As per PEP-475.
2016-06-28 11:28:49 +01:00
Damien George 801d1b3803 py/modthread: Implement lock object, for creating a mutex. 2016-06-28 11:28:49 +01:00
Damien George 707f98f207 py/modthread: Add stack_size() function. 2016-06-28 11:28:48 +01:00
Damien George a791be936a unix: Add basic thread support using pthreads.
Has the ability to create new threads.
2016-06-28 11:28:48 +01:00
Damien George 330165a2cc py: Add MP_STATE_THREAD to hold state specific to a given thread. 2016-06-28 11:09:31 +01:00
Martin Müller f2892bd77d unix: Fix Makefile to handle gc-sections linker flags on Mac OS.
The linker flag --gc-sections is not available on the linker used on
Mac OS X which results in an error when linking micropython on Mac OS X.
Therefore move this option to the LDFLAGS_ARCH variable on non Darwin
systems. According to http://stackoverflow.com/a/17710056 the equivalent
to --gc-sections is -dead_strip thus this option is used for the
LDFLAGS_ARCH on Darwin systems.
2016-06-27 22:31:55 +01:00
Paul Sokolovsky e68811dbbf unix/modmachine: Enable time_pulse_us() function. 2016-06-19 00:21:17 +03:00
Paul Sokolovsky ad229477c6 unix/modmachine: Include PinBase class. 2016-06-18 23:40:28 +03:00
Paul Sokolovsky 07209f8592 all: Rename mp_obj_type_t::stream_p to protocol.
It's now used for more than just stream protocol (e.g. pin protocol), so
don't use false names.
2016-06-18 18:44:57 +03:00
Paul Sokolovsky cbffd0aadd unix: Disable FatFs VFS for normal build, keep enabled for coverage.
It's enabled mostly for unit testing, and we do that in full with coverage
build.
2016-06-18 00:58:57 +03:00
Paul Sokolovsky 7c8d76fad3 unix/alloc: Make coverage build and its overzealous warnings happy. 2016-06-18 00:36:13 +03:00
Paul Sokolovsky c9598604c6 unix/alloc: Add option to use uPy's alloc-exec implementation even for libffi.
When built for Linux, libffi includes very bloated and workaround exec-alloc
implementation required to work around SELinux and other "sekuritee" features
which real people don't use. MicroPython has own alloc-exec implementation,
used to alloc memory for @micropython.native code. With this option enabled,
uPy's implementation will override libffi's. This saves 11K on x86_64 (and
that accounts for more than half of the libffi code size).

TODO: Possibly, we want to refactor this option to allow either use uPy's
implementation even for libffi, or allow to use libffi's implementation even
for uPy.
2016-06-18 00:18:01 +03:00
Paul Sokolovsky 9b43a7d1be unix/Makefile: libffi: Build with -Os.
Also try to use -fno-exceptions. Other options taken from libffi's configure
defaults.
2016-06-18 00:17:47 +03:00
Paul Sokolovsky 6f8880d0ab unix: Move "utime" module config to C level instead of make level. 2016-06-17 23:35:00 +03:00
Paul Sokolovsky df453f0652 unix: Time to build with --gc-sections.
This actually saves "only" 6K for x86_64 build, as we're still more or less
careful to #ifdef unneeded code. But relying on --gc-sections in a "lazy"
manner would allow to make #ifdef'ing less pervasive (not suggested right
away, but an option for the future).
2016-06-17 02:38:20 +03:00
Paul Sokolovsky acaa30b604 unix: Deprecate support for GNU Readline (MICROPY_USE_READLINE=2).
MicroPython own readline implementation is superior now by providing
automatic indentation and completion (completion for GNU Readline was
never implemented). MICROPY_USE_READLINE=2 also wasn't build for a long
time and probably broken.

If GNU Readline is still beneficial for some cases, it can be achieved
with external wrappers like "rlwrap" (there will be the same level of
functionality, as again, there never was deep integration, like completion
support).
2016-06-16 03:28:58 +03:00
Paul Sokolovsky b5190f1df3 unix/mpconfigport_minimal.h: Clearly mark where user-define config ends.
TODO: Do the same for other config files.
2016-06-16 01:20:32 +03:00
Paul Sokolovsky cd796f85af unix: Unbreak "minimal" target by disabling FatFs.
Was broken since introduction of FatFs support.
2016-06-16 00:03:24 +03:00
Paul Sokolovsky 0f5bf1aafe py/mpconfig.h: MP_NOINLINE is universally useful, move from unix port. 2016-06-15 23:52:00 +03:00
Robert HH ee009d713a extmod/vfs_fat.c: Add vfs.stat().
The call to stat() returns a 10 element tuple consistent to the os.stat()
call. At the moment, the only relevant information returned are file
type and file size.
2016-05-31 13:03:12 +03:00
Paul Sokolovsky 4889b6ff9b unix/Makefile: "make axtls": Automatically fetch submodules if missing.
Try to emulate "you can build without reading instructions" behavior as far
as possible.
2016-05-28 21:03:48 +03:00
Paul Sokolovsky 6055d411a9 unix/mpconfigport.mk: Document MICROPY_STANDALONE make-level option.
Avoid using system libraries, use copies bundled with MicroPython as
submodules (currently affects only libffi, other dependencies either
already used as bundled-only (axtls), or can't be bundled (so far),
like libjni).
2016-05-28 20:38:14 +03:00
Paul Sokolovsky f5d63bcd23 unix: Enable "ussl" module.
ussl was in testing mode for 8 months, and now enabled in other ports
(e.g. esp8266), so time for unix port to catch up.
2016-05-26 02:12:11 +03:00
Paul Sokolovsky 37f9761801 unix/Makefile: nanbox build is not compatible with modussl, disable. 2016-05-26 02:11:17 +03:00
Damien George 2133924e46 unix: Support frozen packages.
To use, put your directory structure with .py files in frozen/ and then:

    make FROZEN_MPY_DIR=frozen
2016-05-23 15:19:53 +01:00
Paul Sokolovsky 1b5abfcaae py/objstr: Implement str.center().
Disabled by default, enabled in unix port. Need for this method easily
pops up when working with text UI/reporting, and coding workalike
manually again and again counter-productive.
2016-05-22 00:13:44 +03:00
Paul Sokolovsky b580958216 unix/unix_mphal: Implement mp_hal_ticks_us().
Similar to existing mp_hal_ticks_ms().
2016-05-21 02:16:35 +03:00
Paul Sokolovsky 92a342a011 unix/mpconfigport_coverage.h: Add dedicated config file for coverage build.
This allows to enable the options which aren't enabled in the normal unix
config (as unix port is no longer an enable-all port).
2016-05-18 00:58:32 +03:00
Damien George 1e024de7be unix: Add ability to include frozen bytecode in the build.
To use frozen bytecode make a subdirectory under the unix/ directory
(eg frozen/), put .py files there, then run:

    make FROZEN_MPY_DIR=frozen

Be sure to build from scratch.  The .py files will then be available for
importing.
2016-05-16 23:17:11 +01:00
Damien George 088127d91c unix: Enable uerrno module. 2016-05-10 23:30:39 +01:00
Pavol Rusnak bc7ca7ca01 unix/mphalport: Add mp_hal_delay_us() for consistency with other ports. 2016-05-07 21:18:44 +03:00
Martin Müller 31fc81d3b8 unix/Makefile: Make install more compatible (BSD, etc.).
The current install command uses the flag -D which is specific to the
install command from GNU coreutils, but isn't available for the BSD
version. This solution uses the -d flag which should be commonly
available to create the target directory. Afterwards the target files
are installed to this directory seperately.
2016-04-28 21:45:27 +03:00
Paul Sokolovsky 270dd29320 unix: Make sure build dir exists before accessing it for freezing upip. 2016-04-19 11:41:40 +03:00
Jan Čapek d76c65f599 py: Add rules for automated extraction of qstrs from sources.
- add template rule that converts a specified source file into a qstring file

- add special rule for generating a central header that contains all
  extracted/autogenerated strings - defined by QSTR_DEFS_COLLECTED
  variable. Each platform appends a list of sources that may contain
  qstrings into a new build variable: SRC_QSTR. Any autogenerated
  prerequisities are should be appened to SRC_QSTR_AUTO_DEPS variable.

- remove most qstrings from py/qstrdefs, keep only qstrings that
  contain special characters - these cannot be easily detected in the
  sources without additional annotations

- remove most manual qstrdefs, use qstrdef autogen for: py, cc3200,
  stmhal, teensy, unix, windows, pic16bit:

   - remove all micropython generic qstrdefs except for the special strings that contain special characters (e.g. /,+,<,> etc.)
   - remove all port specific qstrdefs except for special strings
   - append sources for qstr generation in platform makefiles (SRC_QSTR)
2016-04-16 13:18:09 +01:00
Paul Sokolovsky 9b0714b24c py: Declare help, input, open builtins in core.
These are *defined* per-port, but why redeclare them again and again.
2016-04-15 00:07:56 +03:00
Damien George 0a2e9650f5 py: Add ability to have frozen persistent bytecode from .mpy files.
The config variable MICROPY_MODULE_FROZEN is now made of two separate
parts: MICROPY_MODULE_FROZEN_STR and MICROPY_MODULE_FROZEN_MPY.  This
allows to have none, either or both of frozen strings and frozen mpy
files (aka frozen bytecode).
2016-04-13 16:07:47 +01:00
Paul Sokolovsky 47442d9f52 lib/utils/printf: Rework overriding printer of DEBUG_printf().
By default it uses mp_plat_print, but a port may override it to another
value with MICROPY_DEBUG_PRINTER_DEST.
2016-04-13 11:53:12 +03:00
Paul Sokolovsky 1a327c4fa3 unix: Build with MICROPY_PY_UHASHLIB_SHA1 if already building with axTLS. 2016-04-11 21:58:58 +03:00
pohmelie b32b0d38fe unix: freedos strip and size names for binaries
After this you need only one path for build (path/to/djgpp/bin). Original patch made by @dhylands
2016-04-08 20:15:37 +03:00
Paul Sokolovsky 1b811b946e unix/modsocket: Use mp_const_empty_map instead of creating empty map. 2016-04-04 23:43:16 +03:00
pohmelie cee888255b unix: djgpp errno.h have no ENOTSUP, so define it to Linux value. 2016-04-03 23:23:01 +03:00
Paul Sokolovsky a5d2af7949 unix/file: "encoding" arg to open() isn't kw-only.
And with "buffering" arg introduced, it's non possible to make it
non-kwonly.
2016-04-02 17:23:51 +03:00
Paul Sokolovsky e5fa163a4c unix/file: Parse "buffering" argument of open() builtin.
It's ignored (unbuffered, raw I/O is used), but least makes it compatible
with CPython.
2016-04-02 17:23:46 +03:00
Pavol Rusnak 3d4a535208 unix: implement -i option (inspect - start REPL after script is finished) 2016-04-01 12:35:45 +03:00
Paul Sokolovsky 13394a632d unix/unix_mphal: Hack to make uos.dupterm() actually work.
See https://github.com/micropython/micropython/issues/1736 for the
list of complications. This workaround instead of duplicating REPL
to another stream, switches to it, because read(STDIN) we use otherwise
is blocking call, so it and custom REPL stream can't be used together.
2016-03-27 14:02:03 +03:00
Paul Sokolovsky dc320164d8 unix/modsocket: Add comment regarding close() error checking (which is none). 2016-03-02 00:20:48 +02:00
Damien George ac23662550 unix: Enabled importing of persistent bytecode (.mpy files). 2016-02-25 10:12:30 +00:00
Paul Sokolovsky 46a0ac02c5 extmod/vfs_fat_ffconf: Reusable FatFs module, move from stmhal/ffconf.
TODO: Probably merge into vfs_fat_diskio.
2016-02-15 00:19:27 +02:00
Paul Sokolovsky 6b0c88256b extmod/vfs_fat_file: Reusable FatFs module, move from stmhal/file. 2016-02-15 00:16:46 +02:00
Paul Sokolovsky 8cb78e0e53 extmod/vfs_fat_diskio: Reusable FatFs module, move from stmhal/diskio. 2016-02-15 00:08:37 +02:00
Paul Sokolovsky 72085a669b py/mpstate.h: fs_user_mount is now standard, reusable uPy functionality. 2016-02-15 00:02:03 +02:00
Paul Sokolovsky 8a43a41b3a unix: Enable VfsFat support. 2016-02-14 19:15:22 +02:00
Paul Sokolovsky a63a4761cd unix/file: Stop assuming that O_RDWR == O_RDONLY | O_WRONLY.
That's not true e.g. on Linux.
2016-02-06 21:59:44 +02:00