Commit Graph

309 Commits

Author SHA1 Message Date
Paul Sokolovsky c880f91ac0 extmod/modwebrepl: Add readinto() method. 2016-07-02 14:05:37 +03:00
Paul Sokolovsky 03b0279c6d extmod/modwebsocket: Add readinto() method. 2016-07-02 14:03:49 +03:00
Paul Sokolovsky 2f7ebf16de extmod/modbtree: Cleverly implement "for key in btree:" syntax.
I.e. make it work like btree.keys(), while still not using a separate
iterator type.
2016-06-23 20:08:37 +03:00
Paul Sokolovsky 6b088a671a extmod/modbtree: Implement keys(), values(), items() iterators.
Each takes optional args of starting key, ending key, and flags (ending
key inclusive, reverse order).
2016-06-20 15:50:31 +03:00
Paul Sokolovsky 5071ceea07 extmod/modlwip: Store a chain of incoming pbufs, instead of only one.
Storing a chain of pbuf was an original design of @pfalcon's lwIP socket
module. The problem with storing just one, like modlwip does is that
"peer closed connection" notification is completely asynchronous and out of
band. So, there may be following sequence of actions:

1. pbuf #1 arrives, and stored in a socket.
2. pbuf #2 arrives, and rejected, which causes lwIP to put it into a
queue to re-deliver later.
3. "Peer closed connection" is signaled, and socket is set at such status.
4. pbuf #1 is processed.
5. There's no stored pbufs in teh socket, and socket status is "peer closed
connection", so EOF is returned to a client.
6. pbuf #2 gets redelivered.

Apparently, there's no easy workaround for this, except to queue all
incoming pbufs in a socket. This may lead to increased memory pressure,
as number of pending packets would be regulated only by TCP/IP flow
control, whereas with previous setup lwIP had a global overlook of number
packets waiting for redelivery and could regulate them centrally.
2016-06-19 19:15:20 +03:00
Paul Sokolovsky efee577318 extmod/machine_pinbase: Fix nanbox build.
MP_ROM_PTR() is only for data structures initialization, code should use
MP_OBJ_FROM_PTR().
2016-06-19 00:26:57 +03:00
Paul Sokolovsky 3fecbb2462 extmod/machine_pinbase: Implementation of PinBase class.
Allows to translate C-level pin API to Python-level pin API. In other
words, allows to implement a pin class and Python which will be usable
for efficient C-coded algorithms, like bitbanging SPI/I2C, time_pulse,
etc.
2016-06-18 23:40:28 +03:00
Paul Sokolovsky 07209f8592 all: Rename mp_obj_type_t::stream_p to protocol.
It's now used for more than just stream protocol (e.g. pin protocol), so
don't use false names.
2016-06-18 18:44:57 +03:00
Paul Sokolovsky 080137961d extmod/modbtree: open(): Support "in-memory" database with filename=None.
It's not really in-memory though, just uses anonymous temporary file on
disk.
2016-06-18 01:31:57 +03:00
Paul Sokolovsky e9739e3315 extmod/modbtree: __getitem__() should raise KeyError for non-existing key. 2016-06-18 01:30:49 +03:00
Paul Sokolovsky e6e7e0e9c5 extmod/modbtree: items(): Implement DESC flag. 2016-06-18 00:47:26 +03:00
Paul Sokolovsky 332545baa3 extmod/modbtree: items(): Implement "end key inclusive" flag. 2016-06-17 00:08:55 +03:00
Robert HH 23067a1422 esp8266: Use RTC to set date & time stamps for files.
The time stamp is taken from the RTC for all newly generated
or changed files. RTC must be maintained separately.
The dummy time stamp of Jan 1, 2000 is set in vfs.stat() for the
root directory, avoiding invalid time values.
2016-06-16 19:31:58 +03:00
Paul Sokolovsky d0416ff915 extmod/modbtree: Actually implement end key support for .items(). 2016-06-16 18:16:33 +03:00
Paul Sokolovsky 1babeb47a4 extmod/modbtree: Implement .items() iterator. 2016-06-16 17:31:24 +03:00
Paul Sokolovsky 422396cece extmod/modbtree: Handle default value and error check. 2016-06-15 04:18:44 +03:00
Paul Sokolovsky 8072162170 extmod/modbtree: Initial implementation of "btree" module based on BerkeleyDB.
This implements basic wrapping of native get/put/seq API, and then dictionary
access protocol. Native API is intended to be superceded going forward.
2016-06-14 21:51:59 +03:00
Damien George 33168081f4 extmod/machine: Add MICROPY_PY_MACHINE_PULSE config for time_pulse_us.
Since not all ports that enable the machine module have the pin HAL
functions.
2016-05-31 14:25:19 +01:00
Damien George 4940bee62a extmod: Add machine time_pulse_us function (at C and Python level).
The C implementation is taken from the DHT driver.
2016-05-31 13:58:48 +01:00
Paul Sokolovsky 298c2ae2c7 extmod/vfs_fat: Mark anused "self" arg for fat_vfs_stat(). 2016-05-31 15:42:08 +03:00
Robert HH ee009d713a extmod/vfs_fat.c: Add vfs.stat().
The call to stat() returns a 10 element tuple consistent to the os.stat()
call. At the moment, the only relevant information returned are file
type and file size.
2016-05-31 13:03:12 +03:00
Paul Sokolovsky 480159ca8b extmod/vfs_fat: getcwd(): Use mp_obj_new_exception_arg1().
Copy-paste issue, with the original mistake in stmhal.
2016-05-29 20:04:53 +03:00
Paul Sokolovsky ee5e3f6527 extmod/vfs_fat: chdir(), getcwd() methods should accept VFS object (self). 2016-05-29 18:52:41 +03:00
Paul Sokolovsky cac6c9704e extmod/vfs_fat: Add getcwd() method.
Ported from stmhal.
2016-05-29 18:23:59 +03:00
Paul Sokolovsky f12146cbf6 extmod/vfs_fat: Add chdir() method.
Ported from stmhal.
2016-05-29 18:17:00 +03:00
Robert HH 7c004e794c extmod/vfs_fat*: Replace text error messages by POSIX error numbers.
These changes are in line with similar changes in other modules, and
with standard Python interface.
2016-05-27 23:57:01 +03:00
Paul Sokolovsky 751e3b7a82 extmod/virtpin: Initial implementation of open-ended C-level Pin interface.
Using usual method of virtual method tables. Single virtual method,
ioctl, is defined currently for all operations. This universal and
extensible vtable-based method is also defined as a default MPHAL
GPIO implementation, but a specific port may override it with its
own implementation (e.g. close-ended, but very efficient, e.g. avoiding
virtual method dispatch).
2016-05-27 01:08:43 +03:00
Damien George 4b37e775ea extmod/machine_i2c: Redo mp_hal_pin macros to use open_drain and od_low.
mp_hal_pin_config_od is renamed mp_hal_pin_open_drain, and mp_hal_pin_low
is mp_hal_pin_od_low.
2016-05-26 17:06:40 +01:00
Paul Sokolovsky 2ea019f3d3 extmod/modussl: Coverage build fixes. 2016-05-26 02:11:24 +03:00
Paul Sokolovsky 9d04fec5d9 extmod/modussl: Make more compatible with non-default obj representations.
Still not compatible with nanboxing.
2016-05-26 02:10:35 +03:00
Paul Sokolovsky 0ab372585f extmod/moduos_dupterm: Dumpterm subsystem is responsible for closing stream.
Make dupterm subsystem close a term stream object when EOF or error occurs.
There's no other party than dupterm itself in a better position to do this,
and this is required to properly reclaim stream resources, especially if
multiple dupterm sessions may be established (e.g. as networking
connections).
2016-05-20 22:20:37 +03:00
Paul Sokolovsky 3a29db8e58 extmod/modwebrepl: Add close() method. 2016-05-20 21:32:41 +03:00
Paul Sokolovsky ccf4e5ab7b extmod/modwebsocket: Add close() method. 2016-05-20 21:18:49 +03:00
Damien George f9dc644017 extmod: When including extmod headers, prefix path with extmod/. 2016-05-20 12:46:20 +01:00
Paul Sokolovsky 7f7c84b10a py/stream: Support both "exact size" and "one underlying call" operations.
Both read and write operations support variants where either a) a single
call is made to the undelying stream implementation and returned buffer
length may be less than requested, or b) calls are repeated until requested
amount of data is collected, shorter amount is returned only in case of
EOF or error.

These operations are available from the level of C support functions to be
used by other C modules to implementations of Python methods to be used in
user-facing objects.

The rationale of these changes is to allow to write concise and robust
code to work with *blocking* streams of types prone to short reads, like
serial interfaces and sockets. Particular object types may select "exact"
vs "once" types of methods depending on their needs. E.g., for sockets,
revc() and send() methods continue to be "once", while read() and write()
thus converted to "exactly" versions.

These changes don't affect non-blocking handling, e.g. trying "exact"
method on the non-blocking socket will return as much data as available
without blocking. No data available is continued to be signaled as None
return value to read() and write().

From the point of view of CPython compatibility, this model is a cross
between its io.RawIOBase and io.BufferedIOBase abstract classes. For
blocking streams, it works as io.BufferedIOBase model (guaranteeing
lack of short reads/writes), while for non-blocking - as io.RawIOBase,
returning None in case of lack of data (instead of raising expensive
exception, as required by io.BufferedIOBase). Such a cross-behavior
should be optimal for MicroPython needs.
2016-05-18 02:41:45 +03:00
Paul Sokolovsky afce978aca extmod/modlwip: Rework how Python accept callback is called.
Calling it from lwIP accept callback will lead incorrect functioning
and/or packet leaks if Python callback has any networking calls, due
to lwIP non-reentrancy. So, instead schedule "poll" callback to do
that, which will be called by lwIP when it does not perform networking
activities. "Poll" callback is called infrequently though (docs say
every 0.5s by default), so for better performance, lwIP needs to be
patched to call poll callback soon after accept callback, but when
current packet is already processed.
2016-05-15 22:42:12 +03:00
Damien George 0055457b5f extmod/modlwip: Convert errno's to use MP_Exxx symbols. 2016-05-12 12:49:14 +01:00
Paul Sokolovsky cfc94bec9f extmod/modlwip: Implement sendall() method for TCP sockets. 2016-05-03 10:43:11 +03:00
Paul Sokolovsky b8468d12a1 extmod/modwebrepl: Get rid of using strncpy(). 2016-05-02 20:52:34 +03:00
Paul Sokolovsky 859e4e94f3 extmod/modwebrepl: Add support for password.
Request for password then becomes mandatory part of the protocol.
2016-04-30 20:36:32 +03:00
Paul Sokolovsky 18775d3807 extmod/modwebrepl: Set debugging by default to off.
That's production setting. Also, extra UART output may affect behavior of
(subpar) network drivers.
2016-04-29 19:17:37 +03:00
Paul Sokolovsky b0f3ae58e7 extmod/modwebrepl: Add rate-limiting workaround for broken network drivers.
Like ESP8266 has.
2016-04-29 19:14:03 +03:00
Paul Sokolovsky 8811b0af9c extmod/modwebrepl: Use bigger socket receive buffer.
The smaller chunks we send (and receive), the more packets there to
receive, and higher chance to git internal packet buffer overflow in
WiFi driver.
2016-04-29 18:43:19 +03:00
Paul Sokolovsky b3bc2ee1b9 extmod/modwebrepl: More detailed debug output.
So detailed that even commented by default.
2016-04-29 17:37:40 +03:00
Paul Sokolovsky 473b639845 extmod/modwebrepl: GET_FILE: Send length-prefix chunk with one write().
A bit of optimization.
2016-04-29 17:35:21 +03:00
Paul Sokolovsky f41e1f1bb7 extmod/modwebrepl: Keep reading data when there's something to read.
EAGAIN should be returned only if underlying socket returned it. Wrap
existing read function into external loop to process all data available.
2016-04-29 01:05:02 +03:00
Paul Sokolovsky 6514ff6160 extmod/modwebrepl: Initial implementation of "get file" operation. 2016-04-29 01:02:39 +03:00
Paul Sokolovsky 25d0f7d59d extmod/modwebrepl: Module to handle WebREPL protocol.
While just a websocket is enough for handling terminal part of WebREPL,
handling file transfer operations requires demultiplexing and acting
upon, which is encapsulated in _webrepl class provided by this module,
which wraps a websocket object.
2016-04-29 00:52:52 +03:00
Paul Sokolovsky ba61480df5 extmod/modussl: SSL_OK from ssl_read() means "no user data so far".
SSL_OK is numeric 0, and it's *not* an EOF. So, should keep reading.
2016-04-28 17:29:11 +03:00
Paul Sokolovsky 2534bfdb92 extmod/modussl: Support server-side SSL sockets.
wrap_socket(sock, server_side=True)
2016-04-28 17:27:20 +03:00
Paul Sokolovsky d54290f6e2 extmod/modussl: Throw Python exceptions in case of errors. 2016-04-28 00:49:54 +03:00
Paul Sokolovsky 480c212009 extmod/modwebsocket: Handle CLOSE control frame.
This fixes situation when clients hangs waiting for disconnect and does
so only on timeout.
2016-04-27 12:49:30 +03:00
Paul Sokolovsky 1c6d91d968 extmod/modlwip: Add print_pcbs() debug function.
This requires lwIP built with LWIP_DEBUG (or it will be no-op).
2016-04-26 16:30:13 +03:00
Damien George 45ac5a85d5 extmod/modlwip: Workaround esp8266 sendto issue where 1 is returned. 2016-04-26 13:19:08 +01:00
Damien George a63542387d extmod, stmhal: Fix typo of macro that detects if float is enabled. 2016-04-26 12:47:24 +01:00
Paul Sokolovsky 7a012f4793 extmod/modlwip: Protect recv/accept Python callback against exceptions.
Using usual call_function_*_protected() helper, to avoid NLR jump crashes.
2016-04-25 21:27:55 +03:00
Damien George 624738ca64 extmod/machine_i2c: Allow mp_hal_pin_obj_t to be any type, not a ptr. 2016-04-22 09:56:02 +01:00
Paul Sokolovsky 7d57037906 extmod/modlwip: Add ability to run callback on "recv" and "accept" events.
To use: .setsockopt(SOL_SOCKET, 20, lambda sock: print(sock)). There's a
single underlying callback slot. For normal sockets, it serves as data
received callback, for listening sockets - connection arrived callback.
2016-04-17 18:06:45 +03:00
Paul Sokolovsky 8007f84cca extmod/modlwip: lwip_tcp_receive(): Full error handling. 2016-04-17 02:22:26 +03:00
Paul Sokolovsky b830f4c610 extmod/modlwip: lwip_tcp_send(): Full error handling. 2016-04-17 02:20:05 +03:00
Damien George bcd719ea3a extmod/fsusermount: In mount/mkfs, deregister VFS object on error.
Should fix issue #1947.
2016-04-14 23:36:25 +01:00
Paul Sokolovsky 5c1af60e19 extmod/modlwip: More debug messages for various edge conditions. 2016-04-15 00:37:12 +03:00
Paul Sokolovsky 44ab5c3ef1 extmod/modlwip: Start adding debug output. 2016-04-14 01:15:52 +03:00
Paul Sokolovsky fef0d9818a extmod/modlwip: lwip_tcp_receive(): Properly handle EOF for non-blocking sock. 2016-04-14 00:59:09 +03:00
Paul Sokolovsky 59a4fee516 extmod/modwebsocket: Another case to propagate EOF. 2016-04-13 22:17:09 +03:00
Paul Sokolovsky 54ea253f56 extmod/moduos_dupterm: Don't swallow exceptions in dupterm's read()/write().
The idea is that if dupterm object can handle exceptions, it will handle
them itself. Otherwise, object state can be compromised and it's better
to terminate dupterm session. For example, disconnected socket will keep
throwing exceptions and dump messages about that.
2016-04-13 16:34:11 +03:00
Damien George eec8a94f04 extmod/machine_i2c: Implement I2C memory reading/writing. 2016-04-12 15:52:17 +01:00
Damien George 9314b2df4f extmod/machine_i2c: Fix I2C reading by sending ack/nack at end of byte. 2016-04-12 15:46:13 +01:00
Damien George d083712224 extmod: Add generic machine.I2C class, with bit-bang I2C.
Should work on any machine that provides the correct pin functions.
2016-04-12 14:06:54 +01:00
Damien George 53ad681ed1 extmod: Add initial framebuf module. 2016-04-12 14:06:53 +01:00
Paul Sokolovsky f8fb4470a0 extmod/modwebsocket: write(): Support write size beyond 125 bytes. 2016-04-11 14:07:57 +03:00
Paul Sokolovsky 7063210014 extmod/modlwip: Fix for loss of data in unaccepted incoming sockets.
When lwIP creates a incoming connection socket of a listen socket, it
sets its recv callback to one which discards incoming data. We set
proper callback only in accept() call, when we allocate Python-level
socket where we can queue incoming data. So, in lwIP accept callback
be sure to set recv callback to one which tells lwIP to not discard
incoming data.
2016-04-11 01:21:34 +03:00
Paul Sokolovsky b69f798c92 extmod/modwebsocket.h: Split websocket-related defines for reuse. 2016-04-10 13:42:51 +03:00
Paul Sokolovsky d6236e85c2 extmod/modwebsocket: Implement MP_STREAM_SET_DATA_OPTS ioctl.
Allows to set fragment type (txt/bin/etc.) for output records.
2016-04-10 13:19:26 +03:00
Paul Sokolovsky 6837dba6b8 extmod/modwebsocket: Allow to get type of last read data using ioctl(). 2016-04-10 12:50:46 +03:00
Paul Sokolovsky f38e8f5217 extmod/modwebsocket: Record current fragment type (binary/text/etc.)
Also, handle continuation frames (untested).
2016-04-09 16:14:47 +03:00
Paul Sokolovsky 5b1c221785 extmod/modwebsocket: Add option for blocking writes to non-blk sockets.
This is strange asymmetry which is sometimes needed, e.g. for WebREPL: we
want to process only available input and no more; but for output, we want
to get rid of all of it, because there's no other place to buffer/store
it. This asymmetry is akin to CPython's asyncio asymmetry, where reads are
asynchronous, but writes are synchronous (asyncio doesn't expect them to
block, instead expects there to be (unlimited) buffering for any sync write
to completely immediately).
2016-04-09 16:03:38 +03:00
Paul Sokolovsky 397b705647 extmod/modwebsocket: Reset mask between packets. 2016-04-09 12:29:18 +03:00
Paul Sokolovsky 7e9182f3aa extmod/modwebsocket: Make sure to propagate EOF. 2016-04-08 20:26:54 +03:00
Tom Sparks 20d9bc2d76 extmod/modure: re_exec() renamed to ure_exec() due to collison in 4.3BSD.
Addresses issue #1972.
2016-04-08 20:12:03 +03:00
Paul Sokolovsky 05ba2433f6 extmod/modwebsocket: Properly check number of args to constructor. 2016-04-08 16:05:48 +03:00
Paul Sokolovsky 651a188299 extmod/vfs_fat_diskio: Actually support sectors != 512 with Python blockdevs. 2016-03-27 14:21:06 +03:00
Paul Sokolovsky 91031a75a1 extmod/modlwip: lwip_socket_setsockopt: Handle option value properly. 2016-03-25 20:53:52 +02:00
Paul Sokolovsky 4332d72fd8 extmod/modlwip: Add lwip->POSIX error map for lwIP 1.4.0.
Between 1.4.0 and 1.4.1, lwIP errors were renumbered.
2016-03-25 20:32:01 +02:00
Paul Sokolovsky de0c84ebf1 extmod/modlwip: lwip_tcp_send: Handle properly send buffer full condition.
Per POSIX http://pubs.opengroup.org/onlinepubs/9699919799/functions/send.html :
"If space is not available at the sending socket to hold the message to be
transmitted, and the socket file descriptor does not have O_NONBLOCK set,
send() shall block until space is available. If space is not available at the
sending socket to hold the message to be transmitted, and the socket file
descriptor does have O_NONBLOCK set, send() shall fail [with EAGAIN]."
2016-03-25 18:38:13 +02:00
Paul Sokolovsky 5e75f335e6 extmod/modlwip: Implement setsocketopt(SO_REUSEADDR). 2016-03-25 17:44:24 +02:00
Paul Sokolovsky 8f1854ad2d extmod/modlwip: Add SOL_SOCKET and SO_REUSEADDR constants for setsockopt(). 2016-03-25 15:06:12 +02:00
Paul Sokolovsky 3dbd2ee926 extmod/modwebsocket: Implement read support. 2016-03-25 00:51:51 +02:00
Paul Sokolovsky 24342dd65e extmod/modwebsocket: Start module for WebSocket helper functions.
Currently, only write support is implemented (of limited buffer size).
2016-03-24 19:16:00 +02:00
Damien George b894551772 extmod/uctypes: Change param type from void* to byte*. 2016-03-19 22:13:17 +00:00
Damien George da161fd9f0 extmod/uctypes: Finish support for FLOAT32 and FLOAT64 types. 2016-03-19 21:59:42 +00:00
Damien George 12154b1774 extmod/uctypes: Use mp_binary_get_val helper when extracting value.
It handles more cases than mp_binary_get_int.
2016-03-19 21:41:01 +00:00
Damien George 0be6359f39 py: When printf'ing an object as a pointer, pass the concrete pointer. 2016-03-14 22:41:14 +00:00
Paul Sokolovsky 0779409d0d extmod/modlwip: lwip_tcp_receive: Properly map lwIP error to POSIX errno. 2016-03-12 11:42:15 +07:00
Paul Sokolovsky ba8f7d5171 extmod/modlwip: Add socket.setblocking() method. 2016-03-12 10:52:50 +07:00
Paul Sokolovsky 09ed5bcbbb extmod/modlwip: Rework getaddrinfo() data passing.
The code is based on Damien George's implementation for esp8266 port,
avoids use of global variables and associated re-entrancy issues, and
fixes returning stale data in some cases.
2016-03-12 10:50:51 +07:00
Paul Sokolovsky e5b047369b extmod/modlwip: Use MICROPY_EVENT_POLL_HOOK for event polling if defined.
Instead of just delaying 100ms if event isn't yet ready.

So far applies only to default, "infinite" socket timeout.
2016-03-11 09:32:07 +07:00
Paul Sokolovsky 43d497592f extmod/modlwip: Factor out "socket connected" check to a function.
Same code repeated for each send*() and recv*() function.
2016-03-09 12:43:09 +07:00
Paul Sokolovsky fda874e406 extmod/modlwip: Support non-blocking recv(). 2016-03-09 12:39:33 +07:00
Paul Sokolovsky 7379be3673 extmod/modlwip: Add .write() stream method. 2016-03-09 12:35:43 +07:00