Commit Graph

66 Commits

Author SHA1 Message Date
Andrew Leech bbc5a18d09 tools/mpremote: Add ioctl to specify large read buffer size.
Speeds up importing files from mounted filesystem.

Also fix the return code for invalid / unsupported ioctl requests.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
2023-11-09 11:30:11 +11:00
Jim Mussared bbd8760bd9 all: Update Python formatting to ruff-format.
This updates a small number of files that change with ruff-format's (vs
black's) rules.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-11-03 13:30:41 +11:00
Jim Mussared 69e34b6b6b all: Switch to new preview build versioning scheme.
See https://github.com/micropython/micropython/issues/12127 for details.

Previously at the point when a release is made, we update mpconfig.h
and set a git tag. i.e. the version increments at the release.

Now the version increments immediately after the release. The workflow is:
1. Final commit in the cycle updates mpconfig.h to set (X, Y, 0, 0) (i.e.
   clear the pre-release state).
2. This commit is tagged "vX.Y.0".
3. First commit for the new cycle updates mpconfig.h to set (X, Y+1, 0, 1)
   (i.e. increment the minor version, set the pre-release state).
4. This commit is tagged "vX.Y+1.0-preview".

The idea is that a nightly build is actually a "preview" of the _next_
release. i.e. any documentation describing the current release may not
actually match the nightly build. So we use "preview" as our semver
pre-release identifier.

Changes in this commit:
 - Add MICROPY_VERSION_PRERELEASE to mpconfig.h to allow indicating that
   this is not a release version.
 - Remove unused MICROPY_VERSION integer.
 - Append "-preview" to MICROPY_VERSION_STRING when the pre-release state
   is set.
 - Update py/makeversionhdr.py to no longer generate MICROPY_GIT_HASH.
 - Remove the one place MICROPY_GIT_HASH was used (it can use
   MICROPY_GIT_TAG instead).
 - Update py/makeversionhdr.py to also understand
   MICROPY_VERSION_PRERELEASE in mpconfig.h.
 - Update py/makeversionhdr.py to convert the git-describe output into
   semver-compatible "X.Y.Z-preview.N.gHASH".
 - Update autobuild.sh to generate filenames using the new scheme.
 - Update remove_old_firmware.py to match new scheme.
 - Update mpremote's pyproject.toml to handle the "-preview" suffix in the
   tag. setuptools_scm maps to this "rc0" to match PEP440.
 - Fix docs heading where it incorrectly said "vvX.Y.Z" for release docs.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-10-06 12:10:14 +11:00
Jos Verlinde 58c2c503a9 tools/mpremote: Add support for rfc2217, serial over TCP.
Signed-off-by: Jos Verlinde <jos_verlinde@hotmail.com>
2023-09-29 14:40:17 +10:00
Jim Mussared 4bbe879c26 tools/mpremote: Make soft-reset count as an action.
Otherwise `mpremote soft-reset` will implicitly run the repl command.

Fixes issue #10871.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-08-10 14:11:53 +10:00
Jim Mussared b4de697ad1 tools/mpremote: Fix exec_ -> exec in commands.py.
This was missed in the pyboard refactor and is preventing `cp -r` from
working.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-09 13:36:12 +10:00
Jim Mussared 5fd042e7d1 all: Replace all uses of umodule in Python code.
Applies to drivers/examples/extmod/port-modules/tools.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:24 +10:00
Damien George a61d40e5e8 tools/mpremote: Fix use of stdout_write_bytes function.
Signed-off-by: Damien George <damien@micropython.org>
2023-06-02 18:03:16 +10:00
Jim Mussared b4d785fa20 tools/mpremote: Detach mpremote from pyboard.py.
This commit just takes the necessary parts of pyboard.py and merges them
with pyboardextended.py to make a new transport_serial.py, and updates the
rest of mpremote to use this instead.

It is difficult to continue to add features to mpremote (which usually
requires modification to pyboard.py) while also maintaining backwards
compatibility for pyboard.py.

The idea is that this provides a starting point for further refactoring of
mpremote to allow different transports (webrepl, BLE, etc).

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-02 17:42:13 +10:00
Jim Mussared bd5d0163c4 docs/reference/mpremote.rst: Extend the mpremote guide.
Changes in this commit:
- Add a extra detail to each of the commands.
- Add more about handling options and arguments.
- Include shortcut commands that behave like real commands to the command
  list (e.g. bootloader, rtc).
- Add extra information and reword to address common misconceptions, in
  particular how commands chain together.
- Add additional examples showing some more interesting combinations.
- Add descriptions to each of the examples.
- Add pipx installation instructions.
- Describe how user-configuration works.

This work was sponsored by Google Season of Docs.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-02 16:16:28 +10:00
Jim Mussared 46715e370d tools/mpremote: Add `rtc` commands to get and set the RTC.
Replaces the existing functionality provided by the `setrtc` alias to use
the current time, rather than a hard-coded date/time.

Use `rtc` to query the current time.  Use `rtc --set` to set it.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-02 16:15:48 +10:00
Jim Mussared d736a2f3f3 tools/mpremote: Allow terminator for shortcut commands.
For example, the `reset` shortcut previously allowed an optional delay, but
the argument handling cannot handle `reset next-command` as `next-command`
will be interpreted as the delay argument.  The fix in this commit allows
`reset + next-command`.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-02 16:14:26 +10:00
Jim Mussared 7c2c9ea21c tools/mpremote: Add `sleep` command.
This allows the sequence to be paused (e.g. wait for device, etc).

Also removes the t_ms arg in reset/bootloader, because these arguments
don't really need to be changed, and keeping them would mean inconsistent
units used for time delays.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-02 16:11:29 +10:00
Damien George 2771b20d29 tools/mpremote: Add repl option to escape non-printable characters.
This commit adds the "--escape-non-printable" option to the repl command.
When specified the REPL console will escape non-printable characters,
printing them as their hex value in square brackets.

This escaping behaviour was previously the default and only behaviour, but
it is now opt-in.

As part of this change, the speed of echoing device data to the console is
improved by by reading and writing in chunks.

Signed-off-by: Damien George <damien@micropython.org>
2023-05-22 14:13:24 +10:00
Christian Clauss 2a1db770ce all: Fix cases of Python variable assigned but never used.
This fixes ruff rule F841.
2023-05-02 16:36:05 +10:00
Christian Clauss cda292935d tools/mpremote: Remove unused import of serial. 2023-05-02 11:22:37 +10:00
Damien George b1229efbd1 all: Fix spelling mistakes based on codespell check.
Signed-off-by: Damien George <damien@micropython.org>
2023-04-27 18:03:06 +10:00
Jonas Scharpf b525f1c9ec tools/mpremote: Add ctrl-x as additonal mpremote disconnect shortcut.
The mpremote REPL can now be closed with either ctrl+] or ctrl+x, which
gives users a choice, useful if the ']' key is difficult to access.

Fixes issue #11197.

Signed-off-by: Jonas Scharpf <jonas@brainelectronics.de>
2023-04-15 00:07:40 +10:00
Christian Clauss 4376c969f6 all: Fix Python comparison to None and True, and use "not in".
These are basic PEP8 recommendations.
2023-03-10 13:32:24 +11:00
Jim Mussared ce9f7cd00a tools/mpremote: Use hatch to build mpremote package.
This allows the entire configuration to be defined in a single file,
including the logic for including pyboard.py and automatically versioning
based on the git tag.

Building the package works both via `python -m build` as well as
`hatch build`.  `python -m build ` has the advantage of automatically
fetching all dependencies, you don't need to manually install any hatch
packages.

In order to make the versioning work, and also keep things simpler for end
users, mpremote releases will now be the same as MicroPython releases and
use the same tag.  The version strings for mpremote will look like:
- X.Y.Z -- clean build at the tag
- X.Y.Z.postN+gHASH -- clean build, N revisions from the most recent tag
- X.Y.Z.postN+gHASH.dYYYYMMDD -- dirty build, N revisions from out

This commit extends on the idea from #8404.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-02-24 11:40:22 +11:00
Michael Mogenson 921f397acb tools/mpremote: Only auto connect to serial device with USB VID/PID.
On MacOS and Windows there are a few default serial devices that are
returned by `serial.tools.list_ports.comports()`. For example on MacOS:

```
{'description': 'n/a',
 'device': '/dev/cu.Bluetooth-Incoming-Port',
 'hwid': 'n/a',
 'interface': None,
 'location': None,
 'manufacturer': None,
 'name': 'cu.Bluetooth-Incoming-Port',
 'pid': None,
 'product': None,
 'serial_number': None,
 'vid': None}

{'description': 'n/a',
 'device': '/dev/cu.wlan-debug',
 'hwid': 'n/a',
 'interface': None,
 'location': None,
 'manufacturer': None,
 'name': 'cu.wlan-debug',
 'pid': None,
 'product': None,
 'serial_number': None,
 'vid': None}
```

Users of mpremote most likely do not want to connect to these ports. It
would be desirable if mpremote did not select this ports when using the
auto connect behavior. These serial ports do not have USB VID or PID
values and serial ports for Micropython boards with FTDI/serial-to-USB
adapter or native USB CDC/ACM support do.

Check for the presence of a USB VID / PID int value when selecting a
serial port to auto connect to. All serial ports will still be listed by
the `list` command and can still be selected by name when connecting.

Signed-off-by: Michael Mogenson <michael.mogenson@gmail.com>
2022-11-25 17:20:14 -05:00
Tobias Thyrrestrup 2fcd93cdd0 tools/mpremote: Allow EDITOR environment variable to work on Windows. 2022-11-25 15:02:52 +11:00
Damien George 3ed017677b tools/mpremote: Make RemoteFile objects iterable.
So that filesystems mounted with "mpremote mount" can have their files
iterated over, making them consistent with other files.

Signed-off-by: Damien George <damien@micropython.org>
2022-10-31 12:40:17 +11:00
Jim Mussared 3427e12e8f tools/mpremote: Handle FileNotFound exceptions in listdir.
The except handler for OSError didn't include the line that actually calls
os.listdir, so an invalid path wasn't handled correctly.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-10-31 12:38:25 +11:00
Damien George 46d02c2469 tools/mpremote: Bump version to 0.4.0.
Signed-off-by: Damien George <damien@micropython.org>
2022-10-06 01:02:39 +11:00
Jim Mussared f13134e403 tools/mpremote: Fix argument handling for follow and add help strings.
Fixes in this commit are:
- Make --follow the default for "run" (accidentally changed in 68d094358).
- Add help strings for "repl": --capture --inject-file --inject-code
- Update help strings for "run".
- Fix encoding for --inject-code (accidentally broken in 68d094358).
- Remove ability to --no-follow for "eval". It was there previously because
  it shared the same code path with "exec" and "run", but makes no sense
  for "eval", so might as well remove.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-10-06 00:58:31 +11:00
Jim Mussared 12ca918eb2 tools/mpremote: Add `mpremote mip install` to install packages.
This supports the same package sources as the new `mip` tool.
 - micropython-lib (by name)
 - http(s) & github packages with json description
 - directly downloading a .py/.mpy file

The version is specified with an optional `@version` on the end of the
package name. The target dir, index, and mpy/no-mpy can be set through
command line args.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-10-01 22:44:24 +10:00
Jim Mussared 68d094358e tools/mpremote: Use argparse for command line parsing.
No functional change other than to allow slightly more flexibility in how
--foo arguments are specified.

This removes all custom handling for --foo args in all commands and
replaces it with per-command argparse configs.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-10-01 22:44:24 +10:00
Jim Mussared 413a69b94b tools/mpremote: Simplify dispatch of commands.
No functional change.

This makes each built-in command defined by just a handler method and
simplifies a lot of the logic around tracking the board state.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-10-01 22:44:24 +10:00
Damien George 2e386bcf76 tools/mpremote: Print nicer errors for unsupported 'cp -r' arguments.
Also document support for 'cp :a :b'.

Signed-off-by: Damien George <damien@micropython.org>
2022-08-26 13:30:03 +10:00
Damien George 858707181d tools/mpremote: Print a nicer error when a filesystem command fails.
Signed-off-by: Damien George <damien@micropython.org>
2022-08-26 13:30:03 +10:00
Jim Mussared 59e3348c10 tools/mpremote: Add "edit" command.
This allows a remote file to be edited locally by copying it over, running
the local editor, then copying it back.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-18 22:29:13 +10:00
Jim Mussared a311e9e3d4 tools/mpremote: Allow + terminator for fs commands.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-18 22:22:36 +10:00
Damien George 80bc9b3dee tools/mpremote: Don't be verbose when using cat command.
Fixes issue #8828.

Signed-off-by: Damien George <damien@micropython.org>
2022-07-08 22:28:26 +10:00
Damien George 5290bfaefe tools/mpremote: Bump version to 0.3.0.
Signed-off-by: Damien George <damien@micropython.org>
2022-06-07 23:23:14 +10:00
Damien George 646fcdadbf tools/mpremote: Add command to print the version.
Signed-off-by: Damien George <damien@micropython.org>
2022-06-07 23:22:04 +10:00
Rob Knegjens d3d862769a tools/mpremote: Add option to mount cmd to allow "unsafe" symlinks.
Specifying the option `--unsafe-links` (or `-l`) to `mpremote mount` will
allow symlinks to be followed in the local directory that point outside of
the base directory path.

For the unsafe case the `path_check()` method of `PyboardCommand` still
checks for a common path but without expanding symlinks.  While this check
is currently redundant, it makes the purpose of the method clearer for
possible future uses or extensions.
2022-04-11 15:15:04 +10:00
Rob Knegjens 56978c3dde tools/mpremote: Show progress indicator when copying large files.
When copying large files (> 2048 bytes) to or from a device with
`mpremote cp` a progress bar and percentage counter are temporarily shown.
2022-04-11 15:04:20 +10:00
Andrew Leech 79c05bd522 tools/mpremote: Improve reliability of mount after soft reboot.
With the existing code problems can occur with remounting, the "if t -
t_last_activity > QUIET_TIMEOUT:" check can be triggered early before the
REPL string comes through, meaning that the remount doesn't happen.

On certain boards the "MPY: soft reboot" line comes through immediately
(getting the routine past initial timeout) but then there's a slightly
longer delay while the board restarts before it prints out the startup
header and the REPL prompt.

This commit adds some extra pattern monitoring during the timeout loop to
track the state if a soft restart is actually started.
2022-04-04 16:49:38 +10:00
Waterlens 4c252ae067 tools/mpremote: Allow running mpremote with `python -m`.
This is helpful because some scripts are likely to use mpremote with a
specific python path.
2022-03-30 15:29:26 +11:00
Damien George 35dbde163a tools/mpremote: Support any prompt string when detecting soft reset.
The prompt may be changed by sys.ps1.

Signed-off-by: Damien George <damien@micropython.org>
2022-03-28 16:47:50 +11:00
Damien George bf01671a96 tools/mpremote: Bump version to 0.2.0.
Signed-off-by: Damien George <damien@micropython.org>
2022-03-07 16:02:55 +11:00
Damien George d6564a3159 tools/mpremote: Add "umount" command.
Signed-off-by: Damien George <damien@micropython.org>
2022-02-25 13:10:45 +11:00
Damien George ad1f523e7e tools/mpremote: Add "resume" and "soft-reset" commands.
This makes the auto soft-reset behaviour of mpremote more logical, and now
configurable via these new commands.

Signed-off-by: Damien George <damien@micropython.org>
2022-02-25 13:09:28 +11:00
Damien George fecfd52696 tools/mpremote: Fix special handling of ctrl-D when host FS is mounted.
Changes are:
- decision to remount local filesystem on remote device is made only if
  "MPY: soft reboot" is seen in the output after sending a ctrl-D
- a nice message is printed to the user when the remount occurs
- soft reset during raw REPL is now handled correctly

Fixes issue #7731.

Signed-off-by: Damien George <damien@micropython.org>
2022-02-07 13:21:29 +11:00
Andrew Leech 203ec8ca7f tools/mpremote: Correctly manage mounted flag during soft-reset. 2022-02-04 16:19:25 +11:00
Andrew Leech b1519845f5 tools/mpremote: Accept both --help and help to show usage. 2022-02-04 15:11:04 +11:00
Andrew Leech 5a86d8dc42 tools/mpremote: During soft reboot wait long enough for 115200 data. 2022-02-04 15:09:01 +11:00
Andrew Leech d865ca53b5 tools/mpremote: Make ConsolePosix work without .raw attribute.
When running mpremote in the vscode terminal on OSX the sys.stdout.buffer
does not have the raw attribute.  It works fine without it.
2022-02-04 15:04:02 +11:00
Andrew Leech 1f84440538 tools/mpremote: Fix "fs cp -r" on Windows.
A backslash in the directory name will end up being passed through to the
device and becoming a backslash in a filename, rather than being
interpreted as directories.  This makes "cp -r" problematic on Windows.
Changing to simply "/",join() fixes this.
2022-02-04 14:58:29 +11:00