micropython/tests
Nicko van Someren 3aab54bf43 py: Support non-boolean results for equality and inequality tests.
This commit implements a more complete replication of CPython's behaviour
for equality and inequality testing of objects.  This addresses the issues
discussed in #5382 and a few other inconsistencies.  Improvements over the
old code include:

- Support for returning non-boolean results from comparisons (as used by
  numpy and others).
- Support for non-reflexive equality tests.
- Preferential use of __ne__ methods and MP_BINARY_OP_NOT_EQUAL binary
  operators for inequality tests, when available.
- Fallback to op2 == op1 or op2 != op1 when op1 does not implement the
  (in)equality operators.

The scheme here makes use of a new flag, MP_TYPE_FLAG_NEEDS_FULL_EQ_TEST,
in the flags word of mp_obj_type_t to indicate if various shortcuts can or
cannot be used when performing equality and inequality tests.  Currently
four built-in classes have the flag set: float and complex are
non-reflexive (since nan != nan) while bytearray and frozenszet instances
can equal other builtin class instances (bytes and set respectively).  The
flag is also set for any new class defined by the user.

This commit also includes a more comprehensive set of tests for the
behaviour of (in)equality operators implemented in special methods.
2020-01-30 14:53:07 +11:00
..
basics py: Support non-boolean results for equality and inequality tests. 2020-01-30 14:53:07 +11:00
cmdline lib/mp-readline: Add word-based move/delete EMACS key sequences. 2020-01-12 13:09:27 +11:00
cpydiff tests/cpydiff: Add CPy diff-test for using dict.keys() as a set. 2020-01-06 23:26:00 +11:00
extmod tests/extmod: Add basic machine.Timer test. 2020-01-22 17:31:18 +11:00
feature_check lib/mp-readline: Add word-based move/delete EMACS key sequences. 2020-01-12 13:09:27 +11:00
float tests: Add boolean-as-integer formatting tests for fixed regression. 2020-01-24 10:57:17 +11:00
import tests: Add .exp files for basics/parser and import/import_override. 2019-12-13 14:20:47 +11:00
inlineasm tests: Rename "array" module to "uarray". 2019-10-22 19:16:54 +11:00
internal_bench tests: Rename "bench" tests to "internal_bench" and run-internalbench.py 2019-06-28 16:28:59 +10:00
io tests/io: Update tests to use uos.remove() instead of uos.unlink(). 2018-11-26 23:27:28 +11:00
jni tests: Convert remaining "sys.exit()" to "raise SystemExit". 2017-06-10 20:34:38 +03:00
micropython tests/micropython: Add test for yield-from while heap is locked. 2019-12-27 12:34:22 +11:00
misc tests: Rename "array" module to "uarray". 2019-10-22 19:16:54 +11:00
net_hosted extmod/modlwip: Unconditionally return POLLHUP when polling new socket. 2019-10-31 12:54:37 +11:00
net_inet tests/net_inet: Update tls test to work with CPython and incl new site. 2017-10-26 12:29:24 +11:00
perf_bench tests/perf_bench: Add bm_fft test. 2019-10-15 16:38:11 +11:00
pyb tests/pyb: Adjust UART and Timer tests to work on PYBD_SF6. 2019-12-20 16:42:38 +11:00
pybnative py: Fix stack access in thumb native emitter. 2014-05-07 23:27:45 +01:00
qemu-arm qemu-arm: Add testing of frozen native modules. 2019-08-20 15:14:08 +10:00
stress tests/stress/qstr_limit: Tune params to run with stm32 port. 2019-12-13 15:58:28 +11:00
thread py/modthread: Raise RuntimeError in release() if lock is not acquired. 2017-06-14 14:43:50 +10:00
unicode py/objslice: Add support for indices() method on slice objects. 2019-12-28 23:55:15 +11:00
unix tests/unix: Add coverage tests for mp_obj_is_type() and variants. 2020-01-24 10:57:17 +11:00
wipy various: Spelling fixes 2017-05-29 11:36:05 +03:00
README tests: Make run-tests help and README be more descriptive of behaviour. 2020-01-22 16:35:24 +11:00
run-internalbench.py tests: Rename "bench" tests to "internal_bench" and run-internalbench.py 2019-06-28 16:28:59 +10:00
run-natmodtests.py unix: Rename unix binaries to micropython-variant (not _variant). 2020-01-12 10:37:40 +11:00
run-perfbench.py tests/run-perfbench.py: Skip complex tests if target doesn't enable it. 2019-10-15 16:46:06 +11:00
run-tests tests: Make run-tests help and README be more descriptive of behaviour. 2020-01-22 16:35:24 +11:00
run-tests-exp.py unix: Rename "_os" module to "uos" for consistency with baremetal ports. 2015-12-12 00:04:35 +02:00
run-tests-exp.sh run-tests-exp.sh: Typo fix in comment. 2015-02-21 03:22:33 +02:00

README

This directory contains tests for various functionality areas of MicroPython.
To run all stable tests, run "run-tests" script in this directory.

Tests of capabilities not supported on all platforms should be written
to check for the capability being present. If it is not, the test
should merely output 'SKIP' followed by the line terminator, and call
sys.exit() to raise SystemExit, instead of attempting to test the
missing capability. The testing framework (run-tests in this
directory, test_main.c in qemu_arm) recognizes this as a skipped test.

There are a few features for which this mechanism cannot be used to
condition a test. The run-tests script uses small scripts in the
feature_check directory to check whether each such feature is present,
and skips the relevant tests if not.

Tests are generally verified by running the test both in MicroPython and
in CPython and comparing the outputs. If the output differs the test fails
and the outputs are saved in a .out and a .exp file respectively.
For tests that cannot be run in CPython, for example because they use
the machine module, a .exp file can be provided next to the test's .py
file. A convenient way to generate that is to run the test, let it fail
(because CPython cannot run it) and then copy the .out file (but not
before checking it manually!)

When creating new tests, anything that relies on float support should go in the
float/ subdirectory.  Anything that relies on import x, where x is not a built-in
module, should go in the import/ subdirectory.