Compare commits

...

2 Commits

Author SHA1 Message Date
Alice R ca98480482
Merge 34bb71ee5e into f4a71b1337 2023-12-30 13:31:36 +03:00
AliceLR 34bb71ee5e Fix residue classdata bounding for f->temp_memory_required.
Updates `start_decoder`'s calculation of `f->temp_memory_required`
to more accurately reflect the required size of the `part_classdata`
array in `decode_residue`.

In `decode_residue`, the value `actual_size` is derived from the
variable `n`, which is sourced from the variable `n2` in
`vorbis_decode_packet_rest`. `n2` is equal to one of the blocksizes
divided by 2 (of which `f->blocksize_1` will always be the greater
value). For residue type 2, `decode_residue` multiplies its copy of
`n` by 2 (back to the original blocksize).

The faulty bounding in `start_decoder` unconditionally divides
`f->blocksize_1` by 2, even for residue type 2. This patch corrects
the bounding to use the full `f->blocksize_1` for residue type 2.

This bug *may* have implications for anything using the alloc
buffer feature, but I don't have an input that would cause this
(the setup requires a much larger temp buffer than `decode_residue`).
I found this bug via a heap corruption crash while fuzzing libxmp,
which uses a work buffer derived from `f->temp_memory_required`
instead of `alloca`. The `alloca` doesn't use `f->temp_memory_required`,
so it does NOT have this bug.
2023-06-15 18:01:23 -06:00
1 changed files with 2 additions and 1 deletions

View File

@ -4161,7 +4161,8 @@ static int start_decoder(vorb *f)
int i,max_part_read=0;
for (i=0; i < f->residue_count; ++i) {
Residue *r = f->residue_config + i;
unsigned int actual_size = f->blocksize_1 / 2;
unsigned int rtype = f->residue_types[i];
unsigned int actual_size = rtype == 2 ? f->blocksize_1 : f->blocksize_1 / 2;
unsigned int limit_r_begin = r->begin < actual_size ? r->begin : actual_size;
unsigned int limit_r_end = r->end < actual_size ? r->end : actual_size;
int n_read = limit_r_end - limit_r_begin;