[Gs-code-review] A fix for #516530 heap crash with OTGuide.pdf .

[Igor V. Melichev]

Peter,

> From: L. Peter Deutsch [mailto:ghost at aladdin.com]
> Sent: Thursday, February 14, 2002 10:22 AM
> To: igor at artifex.com
> Cc: gs-code-review at ghostscript.com; raph at levien.com
> Subject: Re: [Gs-code-review] A fix for #516530 heap crash with
> OTGuide.pdf .
>
> > +     penum->mask[0].depth = penum->mask[1].depth = 0;
>
> Please explain why this is needed.

See gximag3x.c, line 217.
Need to fall into /* mask not supplied */ if a mask isn't supplied.

> The comment immediately above these
> assignments clearly states that they are initializing pointers in case an
> error occurs; the depth members are not involved in this.

Well, probably need to move it 1 line below and insert empty line before
the initialization of 'depth'. Possibly another comment is useful.

> ! 	     * Hack: we pass pis here, rather we need to
>   	     * create another imager state with default log_op, etc.
> +              */
>
> The English needs improving, but the more serious issue is that passing
pis
> rather than NULL will definitely produce incorrect behavior if the imager
> state has non-default values for the logical op, and perhaps for other
> members as well.

Probably so.

> What goes wrong when passing NULL?

GS crashes some later accessing penum->pis which is NULL.
The test file is available from
http://www.adobe.com/type/browser/pdfs/OTGuide.pdf .

My patch is incomplete, but it allows me to continue my work on FAPI.
Raph had assigned this bug to himself.
I'd like him or you to complete the patch.
My knowledge about this stuff is small. What I did is "run somehow".
Alternatively - commit my patch now with the minor change explained above,
and postpone further improvements.

Igor.