AA fringe? (color contamination from fully transparent areas)

Anti-aliasing may be calculated wrong in Filter Forge?

My setup:
- FF 3.014
- Win7 x64
- Intel i7 2600K
- 8 GB Ram

I generally assume that I'm doing something wrong before I assume that I've actually found a bug. I can't figure this one out though, so any help would be appreciated.

When the Filter Forge AA system computes the final color (RGB values) of a pixel fr om multiple samples, it seems to treat all samples as if they have 100% alpha, regardless of any individual sample's actual alpha value. Initially I thought that I was either overlooking something, or that there might be an option that forces Filter Forge to ignore alpha values when computing the RGB contribution of a sample, but I can't find anything along those lines.

One of the main effects of this is contamination of foreground color fr om the 0% alpha background color. This often causes a tinted fringe to appear in the anti-aliased pixels of foreground objects.



Here are two demo filters that show the effect in action:

Demo Filter 1: https://dl.dropboxusercontent.com/u/38...t_00.ffxml
This shows the fringe most prominently. The fringe is red in this case.


Demo Filter 2: https://dl.dropboxusercontent.com/u/38...r_00.ffxml
This shows a practical situation wh ere there are a number of overlapping objects of different colors. The fringe is violet in this case.




As far as I'm aware, when multiple AA samples are combined to create a single output pixel, the amount of color (RGB) contributed by a sample should be weighted by the alpha (A) of that sample. Filter Forge does not appear to weight sample contribution by the sample's alpha, and instead gives every sample identical weight regardless of alpha. This means that samples with 0% alpha or 1% alpha are given exactly the same weight as samples with 100% alpha, when computing pixel color (RGB values).



For example, suppose you combine 2 samples to create a pixel's final color.
(I know 2 sample AA isn't exactly common, but it makes for a simpler example.)

Example 1 (Equal alphas):
If an AA algorithm combines a 100% opacity black sample (0 0 0 / 1) and a 100% opacity white sample (1 1 1 / 1), you should get a pixel that is 100% opacity middle-gray (0.5 0.5 0.5 / 1).
That is, (0 0 0 / 1) & (1 1 1 / 1) = (0.5 0.5 0.5 / 1).
Filter Forge handles this case without a problem.

Example 2 (Different alphas):
If an AA algorithm combines a 100% opacity black sample (0 0 0 / 1) and a 0% opacity white sample (1 1 1 / 0), you should get a pixel that is at 50% opacity, but entirely black in the RGB channels. (0 0 0 / 0.5).
That is, (0 0 0 / 1) & (1 1 1 / 0) = (0 0 0 / 0.5).
Filter Forge does not handle this case as described above.



Within Filter Forge, the scenario in "Example 2" will instead produce 50% opacity middle-gray (0.5 0.5 0.5 / 0.5).
Filter Forge AA: (0 0 0 / 1) & (1 1 1 / 0) = (0.5 0.5 0.5 / 0.5)
Alpha weighted AA: (0 0 0 / 1) & (1 1 1 / 0) = (0 0 0 / 0.5).

There is one special exception I've found. Filter Forge will "kind of" use alpha weighted samples when using the "Blend" component, but only when the opacity of the background is set to exactly 0. In this case, the 0% alpha samples will be given 0% weight for color calculations. However, if the opacity of the background is set to anything other than 0, such as 0.00001, then every sample suddenly gets identical weight again, and the color contamination returns.



The practical effect of this AA strangeness, is that if you try to generate a "Result" image that contains transparency, you end up with a colored fringe artifact at any edge wh ere an opaque and transparent area meet, but the RGB values are different for the two areas.

The only other post I found related to this subject can be found here,
http://www.filterforge.com/forum/read...ssage89716

Unfortunately the workaround proposed in that thread works only for very special cases, in which you can hardcode the RGB of transparent areas to match the RGB of locally opaque areas. It works for things like a single ellipse, but it becomes essentially impossible for handling transparency on or in between bomber particles, or any case where multiple different colored objects overlap each other on a transparent background (e.g. The "Demo Filter 2" filter linked to earlier).

Again, I may just be overlooking something, but I haven't found any solution to this fringe issue yet. So any help is appreciated.