YOUR ACCOUNT

Login or Register to post new topics or replies
Tim Pease
Posts: 2
I have been writing filter factory-type filters for dozens of years. I'm now looking to write a filterforge image convolution filter to do some simple smoothing, using a 3x3 kernel.

The convolution kernel I'm looking at is:
2 2 2
2 1 2
2 2 2
with the divisor = 17

I've looked at others' techniques for doing this. A common way is to use 9 offset components fed to 9 multiplier components fed to 9 add components.

I'm hoping my lua script would be simpler, faster, and more flexible. I think it works, in general. The script is:

Code
function prepare()
   -- This function is called once,
   -- before the rendering begins.
   -- Use it for querying non-mapped
   -- inputs and precalculation.
   dx=1/OUTPUT_WIDTH
   dy=1/OUTPUT_HEIGHT
end;

function get_sample(x, y)
   -- Image generation code goes here.
   local rul, gul, bul, aul = get_sample_map(x-dx, y-dy, SOURCE)   
   local ruc, guc, buc, auc = get_sample_map(x, y-dy, SOURCE)   
   local rur, gur, bur, aur = get_sample_map(x+dx, y-dy, SOURCE)   
   local rml, gml, bml, aml = get_sample_map(x-dx, y, SOURCE)   
   local rcent, gcent, bcent, acent = get_sample_map(x, y, SOURCE)   
   local rmr, gmr, bmr, amr = get_sample_map(x+dx, y, SOURCE)   
   local rll, gll, bll, all = get_sample_map(x-dx, y+dy, SOURCE)   
   local rlc, glc, blc, alc = get_sample_map(x, y+dy, SOURCE)   
   local rlr, glr, blr, alr = get_sample_map(x+dx, y+dy, SOURCE)   
   local rout = (rcent+2*(rul+ruc+rur+rml+rmr+rll+rlc+rlr))/17
   local gout = (gcent+2*(gul+guc+gur+gml+gmr+gll+glc+glr))/17
   local bout = (bcent+2*(bul+buc+bur+bml+bmr+bll+blc+blr))/17
   local aout = 1
   return rout, gout, bout, aout
end;


But when I use this map filter, the resulting pixels do not exactly match the results from my Paint-Shop-Pro User Defined Filter, that I've been using forever.

I've turned anti-aliasing off and disabled automatic gamma-correction, but this had no effect.
Is there an HDR setting somewhere that is affecting things?

Thanks in advance!
  Details E-Mail
Rachel Duim
So Called Tortured Artist

Posts: 2498
Filters: 188
The matrix needs to be square. In landscape images for example, the matrix would be stretched to a rectangle in your code. I think 1 / minimum of the image width and height will be your new dx and dy (that is equal). Hopefully... will look at this again tomorrow, my quick take on this.
Math meets art meets psychedelia.
  Details E-Mail
Tim Pease
Posts: 2
I used the minimum of image width and height to calculate dx and dy. The filter performed *almost* perfectly.

The resulting pixel channels differ from the results from my Paint-Shop-Pro User Defined Filter by 0, 1, or -1.

What rounding scheme do you suggest to remedy this?
  Details E-Mail
Rachel Duim
So Called Tortured Artist

Posts: 2498
Filters: 188
I have no idea. We can't be sure whose round off error it might be. GIMP also has convolution matrix, but there are additional options that might effect the result. Is the error exactly 1 or -1? If so that seems odd... Also there may be edge artifacts which is a by product of kernel processing and is dealt with in different ways by different programs. Good luck.
Math meets art meets psychedelia.
  Details E-Mail
Rachel Duim
So Called Tortured Artist

Posts: 2498
Filters: 188
Here's a non-lua version hard coded with your values. You can use it for comparison, this one should be mathematically accurate. I'm going to refine it with nine inputs so you can use a different matrix and release it as a snippet. FF13 only.

3x3 Convolution Matrix 2.ffxml
Math meets art meets psychedelia.
  Details E-Mail

Join Our Community!

Filter Forge has a thriving, vibrant, knowledgeable user community. Feel free to join us and have fun!

33,711 Registered Users
+18 new in 30 days!

153,531 Posts
+36 new in 30 days!

15,347 Topics
+72 new in year!

Create an Account

Online Users Last minute:

18 unregistered users.