Tim Pease
Posts: 2 |
I have been writing filter factory-type filters for dozens of years. I'm now looking to write a filterforge image convolution filter to do some simple smoothing, using a 3x3 kernel.
The convolution kernel I'm looking at is: 2 2 2 2 1 2 2 2 2 with the divisor = 17 I've looked at others' techniques for doing this. A common way is to use 9 offset components fed to 9 multiplier components fed to 9 add components. I'm hoping my lua script would be simpler, faster, and more flexible. I think it works, in general. The script is:
But when I use this map filter, the resulting pixels do not exactly match the results from my Paint-Shop-Pro User Defined Filter, that I've been using forever. I've turned anti-aliasing off and disabled automatic gamma-correction, but this had no effect. Is there an HDR setting somewhere that is affecting things? Thanks in advance! |
|||
Posted: December 26, 2023 12:39 am | ||||
Rachel Duim
![]() |
The matrix needs to be square. In landscape images for example, the matrix would be stretched to a rectangle in your code. I think 1 / minimum of the image width and height will be your new dx and dy (that is equal). Hopefully... will look at this again tomorrow, my quick take on this.
Math meets art meets psychedelia. |
|||
Posted: December 26, 2023 1:16 am | ||||
Tim Pease
Posts: 2 |
I used the minimum of image width and height to calculate dx and dy. The filter performed *almost* perfectly.
The resulting pixel channels differ from the results from my Paint-Shop-Pro User Defined Filter by 0, 1, or -1. What rounding scheme do you suggest to remedy this? |
|||
Posted: December 26, 2023 10:25 pm | ||||
Rachel Duim
![]() |
I have no idea. We can't be sure whose round off error it might be. GIMP also has convolution matrix, but there are additional options that might effect the result. Is the error exactly 1 or -1? If so that seems odd... Also there may be edge artifacts which is a by product of kernel processing and is dealt with in different ways by different programs. Good luck.
Math meets art meets psychedelia. |
|||
Posted: December 27, 2023 2:47 am | ||||
Rachel Duim
![]() |
Here's a non-lua version hard coded with your values. You can use it for comparison, this one should be mathematically accurate. I'm going to refine it with nine inputs so you can use a different matrix and release it as a snippet. FF13 only.
3x3 Convolution Matrix 2.ffxml Math meets art meets psychedelia. |
|||
Posted: December 27, 2023 1:51 pm |
Filter Forge has a thriving, vibrant, knowledgeable user community. Feel free to join us and have fun!
33,711 Registered Users
+18 new in 30 days!
153,531 Posts
+36 new in 30 days!
15,347 Topics
+72 new in year!
18 unregistered users.