FF DEV Question about exe instance detection

I have need to run multiple instances of the FFXCmdRenderer and would like to know how you guys went about limiting the instance count to a single instance so I can more easily work around it. My plan is to run 1 instance of it on each of my 32 processor cores by setting the processor affinity for each instance to a different core. The filter in question for each batch will be set to only use single core rendering as it seems to run at double speed on that setting anyway. This will allow me to process 32 images simultaneously instead of 1 at a time as I currently am. Each image being processed is 2560x2560 and takes around 9.5 hrs. to finish. I have hundreds of these images so running 32 of them at once will drastically improve my overall turnaround time.

Any insight on this would be greatly appreciated and would save me a lot of time.

Ok never mind. I figured it out. It wasn't that bad really once I dug into it. For those that are interested in doing the same here's what you need to do. First get Windows Process Explorer (it's free provided by Microsoft) use it to locate this process handle:

\BaseNamedObjects\FilterForgeCMDRendererProgramMutex

Then close it. After that you should set the processor affinity of the active FFXCmdRenderer process (the one using the most CPU) to a single core or set of cores (your choice) that will be specific for that process. It seems if there are multiple instances running on the same core(s) the first one gets priority and the remainder have to wait. After that just repeat the process for each instance you need. Using this in tandem with the Filter Forge Batch Tool (linked elsewhere on the forums) is pretty awesome. I haven't confirmed any issues with doing this yet but we'll see how things go after tonight. It seems to be kicking along just fine. Be mindful of your resources, if you don't have enough cores or ram you shouldn't be doing this. But I'll let you make that call.

It's a bit slow to set up 32 instances like I want but it's well worth it and maybe I'll find a quicker way later to kill the lock handle. Also, I don't seem to be able to set affinity for the 32nd core only, not sure why, but I can set affinity to it if I also set another core with it. I guess in the end I can only have 31 instances because of that. Whatever, that works for me

@Grimbly: an impressive setup indeed!

@Sphinx: the GUI version relies on the assumption that its files won't be messed up from the outside. For example, preferences and library structure files are opened exclusively. I guess the command-line renderer inherits this behavior from the GUI version.

@GMM: Thanks, My computer setup is the best I've ever had. It's mind blowing what this thing can do when fully utilized. This newly discovered instancing setup is less than ideal but far nicer than running only one instance so despite it's failing points it's still quite nice

@Sphinx.: I have still seen no issues with it yet. Last nights tests, some of which are still running, have been entirely successful thus far. I will detail them more below.

@primitivedesign: The primary filter that started all of this benefits tremendously from single core processing over using all 32 cores. It has a lot of Blurs and other non-parallel nodes in it. It is the final filter in the chain of filters I'm working with. The initial filter I use to generate the base image swings the other direction and drastically benefits from multi-core processing over single core. It really just depends on the filter's construction and composition. It's always a good idea for long winded projects like mine to check these things as they could save you lots and lots of time. Just runs some stress tests with each option and it will show you the way

OK, now for the test scenario from last night. I ran only 3 instances as I'm not currently ready to make full use of the 31 instances I apparently can have. The first 2 instances were the single core (9.5hrs. per image) filter set to use one core each (by way of affinity) using cores 29 and 30. The 3rd instance was using the initial filter for base image generation and was set to use all remaining cores (0-28 and 31). All 3 instances have successfully saved images to their respective folders and as needed are either finished or continuing to run right now. The first 2 instances used the same folder for their reference images and saved images. The 3rd instance is saving to a different folder for my own convenience and needs no reference image.

I hope you guys have fun with this and that it helps others having the same issue. If I do find any issues in the coming weeks, since I'll be using it much more extensively, I'll make it a point to drop in here and let everybody know.

If anyone finds a more streamlined method to repeatedly kill the lock handle while opening instances I would very much like to hear about it. Having to do 31 of these for each major batch I do will take probably around 1-1.5hrs. to get up and running so I'm all for something speeding that up

Ok, that's all for now. Enjoy and have a good weekend