YOUR ACCOUNT

Login or Register to post new topics or replies
SpaceRay
SpaceRay

Posts: 12298
Filters: 35
Quote
Moderator's note: there are numerous threads about this issue on the forum. All official replies will be made in this thread only.


Hello,

I am getting frequently this error sometimes using the standalone FF 3.06 Pro version

<class XFW::Kernel::StdException> bad allocation

Quote
oniXMaster wrote:

"bad allocation" means that you ran out of available memory.

Please try the following:

1) ensure your paging file size isn't limited by a setting in Control Panel->System->Advanced->Settings(Performance)->Advanced->Change (ideally it should be System Managed and you should have enough space on disk, I recommend several tenths of gigabytes)

2) reduce the amount of memory Filter Forge uses for it's internal computations (go to Tools->Options...->Rendering and set the Maximum RAM usage slider to something like 60 instead of the default 75)

Also please ensure that you have enough disk space for both paging file, you can check it in the same location as 1) ) and Filter Forge temporary files (available in the same location as 2) ).

P.S. Test With 10000 x 5000 image sizes with properly set up paging file. The fact that other filters render fine makes me think that you're running out of disk space (there are two bitmap-based components in there, with such resolutions the temporary files alone might take from 3 to 6 gigabytes


It is NOT possible that the out of memory error could be from not enough RAM because I have 16 GB so is not this

What I find very interesting and surprising that is NOT only RAM needed, is ALSO hard disk storage size for Filter Forge temporary files !!!! smile:?: smile:?: smile:?: smile:?: smile:?:

So, the problem could be NOT from FF alone, and would be because I have it bad configured in the paging file size ? smile:?: smile:?:

Is this why there are many problems on SOME COMPUTER to render high resolutions because probably what is wrong is that there is NOT ENOUGH STORAGE SPACE AVAILABLE FOR FF TEMPORAL FILES ?? smile:?: smile:?:
  Details E-Mail
SpaceRay
SpaceRay

Posts: 12298
Filters: 35
Quote
oniXMaster wrote:
1) ensure your paging file size isn't limited by a setting in Control Panel->System->Advanced->Settings(Performance)->Advanced->Change (ideally it should be System Managed and you should have enough space on disk, I recommend several tenths of gigabytes)


For point 1 my own information is as follows

On C: I do not have any paging file because is a SSD drive, I have configured and transfered the system paging file to the E: drive where it puts that recommended is 24451 MB (24 GB) and there is configured 32602 MB (36 GB)

In this E: drive there is 75 GB space free.

In the C: drive where the Filter Forge software is installed and where it is supossed that the temporal files need to be stored have 27 GB free.

Quote
oniXMaster wrote:
2) reduce the amount of memory Filter Forge uses for it's internal computations (go to Tools->Options...->Rendering and set the Maximum RAM usage slider to something like 60 instead of the default 75)


Well, I had already by default 60 in the MAX ram usage slider so I do not need to change it, unless I need to reduce it even more.

I do not understand why reducing the FF maximum ram usage will help, when the problem is that there is NOT ENOUGH RAM and so reducing it will give LESS RAM and not more

Quote
oniXMaster wrote:

Also please ensure that you have enough disk space for both paging file, you can check it in the same location as 1) ) and Filter Forge temporary files (available in the same location as 2) ).


There is 32 GB paging file and 75 GB free in this drive, and in the C: filter forge temporay files there is 27 GB free.

Here below I put a screen shot of the Virtual Memory found in Control Panel->System->Advanced->Settings(Performance)->Advanced->Change

  Details E-Mail
SpaceRay
SpaceRay

Posts: 12298
Filters: 35
Could I be having this lots of errors in FF because I do not have the paging file in the original default C: drive ?
  Details E-Mail
Morgantao
Can't script

Posts: 2185
Filters: 20
That's not supposed to matter...
  Details E-Mail
SpaceRay
SpaceRay

Posts: 12298
Filters: 35
I have found that this ONLY happens in the standalone version !!

I have got this error many times and with just a 2500x2500 photo smile:evil: smile:cry:

USING FILTER FORGE AS A PLUGIN OF PHOTOSHOP CS5 SOLVES THE PROBLEM !!! smile:)

BUT then I have used the SAME photo with Filter Forge as a plugin inside Photoshop and using the same filter and same settings, AND DO NOT GET ANYMORE this error , So I am happy that at least I know that to fix this undesired and unwanted error I just need to use Filter Forge as plugin and this SOLVES the problem. smile:)
  Details E-Mail
SpaceRay
SpaceRay

Posts: 12298
Filters: 35
I have asked for more information from the FF team but there has not been any answer until now smile:(
  Details E-Mail
Avalanche Studios
Technical Artist
Posts: 8
Hi Guys

We have the same problem and we have both memory and diskspace on our highend pcs.

And it has todo with plugin mode as well as standalone when it comes to larger formats and for some reason only when a blur node has been used. If I remove the blur nodes I can do 65k textures with no errors.

We are trying to 16k x 16k textures and it crashes with this error even in plugin mode (if I use blur nodes).

Any ideas ?

- Avalanche Studios
  Details E-Mail
Morgantao
Can't script

Posts: 2185
Filters: 20
Quote
If I remove the blur nodes I can do 65k textures with no errors.

OMG, I hope you are right and this only happens because of a bug in the blur component. That would help the dev team take care of it once and for all! smile:)
  Details E-Mail
Avalanche Studios
Technical Artist
Posts: 8
Yes to my understanding the problem sits in the blur node.
I have tested several blur combos but a single nodes still breaks the render.

If the allocation is the problem i recommend that you first allocate and do a horizontal blur then the same with vertical blur. Doing that the memory footprint will be far less.

Also this is only a problem when you have large input image, 16384x16384 is what I try with.

(fun fact: photoshop cs5.5 cannot save psd files larger than 2gb)
  Details E-Mail
Morgantao
Can't script

Posts: 2185
Filters: 20
I think you should write the support team of your findings about the blur node, just in case they missed this thread.

Quote
fun fact: photoshop cs5.5 cannot save psd files larger than 2gb

That's why since CS2 they have PSB files, which are supposed to be able to go beyond 2GB (B in PSB stands for big).
  Details E-Mail
Avalanche Studios
Technical Artist
Posts: 8
I have emailed both contact and support but got no response beyond automated answers, I you any other channel in feel free to relay the information.

We are quite desperate to get a solution =P, The product clearly states that it can do renders up to 65k so we bought it and .. yeah well =P
  Details E-Mail
Sharandra
Filter Forge Addict

Posts: 863
Filters: 26
Wow if that is really the source of that bug and it enables them to finally get rid of it, I think you deserve lots of bug points for finding it smile:D
  Details E-Mail
GMM
Moderator
Filter Forge, Inc
Posts: 3491
Quote
only when a blur node has been used. If I remove the blur nodes I can do 65k textures with no errors.


Avalanche Studios, this is very very interesting. Could you please post a filter that crashes with the Blur component and works correctly without it?
  Details E-Mail
Avalanche Studios
Technical Artist
Posts: 8
Again its a theory but this always seems todo the trick.

Using version 3 with latest update.

Image = any 16384x16384 texture
out = same size different name

Break
image -> blur -> perlin noise -> perlin noise -> blur -> out

Works
image -> perlin noise -> perlin noise -> out

note: restart of the software might be needed as the software locks the broken image in memory and any renders after it will also get the crash unless you restart.
  Details E-Mail
GMM
Moderator
Filter Forge, Inc
Posts: 3491
I replaced Blurs with High Passes in your example and it still crashes. Needs more testing with bitmap-based and procedural components.
  Details E-Mail
Avalanche Studios
Technical Artist
Posts: 8
I am sorry but I have to disagree, replacing blur with highpass works fine.

Remember to close down (taskmanager) the software after it has crashed and also save the new file using a new name in case the pic is stilled locked.
  Details E-Mail
SpaceRay
SpaceRay

Posts: 12298
Filters: 35
I think that this "Bad allocation" bug is the worst thing FF has IF it is used as standalone, as this does NOT happen nearly on the plugin version using Photoshop, which I do not understand why.

I have NOT tried the above suggestion of the Blur component breaking the filter and making a crash, BUT I do not think that this could be the reason, as this "bad allocation" crash happens with nearly all the filter I have tried, and not all have blur components.
  Details E-Mail
Avalanche Studios
Technical Artist
Posts: 8
It is true that I have had the issues on large files using the standalone version using other nodes but its more of problems where it does not release data from memory and then get a bad allocation.

The only time I got the plugin to crash (bad allocation) is when I used blur nodes.

I think its two separate problems, I think the standalone is due to that the preview is rendering at the same time and sometimes (or always) tries to access the same data / write to it.

The second problem I think its close to the first one but happens when the blur is allocating memory to do a full blur pass.
  Details E-Mail
Kraellin
Kraellin

Posts: 12749
Filters: 99
this is a memory error. i get it all the time. i have a cpu/ram usage meter running right next to FF when it's running. my system has 6 gigs. when it hits about 3.1 or above, the bad allocation can occur. if it's below that, it wont. so, if you have a ton of filters sitting in FF in storage, it does use ram. if you're using big pictures, you are going to run into this at some point. if you're using a lot of high memory components and doing a lot of bitmap type stuff, all of this seems to have to be held in memory the whole time the filter is rendering. as memory gets used up and you go over whatever this bug limit is, boom!

one interesting thing about this bug is that you can actually get the error in various numbers of iterations all happening at once. my typical number is i'll get 8 bug messages, all the same. if i get all 8 my rendering will stop and wont finish. if i get less than 8, say only 3, the rendering will continue. this usually only seems to happen if i'm right on the cusp of memory usage.

you can do things to help yourself out. offload some of the filters you keep IN FF. use smaller pictures if you can. cut down the little percentage thing you hit when you save... the 2nd window after you hit the save button. all of those things use memory.

i've reported this bug many times and every time i've been told there is no fix for it and that they're not sure what is doing this. i'm very convinced it's a resource management problem. i also seem to remember this bug showing up more when they fixed the old memory leak, though this last may just be my faulty, old age memory.

my system and FF are almost unusable at this point. almost any render i do on a photograph will crash before i can get the end of the progress bar. when i try to save filters in the editor, they almost all crash during the save. and every single time the memory meter is at 3.1 gigs or higher, which is pretty much the limit that windows can handle with a 32 bit program. (and FF is ONLY 32 bit, even if you have 64 bit windows, so the memory limit rules apply)

and here's a real kicker. there doesnt seem to be ANY activity kicked over to windows paging/swap file when ram starts to back up. i NEVER see a harddrive light go on when memory is approaching the limit, but i'm going to watch this closer now to make sure that's right, too. i have an SSD drive as my C: drive, so my swap file wwould be almost as fast as ram and windows shld be able to handle all this quite easily. so, i dont know quite what's going on.

if there was any reason for FF inc. to go to a 64 bit coding, this is it. i hope we can nip this in the bud quickly. it's making my FF experience a pain in the butt.
If wishes were horses... there'd be a whole lot of horse crap to clean up!

Craig
  Details E-Mail
Avalanche Studios
Technical Artist
Posts: 8
I have come to the same conclusion =)

The software does not seem to release and reuse memory and running a 32-bit version (which it always does) which have a limit of 4 gigs and a thread-size of 1. This really does not work for big data.

There are only two options (I recommend doing both to have a 32-bit fallback):

1. Allocate memory for the specific operation only and for the blur do a horizontal then a vertical operation. On other operations you can for example do them in pixel blocks, say 64x64 allocating and releasing memory between each block.

2. Make a 64bit version so unlimited of memory and Thread size can be used.

OBS: Not being able to render out: and I quote "... and render huge images up to 65000 x 65000 pixels in size" which is the biggest selling point for the pro version, might in fact be a legal issue as this fits into the frames of false marketing.
  Details E-Mail
Kraellin
Kraellin

Posts: 12749
Filters: 99
no, it's not false advertising, at least as i recall things. the bad allocation thing doesnt happen on every machine. some folks can render on the huge sizes. i think Sign Guy is one that can. he makes textures for the sign industry and uses very large file sizes. it just takes a while. so, it seems to be something specific to certain machines. it could even be something like your anti-virus or firewall or who knows what.

also, 32 bit can actually only use 3 gigs on a windows machine, 2 gig plus the patched version allowing 3. it's a numbers thing.

and yes, 64 bit would be the ultimate solution but FF, inc. seems reluctant to go that way yet. so, we'll see.
If wishes were horses... there'd be a whole lot of horse crap to clean up!

Craig
  Details E-Mail
Avalanche Studios
Technical Artist
Posts: 8
Well I don't know about that hehe =p

I have tried it on 4 different machines with different hardware / os and I get the same results =/. If anyone knows of a configuration that works with ff please let me know! I still think that it should be mentioned in the hardware requirements in this case.

Yes the actual size in 32-bit is just above 3gigs =)
  Details E-Mail
Sharandra
Filter Forge Addict

Posts: 863
Filters: 26
Usually I can´t render 4096x4096 in the standalone version without getting the bad allocation error.
I just found out that when I set the preview size to actual and let it render, it works just fine. Takes ages, but I can save afterwards.
  Details E-Mail
Skybase
2D/3D Generalist

Posts: 4025
Filters: 76
Man, I really wonder about this bug. It never happens on my machine and I don't do anything particularly special. I typically process pictures I take with my 60D with it and the resolution of those images exceeds 5000x3000 pixels. I also render textures at 4096x4096 (where applicable with what I'm doing.) And I typically like to let it render in the window itself and save it later. I mean sure, once in a while it kinda coughs stupid things up and crashes but I'm typically editing the image during those situations.

I'm using the Mac version of FilterForge by the way. Seems like the windows edition has the rusty nail. But if in any case I wonder what's happening after all with this error.
  Details E-Mail
SpaceRay
SpaceRay

Posts: 12298
Filters: 35
Quote
Sharandra

Usually I can´t render 4096x4096 in the standalone version without getting the bad allocation error


Yes, for me 98% of the times that I try to render above 3500 pixels in standalone mode it gives this error, and it happens in both ways, I mean if you use the reduced size and then go "save image as" or if you set the preview size to "Actual" it normally after more than half of the render or reaching the end will give this error.

I HAVE 16 GB but FF is 32 bit, so it means that it will not use more than 3 GB, or worse is that is said that the it has it own custom memory controller that can not use more than 1,5 GB.

Quote
Kraellin

one interesting thing about this bug is that you can actually get the error in various numbers of iterations all happening at once. my typical number is i'll get 8 bug messages, all the same. if i get all 8 my rendering will stop and wont finish. if i get less than 8, say only 3, the rendering will continue. this usually only seems to happen if i'm right on the cusp of memory usage.


ALL the information that Kraellin has given is true for me, I have seen twice that I was looking at the memory manager in windows, that it was close to the 90% and I was ONLY using Filter Forge, and not multistasking, so I can´t know how FF can take all this amount of RAM.

Also as told in the quote, the error is replicated between 3 or more times (after the 5th I hit always the "Kill this task" in the "Process Explorer" tool utility. It has happened to me around twice that after 3 errors it continues rendering and finishes.

Quote
Skybase

Man, I really wonder about this bug.

I'm using the Mac version of FilterForge by the way. Seems like the windows edition has the rusty nail. But if in any case I wonder what's happening after all with this error.


YES, I really wonder and would like to know what and why happens this bug too, and is like a mistery, and it seems that it only happens in the Windows version, perhaps because the MacOS is managing the memory in a different way.

What is MUCH MORE misterious and strange is that this really happens the most of the times in STANDALONE MODE and NOT IN PLUGIN VERSION, well it happens also in plug in version BUT only about 10% versus 98% in the standalone.

What is different from the standalone version to the plugin version memory management ?
  Details E-Mail
Sharandra
Filter Forge Addict

Posts: 863
Filters: 26
Quote
Skybase wrote:
And I typically like to let it render in the window itself and save it later.


Instead of saving it and letting it render while saving?
  Details E-Mail
Skybase
2D/3D Generalist

Posts: 4025
Filters: 76
I mean, we'd have to wait and see if FilterForge 4 solves anything of this matter. I thought this was one of those "things to be tackled" for version 4. I thought I remember seeing some word about it somewhere.

Quote
Instead of saving it and letting it render while saving?


I donno, I also just hit render too. But I like resizing windows so I'd rather have FilterForge not become static during rendering.
  Details E-Mail
GMM
Moderator
Filter Forge, Inc
Posts: 3491
We have solid grounds to suspect that Progressive Previews may cause bad memory allocation. Please set the Multi-pass Preview to Legacy in Tools > Options > Interface and try rendering a large image.
  Details E-Mail
Sharandra
Filter Forge Addict

Posts: 863
Filters: 26
Quote
Please set the Multi-pass Preview to Legacy in Tools > Options > Interface and try rendering a large image.


Nope, sorry, still getting the bad allocation error.
  Details E-Mail
Vladimir Golovin
Administrator
Posts: 3446
Filters: 55
We just fixed a bug that may have been the cause of Bad Allocation:
http://www.filterforge.com/forum/read...&TID=10903
  Details E-Mail
inujima

Posts: 176
Filters: 26
Quote
Please set the Multi-pass Preview to Legacy in Tools > Options > Interface and try rendering a large image.


This error occurs even if set to "Legacy".
When setting to "Off", it seems to work well like plugin mode.
  Details E-Mail
GMM
Moderator
Filter Forge, Inc
Posts: 3491
A bit of info about better allocation that wouldn't go into the official news.

We haven't eradicated the issue completely but we've found its cause and made it much less likely. The following factors contribute to the chance of getting BadAlloc:

- the image size (obviously);
- the presence of bitmap-based components in the filter;
- fiddling with the interface (panning and zooming) during render – this eats up memory quickly!
- position of the Memory usage limit slider. Higher settings speed up the render and increase the chance to get BadAlloc. That is, if you never experience BadAlloc feel free to wind this slider up to 90; if you get BadAlloc set it to 80, or 70, or whatever value that gets rid of BadAllocs on your PC. It is no coincidence that the default value is 60.

We've successfully rendered a Blurred Bomber at 64kx64k with Memory usage limit set to 50.
  Details E-Mail
SpaceRay
SpaceRay

Posts: 12298
Filters: 35
Good to know this GMM, thanks, so we can test it more and better.

Quote
we've found its cause and made it much less likely


Yes, is true I have tried some tests and before would have always given an error and now it works much better!! until now only in very high resolutions (14500 x 8500) I have got the error.
  Details E-Mail
Wolvman
Posts: 3
Just dropping in to say that I still get the error at 4096 x 4096 resolution on a few filters with the latest version but it does seem to be at least a little better. This is with memory usage limit slider at 90 though. Still have yet to give it a shot with the slider at 80 or 70.
  Details E-Mail
Skybase
2D/3D Generalist

Posts: 4025
Filters: 76
Wolvman, you should write your report down in this thread: http://www.filterforge.com/forum/read...&TID=10903

smile;)
  Details E-Mail
fisholith
Posts: 24
Filters: 2
Just wanted to say that I tried two different solutions that have been mentioned on this thread, and on the possible fix thread, and both worked for me. I tried each of them separately.

The solutions are described below.

Also, thanks to SpaceRay, oniXMaster, GMM, and Vladimir Golovin, for suggesting them. smile:)



My setup:

I was rendering an 8,000 x 5,500 image, that included two high-pass components (bitmap based components).

* FF: v3.014
* OS: Win 7 x64
* CPU: Intel Core i7 2600K
* Mem: DDR3 8GB



Solutions:

* Max RAM usage - (Mentioned by oniXMaster, GMM, and Vladimir Golovin):
From the Filter Forge menu bar, choose Tools > Options, choose Rendering (tab), and set "Maximum RAM usage, %" (slider) to "60".

(Note: I had been using "90", and I was worried that setting it to 60 would slow down renders, but at "60", even on half-hour-long renders, FF only took 1 or 2 extra seconds. And even that could just be random fluctuation.)

* Plugin Mode - (Mentioned by SpaceRay):
Try running FF as a plug-in instead of as a standalone program.



My prior unstable settings:

When I got the "Bad Allocation" error, I was using FF in standalone form, with "Maximum RAM usage, %" set to "90".

With these original unstable settings, I was never able to render and save my 8,000 x 5,500 image, as the error would either crash FF during rendering, or during saving. I tried rendering this image 6 times, with the "Bad Allocation" error coming up at some point each time.

Thanks again, for the suggested solutions. smile:)
  Details E-Mail

Join Our Community!

Filter Forge has a thriving, vibrant, knowledgeable user community. Feel free to join us and have fun!

33,711 Registered Users
+18 new in 30 days!

153,531 Posts
+36 new in 30 days!

15,347 Topics
+72 new in year!

Create an Account

Online Users Last minute:

28 unregistered users.