Absurdly high memory requirements when opening a project with animation in Timeline
Michal P.
Posts: 77
110 GB RAM needed to open project with single character and 3.6 minutes of animation in Timeline? Really?
I have created animation in aniMate2 and converted it into Timeline. I deleted everything in aniMate2. The animation is only in Timeline now. Memory usage was 12.5 GB (all programs). I am working under Windows 10 with DAZ 4.22.0.16. I saved project and closed DAZ. The memory usage droped to 7.5 GB.
I started DAZ. I decided to open the saved project and the memory consumption went up to 110 GB virtual RAM during opening project. I have only 64 GB RAM. After opening the memory usage dropped down slowly down to 28 GB RAM. It is 15.5 GB more than necessary (memory usage was 12.5 GB before saving).
Can someone confirm this behaviour? I have attached the project with animation. It is single G8F. Be careful! Your applications may crash due to lack of memory!
Daz 3D is part of 
Connect
DAZ Productions, Inc.
7533 S Center View Ct #4664
West Jordan, UT 84084
Licensing Agreement | Terms of Service | Privacy Policy | EULA
© 2024 Daz Productions Inc. All Rights Reserved.
Comments
Do you have any dForce simulations? If so they need to store the vertex positions for each simulated item in each frame.
No, only animation in timeline. And as I wrote, memory usage was 12.5 GB before reopening. So 12.5 GB should be also after reopening or less (no undo has to be stored after reopening).
It's highly uncommon for anyone - even filmmakers using a camera - to do a character shot for more than just a few seconds. 3.6 minutes is a "Forever" shot and should be avoided.
First, it's a highly inefficient way to work. 6480 frames (assuming 30 fps) is a LOT of timeline! Imagine if you're Finally done rendering it only to find that you need to render it again with different settings!
Second, it's often boring on screen.
That said, there are reasons for the things we do - I was only mentioning 'commonplace' practice above - not trying to abolish your project by any means!
12.5 GB sounds about right to be - but that's because I use dForce, and those calculations contribute a Lot of weight to the saved scene file. Perhaps you've loaded something onto your character that was simulated and saved before clearing the simulation? It's hard to say. When we say "Just a single Character", we still know very little about the scene.
My animations are usually 5 or 6 seconds (210 - 240 frames) and can be upwards of around 2GB. So if the file size grows in a linear fashion, one of mine at 3 minutes would be 5,400GB
Just as an example of what I meant by 3.6 minutes being a "Forever" shot:
This short video is made up of a whole bunch of various animations rendered completely separately - and I normally render the character separate from the scene, which saves a Lot of time and makes the end result easier to get into a "Look" that I really want. An example being shadows. In filmmaking, the Director relies upon the Cinematographer to make sure that the characters are well lit - even in dark scenes. In CG, I use joelegecko's HDRI Photoshoot to light my characters, allowing me to render my scenes with their own lighting solutions. In this way, my character renders can render much quicker and scenes often don't need animation.
The exception to what I've said above is the car chase scene, which was (if memory serves) over 3,000 frames (still less than half of 3.6 minutes) and, in the end I only used around 900 frames of that - and it was still such a l o n g animation that I had to cut within the action or it just got to be way too much of the same thing all at once. I had already cropped the animation to 900(ish) frames before adding over 100 of Predatron's LoREZ people - animated to be walking or talking all throughout the massive city. That Car Chase scene took a lot of work to optimize.
Now, the point is that - just a few seconds on a single shot provides a lot of visuals for our fast-refresh eyes, and our brains are even faster.
This entire video is less than three minutes and there's a LOT going on in it. I'm not saying that it's a really great video, I just want to use it as a visual reference to what I'm talking about.
By this, I don't mean that the animation becomes boring - no! I'm certainly not trying to criticize your work - not at all!!!
It has to do with studies that have been done long ago.
When filmmakers don't cut to, say, a different camera angle of the same shot, our minds tend to wander - even begin to fall alseep.
This is why we never see the same perspective of a scene for longer than a few seconds in everything from movies to TV. Commercials are highly effective because they have to be really quick due to their cost. The side effect of having to stick within a minute or even half of a minute actually works to their advantage. They end up using the Best visuals quickly, and the most memorable audio.
Using this in our terms, as Daz Studio animators, we use this information as an asset.
Instead of making a really long animation and several cameras, we make a new animation for each camera angle.
I demonstrate in my Dynamic Character Animation course that, in order to do this efficiently, I create my own Base Scene.
This "Base Scene" has my render settings, including the HDRI I'm using most at that time. As any of these things change - often because of the character(s) being in a different environment, I save the new settings into my base scene - or make a new one - most often just editing the main one that I have open as my default Daz Studio scene as well as my "New" scene (Edit > Preferences)
joelegecko's HDRI Photoshoot product has a mirrored version for each of the HDRI, making it really easy to do the Cinematographer's job of reversing all of the lights when we film from the opposite perspective, which is what has made this particular lighting solution my favorite.
Most of the more modern Daz Scenes I've purchased over the years come with their own lighting, and look fantastic - just load and shoot!
Drop a character in there, however, and we need to do something different to get the character lit.
Adding lights to the scene adds a lot to the render times.
Shooting the character animations separately gives us the perfect opportunity to light them using nothing more than the HDRI Photoshoot and the Iray Ground settings, which can have reflections and shadows. I've since added 3D Universe's "Catcher Plus", which allows us to have the character cast shadows and reflections on other things in the scene that aren't actually in the scene. I don't use this a lot for my character renders, but sometimes it's really nice for that. What I like to use this for mostly is to punch alpha holes through the scene - like a Matte Painting mask - works like a charm!
Doing stuff like that and rendering in layers, etc., is what my Movie Magic course is all about - which I made because this little presentation wasn't giving out enough information.
But this is the basic principles of how I render everything separately
You can try the project. I have attached it to the first message. There is only G8F, no dForce, no other items, no textures, I hope no customs morphs (if so, they can be ignored). I wonder how much memory it will use when you open it.
If you have the same problem, there is something wrong with DAZ. As I said it should not eat more than 13 GB because it was the memory comsumtion before reopening. During the reopening (the DAZ was restarted) DAZ needs 110 GB! And after reopening it needs 28 GB which is still too much.
I understand your sugestions but that is another topic (how to bypass another bug in DAZ).
When I started with DAZ I thought that people do not use DAZ for animation because of long rendering time. Now I changed my mind. DAZ is full of bugs when working with animation and the same is with aniMate2. AniMate2 is not only full of bugs, it is also obsolete. The last truly supported Genesis is 1 or 2, am I right? And there are realy serious bugs in aniMate2!
I have reported some bugs in DAZ when working in Timeline quite long time ago. These bugs were confirmed and accepted by technical support but never fixed. Several versions have already been released but the bugs were not fixed.
Well... no. I'm not downloading a 12+ GB, 3.6 minute animation!
As per my notes earlier:
that 5,400 GB would be less than 3 minutes of animation.
The difference you might be seeing is the fact that, unless you're not using default saving methods, it's compressing your file as it's saved. So the massive 110 GB is during the decompression phase, ending in 28 GB.
These issues are not bugs. It's being asked to save Massive amounts of data, and so it is doing that.
One full minute of animation is Enormous. Two is twice that! Three....
aniMate 2 might not be updated to be able to exclude certain joints from sub tracks for Genesis 3 and newer figures, but it is the sauce that makes animation truly wonderful in Daz Studio!!! Once we get a good workflow going, animation becomes fun and addicting. Any application we use to animate a figure with the same resolution as a Daz figure will save large file sizes at lengths that long. That's not on Daz 3D, that's a workflow issue.
Seriously, even a l o n g camera pan across a grand landscape should be less than one minute long.
Just trying to help is all.
Unfortunately, this is normal.
Timeline uses an exponentially larger amount of ram than animate does.
In doing some conversion testing, every time i'd convert an animate animation, then save/reload, there was always an increase in ram usage.
A 60 second animation(V8 forward 2 cycles, Walk construction kit for victoria 8, repeated to reach 60 seconds) used ~1GB of total ram for DS in animate. When baked to timeline, then saved/reloaded, the ram usage jumped to ~8GB.
When testing timeline only files,the initial and reloaded file didn't use a significantly different amount of ram. The initial ram usage was signiificantly higher though, with a 60 second animation using ~6GB with those particular files(Beg for life, Scared animation collection p2 for Victoria 8, repeated to reach 60 seconds/1800 frames)
The only way i found to reduce the ram usage, bake the animation back to animate.
When i did that, for op's file, saved as a new GFA, and applied that to a G8F, the ram usage was only ~1GB total for DS.
The ram usage didn't change significantly when i saved and reloded the scene.
EDIT:d'oh, i forgot to specify in the last part that it was op's file that got reduced to 1GB of sysram usage on reload.
Sweet! I nomally save mine baked to the timeline, leaving the aniBlocks still in place. I wonder if I'll see storage savings if I remove the aniBlocks?
I save all of my character animations - and I do like to keep the aniBlock in place, even though they're disabled. I like it like that, so I think I'll just get more large HDDs! :)
Pardon my ignorance (I dont use the timeline for animation) but could it be that "converting into timeline" just dumbly sets a keyframe on *every* parameter on *every* frame - even if the parameter doesnt change (e.g. a character morph) or if the motion could be captured by just two or three keys over multiple seconds. There maybe some tools/scripts to remedy that, if that's the case.
Well, in "Baking to the Timeline" we end up with a key on every frame for the joints/morphs controlled by the aniBlock that's getting baked, but I'm fairly certain that it a joint or parameter is not controlled by the aniBlock, nothing gets baked.
However, even if the aniBlock makes a single change on frame 3, for example, that parameter will still get a key written for every frame.
The hard part about using a script to clean up the timeline is that these scripts often delete keys according to an interval we give it - like "Keep ever 5th frame" sort of thing. I don't use those because they stand a chance of deleting a key that's important to what I'm doing.
Especially since the latest updates to the timeline in Daz Studio 4.22, the timeline isn't difficult to work in. So I feel at ease to just open up the hierarchy in the timeline and selectively delete what I don't want. But you're right. It would be nice if either the Baking process only wrote the keys within the aniBlock to their specified locations along the timeline, or we had an easy way to recognize what is important and what's not - making it possible to have a script that can search for 'unused/unnecessary' keys and obliterate them.
Still... I love the system we have. The combination of the Key Editor in the timeline, the timeline itself, and aniMate 2 makes for a wonderful animation solution in my opinion.
It may not be that significant.
In testing the 60 second walk cycle i made, the difference between with and without the animate blocks was...1.7MB, uncompressed or compressed.
Compression made the bigger difference with ~7MB reduction in file size.
I totally get this now - as I'm working out my animations, simulations, etc.,
Captured within an aniBlock or perhaps even an animated pose file, the only data necessary to be stored is the actual rotation, translation, and scale information for each joint.
Baked to the timeline aboard a figure - an optimized Daz Figure, each joint rotation has the potential of firing JCMs and possibly other things - hence the added weight.
At least it seems to make sense to me.
I don't use DAZ studio to animate because I find it incredibly old and clunky in this respect, but for the keyframes issue may be you just need to select what types of keyframes to store, at least this works for standard animation in the timeline. If you select TR it should store only translation and rotation keyframes for example.