Why Does It Take So Long To Load Figures? ("Solved", with GUIDE)

2456

Comments

  • PerttiAPerttiA Posts: 10,024
    LenioTG said:

    I run the Windows 10 and Daz Studio from a 512GB M.2 PCIe x4 SSD and the Daz Studio content from a second 512GB M.2 PCIe x4 SSD but as far as I can tell there is no benefitn the content may just as well be on a normal SATA3 SSD.
    This appears to be a major limitation in Daz Studio's design.

    The only way is to keep a base library of essential content and then switch in and out other librarys of content as you require them.

    Best Wishes
    Steve.

    You know this is the best idea I've heard so far?

    Of course it's unpractical to install and uninstall everything, but installing all the figures on the same drive, but in a different folder, and turning it on and off might have sense.

     

    One question: what if I created morphs from my figures, then would I be able to delete the figures I have installed just to not lose the reference?

    If talking about storage space and placing the DS files and resources, the Sysinternals Juction is a "must have" tool.

    Windows on it's own lets you map complete drives onto a directory on another drive, but Junction allows you to map individual directories/folders onto directories/folders created on another drive.

    DAZ Studio would see only one Content folder, but you could for example move Data-folder to a fast drive and Runtime/Textures to a slower drive, or even pick and choose any directories and move them somewhere else to balance space usage between the drives.

    This way you can update your total storage space in smaller (cheaper) portions, instead of having to pay by the nose for every update.

  • LenioTGLenioTG Posts: 2,118
    edited June 2020
    PerttiA said:
    LenioTG said:

    I run the Windows 10 and Daz Studio from a 512GB M.2 PCIe x4 SSD and the Daz Studio content from a second 512GB M.2 PCIe x4 SSD but as far as I can tell there is no benefitn the content may just as well be on a normal SATA3 SSD.
    This appears to be a major limitation in Daz Studio's design.

    The only way is to keep a base library of essential content and then switch in and out other librarys of content as you require them.

    Best Wishes
    Steve.

    You know this is the best idea I've heard so far?

    Of course it's unpractical to install and uninstall everything, but installing all the figures on the same drive, but in a different folder, and turning it on and off might have sense.

     

    One question: what if I created morphs from my figures, then would I be able to delete the figures I have installed just to not lose the reference?

    If talking about storage space and placing the DS files and resources, the Sysinternals Juction is a "must have" tool.

    Windows on it's own lets you map complete drives onto a directory on another drive, but Junction allows you to map individual directories/folders onto directories/folders created on another drive.

    DAZ Studio would see only one Content folder, but you could for example move Data-folder to a fast drive and Runtime/Textures to a slower drive, or even pick and choose any directories and move them somewhere else to balance space usage between the drives.

    This way you can update your total storage space in smaller (cheaper) portions, instead of having to pay by the nose for every update.

    Thank you PerttiA :D

    Storage is not a problem right now. 2TB is plenty of space for me, and if it'll fill up, I'll just delete a bunch of stuff I don't really need! For example, I don't even remember the last time I've used a freebie from sharecg etc.

    And if I'll continue to receive support for my comics, in a few years I could get a 4TB NVME PCIe 4.0 drive, that'll be much cheaper by then. For now, my wallet is already bleeding for those high-end motherboard and RAM xD

    Post edited by LenioTG on
  • PerttiAPerttiA Posts: 10,024
    LenioTG said:
    PerttiA said:
    LenioTG said:

    I run the Windows 10 and Daz Studio from a 512GB M.2 PCIe x4 SSD and the Daz Studio content from a second 512GB M.2 PCIe x4 SSD but as far as I can tell there is no benefitn the content may just as well be on a normal SATA3 SSD.
    This appears to be a major limitation in Daz Studio's design.

    The only way is to keep a base library of essential content and then switch in and out other librarys of content as you require them.

    Best Wishes
    Steve.

    You know this is the best idea I've heard so far?

    Of course it's unpractical to install and uninstall everything, but installing all the figures on the same drive, but in a different folder, and turning it on and off might have sense.

     

    One question: what if I created morphs from my figures, then would I be able to delete the figures I have installed just to not lose the reference?

    If talking about storage space and placing the DS files and resources, the Sysinternals Juction is a "must have" tool.

    Windows on it's own lets you map complete drives onto a directory on another drive, but Junction allows you to map individual directories/folders onto directories/folders created on another drive.

    DAZ Studio would see only one Content folder, but you could for example move Data-folder to a fast drive and Runtime/Textures to a slower drive, or even pick and choose any directories and move them somewhere else to balance space usage between the drives.

    This way you can update your total storage space in smaller (cheaper) portions, instead of having to pay by the nose for every update.

    Thank you PerttiA :D

    Storage is not a problem right now. 2TB is plenty of space for me, and if it'll fill up, I'll just delete a bunch of stuff I don't really need! For example, I don't even remember the last time I've used a freebie from sharecg etc.

    And if I'll continue to receive support for my comics, in a few years I could get a 4TB NVME PCIe 4.0 drive, that'll be much cheaper by then. For now, my wallet is already bleeding for those high-end motherboard and RAM xD

    I totally understand the bleeding wallet... I'm planning a move from i7 system to something better, and the last time I checked the total would come to around 2500-3000 euros for just the bare essentials (mobo/processor/memory/display adapter), when they sell "High End" complete systems for a thousand at the stores... And I have even made my plans using cost effective components, ie. Not paying for the bragging rights.

  • LenioTGLenioTG Posts: 2,118
    PerttiA said:
    LenioTG said:
    PerttiA said:
    LenioTG said:

    I run the Windows 10 and Daz Studio from a 512GB M.2 PCIe x4 SSD and the Daz Studio content from a second 512GB M.2 PCIe x4 SSD but as far as I can tell there is no benefitn the content may just as well be on a normal SATA3 SSD.
    This appears to be a major limitation in Daz Studio's design.

    The only way is to keep a base library of essential content and then switch in and out other librarys of content as you require them.

    Best Wishes
    Steve.

    You know this is the best idea I've heard so far?

    Of course it's unpractical to install and uninstall everything, but installing all the figures on the same drive, but in a different folder, and turning it on and off might have sense.

     

    One question: what if I created morphs from my figures, then would I be able to delete the figures I have installed just to not lose the reference?

    If talking about storage space and placing the DS files and resources, the Sysinternals Juction is a "must have" tool.

    Windows on it's own lets you map complete drives onto a directory on another drive, but Junction allows you to map individual directories/folders onto directories/folders created on another drive.

    DAZ Studio would see only one Content folder, but you could for example move Data-folder to a fast drive and Runtime/Textures to a slower drive, or even pick and choose any directories and move them somewhere else to balance space usage between the drives.

    This way you can update your total storage space in smaller (cheaper) portions, instead of having to pay by the nose for every update.

    Thank you PerttiA :D

    Storage is not a problem right now. 2TB is plenty of space for me, and if it'll fill up, I'll just delete a bunch of stuff I don't really need! For example, I don't even remember the last time I've used a freebie from sharecg etc.

    And if I'll continue to receive support for my comics, in a few years I could get a 4TB NVME PCIe 4.0 drive, that'll be much cheaper by then. For now, my wallet is already bleeding for those high-end motherboard and RAM xD

    I totally understand the bleeding wallet... I'm planning a move from i7 system to something better, and the last time I checked the total would come to around 2500-3000 euros for just the bare essentials (mobo/processor/memory/display adapter), when they sell "High End" complete systems for a thousand at the stores... And I have even made my plans using cost effective components, ie. Not paying for the bragging rights.

    I build my PCs keeping in mind that I'll upgrade them down-the-line!

    For example, when you buy Intel, that motherboard will only be okay for a single generation, while something like a X570 will give you a lot of room for upgrade in the future, maybe supporting something like a Ryzen 9 6000something.
    Then, I rarely upgrade storage, and power supply, and case, and RAM.
    And I rarely upgrade everything at the same time. If a month I buy a GPU, at least a couple of months will pass before I'll get a CPU, and so on.

    This time I had to buy more things at the same time, because of covid: I already have a powerful PC, but it's in another part of Italy, and I won't be able to access it for months. When I'll be able to do that, I'll probably sell its components. But for now I need stuff that's capable of basic Daz things, because I do that every day.

  • That's good to know!

    How do you switch out your content? Do you add/remove folders via the Content Directory Manager or do you actually move them around on the hard drive?

  • LenioTGLenioTG Posts: 2,118

    That's good to know!

    How do you switch out your content? Do you add/remove folders via the Content Directory Manager or do you actually move them around on the hard drive?

    I think via the Content Directory Manager would be the quickest way

  • Richard HaseltineRichard Haseltine Posts: 100,842
    LenioTG said:

    That's good to know!

    How do you switch out your content? Do you add/remove folders via the Content Directory Manager or do you actually move them around on the hard drive?

    I think via the Content Directory Manager would be the quickest way

    Content Directory Manager does allow you to have multiple directory sets, each with its own database, so if you don't mind have duplicate content installs you can set things up for one-click swapping (well, several clicks to open the dialogue etec but still not many).

  • LenioTGLenioTG Posts: 2,118
    LenioTG said:

    That's good to know!

    How do you switch out your content? Do you add/remove folders via the Content Directory Manager or do you actually move them around on the hard drive?

    I think via the Content Directory Manager would be the quickest way

    Content Directory Manager does allow you to have multiple directory sets, each with its own database, so if you don't mind have duplicate content installs you can set things up for one-click swapping (well, several clicks to open the dialogue etec but still not many).

    It would cost a lot of money to have double the storage!
    I was thinking about putting the figures-related folders into a different folder, and to just disable it when I don't need those figures.
    But I see little use for that, since 99,99% of my renders have figures.

    Another way would be to have the majority of the figures in that 2nd folder, and to use it only when I create new custom characters, moving in the other one the needed characters.
    But all that micromanaging would probably take even more time, and it's surely more tedious.

    So I just hope that the bottleneck is my hardware, and things will improve when the new components will arrive.
    Or that the next Daz Studio update will address, at least in part, this issue.
    Honestly, I've never used things like Strand Hair or Face Transfer, that required a ton of developing time, but I'd surely benefit a reduction in figure loading times.

  • Hurdy3DHurdy3D Posts: 1,047

    Are you guys sure that a SSD helps?

     

    I have a SATA SSD with 4TB and if I load a figure the parts what takes really long is "deleting objects".

    And if you watch on the HHD activity I don't see any heave reading requests on the HDD. Once the deleting objects part is done (where the HDD writes around 10MB/s) the figure loads instant.

    To be honest, I don't understand why DAZ needs to delete obejcts in a new scene... but that's how it is.

  • LenioTGLenioTG Posts: 2,118
    gerster said:

    Are you guys sure that a SSD helps?

     

    I have a SATA SSD with 4TB and if I load a figure the parts what takes really long is "deleting objects".

    And if you watch on the HHD activity I don't see any heave reading requests on the HDD. Once the deleting objects part is done (where the HDD writes around 10MB/s) the figure loads instant.

    To be honest, I don't understand why DAZ needs to delete obejcts in a new scene... but that's how it is.

    Yes, the impact of having the library on an SSD compared to an HDD is HUGE.

    You don't see any activities on the HDD because the "delete" action is not bottlenecked by it, I guess. It's always software stuff, probably influenced by CPU/RAM.

    My Genesis 3 and prior figures load instantly as well, because I don't have many morphs installed for them. The relationship is directly proportional.

  • jkim5453jkim5453 Posts: 6

    Fast CPU and fast SSD doesn't really help with figure load time related to having a large number of morph files. I'm afraid you can't throw money at it buying faster parts to solve this issue. Your system in general will benefit (e.g faster boot time, application load time, and many operations within different applications in general). However, for the problem at hand for DAZ3D users with a large number of morph files, there's basically no benefit I observed per each extra $ spent for this particular load time problem (and the equally long "deleting objects" time taken when you open a new scene or exiting DAZ Studio after having loaded a Gen 8 figure into the existing scene).

    It will need to be optimized elsewhere.

  • LenioTGLenioTG Posts: 2,118
    jkim5453 said:

    Fast CPU and fast SSD doesn't really help with figure load time related to having a large number of morph files. I'm afraid you can't throw money at it buying faster parts to solve this issue. Your system in general will benefit (e.g faster boot time, application load time, and many operations within different applications in general). However, for the problem at hand for DAZ3D users with a large number of morph files, there's basically no benefit I observed per each extra $ spent for this particular load time problem (and the equally long "deleting objects" time taken when you open a new scene or exiting DAZ Studio after having loaded a Gen 8 figure into the existing scene).

    It will need to be optimized elsewhere.

    I guess next week we'll know! :D

    I'll measure exactly the loading times before and after I install the new hardware.

  • Hurdy3DHurdy3D Posts: 1,047
    edited June 2020

    I had before I got the SSD my DAZ lib on a HDD.

    The only speed performance I regonized when I moved to my SSD was that scrolling through smart content became much smoother, because the thumbnails images are loading faster.

    Post edited by Hurdy3D on
  • Update. Removing the morphs I converted with Riversoft's converter greatly speeded up the loading process. Dev version loads in just about 2 minutes, now.

    Having a really hard time deciding to uninstall anything, because my workflow requires versatility in character creation. Still, 2 minutes is better than 5.

  • LenioTGLenioTG Posts: 2,118
    gerster said:

    I had before I got the SSD my DAZ lib on a HDD.

    The only speed performance I regonized when I moved to my SSD was that scrolling through smart content became much smoother, because the thumbnails images are loading faster.

    I've done extensive testings, and at least in my system basically everything improved, from poses applying time, and loading of everything, maybe even figures.

    Hanabi said:

    Update. Removing the morphs I converted with Riversoft's converter greatly speeded up the loading process. Dev version loads in just about 2 minutes, now.

    Having a really hard time deciding to uninstall anything, because my workflow requires versatility in character creation. Still, 2 minutes is better than 5.

    I absolutely understand the necessity of versatility in character creation! (

    I'm glad you managed to speed up things a bit!

  • LenioTGLenioTG Posts: 2,118
    edited June 2020

    UPDATE

    The new hardware has arrived, and I've made extensive tests. We're talking about

    • Ryzen 5 3600 (compared to Ryzen 5 1600)
    • RAM 32GB 3600MHz (compared to 16GB 2400MHz)
    • X570 mobo (compared to B350M)

    The results are very interesting IMHO!

    I have repeated each test twice, to make sure the results were reliable. I have tested with the old and with the new hardware, in the same day.
    I have tested 4 different files:

    • A real-world usage scene, with a complete apartment and 4 figures (3 G8F + 1 G8M). I've loaded it, cleared the scene and rendered it up to 200 iterations.
    • Basic G8F: load and clear scene.
    • Basic G8M: load and clear scene.
    • Preload environment without figures: load and clear scene.

    RESULTS

    • Real-world scene
      • Loading: 235s vs 318s (-26%)
      • Rendering: 288s vs 320s (-10%)
      • Clearing: 228s vs 297s (-23%)
    • G8F
      • Loading: 48s vs 64s (-25%)
      • Clearing: 62s vs 79s (-22%)
    • G8M
      • Loading: 25s vs 27s (-7%)
      • Clearing: 17s vs 23s (-26%)
    • Environment
      • Loading: 16s vs 24s (-33%)
      • Clearing: I don't know, it's too fast

    So we can see an average reduction in loading times of 23%, an average reduction of clearing time of 24%. And what surprised me the most is the 10% reduction in rendering times, considering that I've kept the same GPU (RTX 2060).

    My Cinebench 20 score has gone up by 51%, and my Passmark Test CPU score has gone up by 51%.
    So, maybe, the loading/clearing times improve by half of the CPU performance gain?
    Has anyone else tested this?

    Post edited by LenioTG on
  • robertswwwrobertswww Posts: 790

    It looks like the hardware upgrades paid off for you... Around 25% less time for G8F and Real-world Scene loading is a nice improvement in time-savings.  I bet you are happy with those numbers!

  • LenioTG said:

    UPDATE

    The new hardware has arrived, and I've made extensive tests. We're talking about

    • Ryzen 5 3600 (compared to Ryzen 5 1600)
    • RAM 32GB 3600MHz (compared to 16GB 2400MHz)
    • X570 mobo (compared to B350M)

    The results are very interesting IMHO!

    I have repeated each test twice, to make sure the results were reliable. I have tested with the old and with the new hardware, in the same day.
    I have tested 4 different files:

    • A real-world usage scene, with a complete apartment and 4 figures (3 G8F + 1 G8M). I've loaded it, cleared the scene and rendered it up to 200 iterations.
    • Basic G8F: load and clear scene.
    • Basic G8M: load and clear scene.
    • Preload environment without figures: load and clear scene.

    RESULTS

    • Real-world scene
      • Loading: 235s vs 318s (-26%)
      • Rendering: 288s vs 320s (-10%)
      • Clearing: 228s vs 297s (-23%)
    • G8F
      • Loading: 48s vs 64s (-25%)
      • Clearing: 62s vs 79s (-22%)
    • G8M
      • Loading: 25s vs 27s (-7%)
      • Clearing: 17s vs 23s (-26%)
    • Environment
      • Loading: 16s vs 24s (-33%)
      • Clearing: I don't know, it's too fast

    So we can see an average reduction in loading times of 23%, an average reduction of clearing time of 24%. And what surprised me the most is the 10% reduction in loading times, considering that I've kept the same GPU (RTX 2060).

    My Cinebench 20 score has gone up by 51%, and my Passmark Test CPU score has gone up by 51%.
    So, maybe, the loading/clearing times improve by half of the CPU performance gain?
    Has anyone else tested this?

    The video card would have no affect on the loading times as that only comes in to play when you actually render a scene.

    How are you measuring the times? Is there a utility for this? I would like to test my system as well.

    I am curious why you went with Ryzen 5 instead of going up to Ryzen 7? I'm guessing that it was cost based on your location...

     

  • dijituldijitul Posts: 146

    A useful option might be a verbose log that can be enabled to record load times for every morph.  A lot of data at first, but may reveal the packages which take the longest load times.

    I also wonder if it's possible to "integrate" all the morphs into a single character file?  Not sure it would save time or even be feasible, so just a passing thought.  Might even be a feature request to have a "cached" load to use until the user manually refreshes/clears it (such as after a new package is installed).  Might allow for faster loading since error-checking and linking would only need to be done once.

     

  • Catherine3678abCatherine3678ab Posts: 8,339
    edited June 2020
    LenioTG said:

    Interesting, the 2 TB NVMe drives are roughly half that here in the US. :(

    You may see some speed improvement with a new MB/CPU/memory but it won't be nearly as much as using an NVMe drive as your boot drive (even if you don't load your entire library onto it). Also there is a point of diminishing returns when it comes to RAM speed. That is to say it probably won't be worth the extra money to go over 3200 and you probably won't see much difference between 2400 and 3200. Having more cores sounds like a great idea but if the software isn't written so support it - and I doubt Daz Studio is - then you won't see any gain beyond the increased clock speed.

    I have an AMD Ryzen 7 2700 8 core / 16-thread CPU on a 470 board with 32 GB RAM, a GTX 1060 Ti, and the aforementioned NVMe drive. The bottleneck IMO - at least in terms of loading - will still be the SATA speed. If the .duf files are compressed then a CPU upgrade will help somewhat.

    Best of luck, and I hope you do see some better performance!

    Lucky you :D

    Consider that that model is PCIe Gen4, and that in Italy all prices are VAT included.

    Yeah but everything other than figures loads crazy fast, so I struggle to think that the problem might be the SATA connection here!

    I already have an NVME as boot drive, I use it for Windows only. That's where Daz is installed.

    I've already purchased a Ryzen 5 3600, 32GB of 3600MHz C16 RAM and a X570 Motherboard (in the future I'll probably get a Ryzen 9 4000something and 2 high-end GPUs), that should allow me to overclock properly. We'll see.

    Hay all, made an interesting discovery. How fast the figures load can have nothing to do with D/S per sae. Check how much 'empty' space is on the drive holding D/S. It works better if there is more than 15GB free, even better if there is more than 20GB free. The problem is how in the world that drive keeps filling up in the first place! But, some claim that's because of all the various updates for the OS,  and its parts.

    And yes I too have deleted a few purchased programs from the C drive - installing some of them onto external drives. Bit of a lag but nothing super note worthy.

    D/S is on the C drive and ALL its content is on the D drive. 3D Coat is entirely on an external drive. Carrara has yet to be reinstalled.

    I highly doubt that's it! I have 600GB of free space in my drive.

    I think you're using a Hard Drive?

     

    SATA SSDs are crazy fast for Daz: again, if I take a complex complete scene, delete the figure, and load it, it loads in around just 10 seconds. Hard Drives are much slower than that.
    My problem regards Genesis 8 figures and only those!

    Well that's definitely not all that's involved then. It helps me though. Yes I'm using the internal harddrives on this computer for D/S. One is normal, the other is SSD.

    Post edited by Catherine3678ab on
  • ProPoseProPose Posts: 527

    I don't understand these Loading and Clearing times you guys are getting 

    My G8F loads into a blank scene in about 7 seconds, G8M loads in about the same time. This is on an i7-3820 with 32 GB ram,

    GTX 1060 with 6 GB vram driver ver. 441.87.  No ssd drives, All Sata III

  • LenioTGLenioTG Posts: 2,118
    edited June 2020

    It looks like the hardware upgrades paid off for you... Around 25% less time for G8F and Real-world Scene loading is a nice improvement in time-savings.  I bet you are happy with those numbers!

    Yes! :D

    I'm just quite angry because I decided to get a X570 motherboard, and I didn't expect they would be SO loud. I'll have to switch to a B450 and to build the entire rig again :(

    LenioTG said:

    UPDATE

    The new hardware has arrived, and I've made extensive tests. We're talking about

    • Ryzen 5 3600 (compared to Ryzen 5 1600)
    • RAM 32GB 3600MHz (compared to 16GB 2400MHz)
    • X570 mobo (compared to B350M)

    The results are very interesting IMHO!

    I have repeated each test twice, to make sure the results were reliable. I have tested with the old and with the new hardware, in the same day.
    I have tested 4 different files:

    • A real-world usage scene, with a complete apartment and 4 figures (3 G8F + 1 G8M). I've loaded it, cleared the scene and rendered it up to 200 iterations.
    • Basic G8F: load and clear scene.
    • Basic G8M: load and clear scene.
    • Preload environment without figures: load and clear scene.

    RESULTS

    • Real-world scene
      • Loading: 235s vs 318s (-26%)
      • Rendering: 288s vs 320s (-10%)
      • Clearing: 228s vs 297s (-23%)
    • G8F
      • Loading: 48s vs 64s (-25%)
      • Clearing: 62s vs 79s (-22%)
    • G8M
      • Loading: 25s vs 27s (-7%)
      • Clearing: 17s vs 23s (-26%)
    • Environment
      • Loading: 16s vs 24s (-33%)
      • Clearing: I don't know, it's too fast

    So we can see an average reduction in loading times of 23%, an average reduction of clearing time of 24%. And what surprised me the most is the 10% reduction in loading times, considering that I've kept the same GPU (RTX 2060).

    My Cinebench 20 score has gone up by 51%, and my Passmark Test CPU score has gone up by 51%.
    So, maybe, the loading/clearing times improve by half of the CPU performance gain?
    Has anyone else tested this?

    The video card would have no affect on the loading times as that only comes in to play when you actually render a scene.

    How are you measuring the times? Is there a utility for this? I would like to test my system as well.

    I am curious why you went with Ryzen 5 instead of going up to Ryzen 7? I'm guessing that it was cost based on your location...

    Why are you telling this? We know the GPU doesn't affect loading times :)

    One of the tests I've made was a rendering one, that showed a 10% performance increase. My RTX 2060 was probably bottlenecked by the RAM/CPU before.

    For the rendering times you can see the log.

    For loading/clearing times I'm using a normal stopwatch from when I click on the file until Daz is responsive.

    Zen 3 CPUs are coming soon, so I preferred to take a 3600 for some months so that I'll have the budget to switch to a Ryzen 7/9 at the end of the year.
    Plus, single-thread performance are basically the same, and they say that's what matters in Daz.

    dijitul said:

    A useful option might be a verbose log that can be enabled to record load times for every morph.  A lot of data at first, but may reveal the packages which take the longest load times.

    I also wonder if it's possible to "integrate" all the morphs into a single character file?  Not sure it would save time or even be feasible, so just a passing thought.  Might even be a feature request to have a "cached" load to use until the user manually refreshes/clears it (such as after a new package is installed).  Might allow for faster loading since error-checking and linking would only need to be done once.

    That's actually what I'd like to do: to join the morphs I need for specific characters into just one new custom morph (there are tutorials on YouTube), and then uninstall them.

    But I'm behind my schedule for Patreon, so no more learning/optimizing Daz this month, I need to render! :)

     

    LenioTG said:

    Interesting, the 2 TB NVMe drives are roughly half that here in the US. :(

    You may see some speed improvement with a new MB/CPU/memory but it won't be nearly as much as using an NVMe drive as your boot drive (even if you don't load your entire library onto it). Also there is a point of diminishing returns when it comes to RAM speed. That is to say it probably won't be worth the extra money to go over 3200 and you probably won't see much difference between 2400 and 3200. Having more cores sounds like a great idea but if the software isn't written so support it - and I doubt Daz Studio is - then you won't see any gain beyond the increased clock speed.

    I have an AMD Ryzen 7 2700 8 core / 16-thread CPU on a 470 board with 32 GB RAM, a GTX 1060 Ti, and the aforementioned NVMe drive. The bottleneck IMO - at least in terms of loading - will still be the SATA speed. If the .duf files are compressed then a CPU upgrade will help somewhat.

    Best of luck, and I hope you do see some better performance!

    Lucky you :D

    Consider that that model is PCIe Gen4, and that in Italy all prices are VAT included.

    Yeah but everything other than figures loads crazy fast, so I struggle to think that the problem might be the SATA connection here!

    I already have an NVME as boot drive, I use it for Windows only. That's where Daz is installed.

    I've already purchased a Ryzen 5 3600, 32GB of 3600MHz C16 RAM and a X570 Motherboard (in the future I'll probably get a Ryzen 9 4000something and 2 high-end GPUs), that should allow me to overclock properly. We'll see.

    Hay all, made an interesting discovery. How fast the figures load can have nothing to do with D/S per sae. Check how much 'empty' space is on the drive holding D/S. It works better if there is more than 15GB free, even better if there is more than 20GB free. The problem is how in the world that drive keeps filling up in the first place! But, some claim that's because of all the various updates for the OS,  and its parts.

    And yes I too have deleted a few purchased programs from the C drive - installing some of them onto external drives. Bit of a lag but nothing super note worthy.

    D/S is on the C drive and ALL its content is on the D drive. 3D Coat is entirely on an external drive. Carrara has yet to be reinstalled.

    I highly doubt that's it! I have 600GB of free space in my drive.

    I think you're using a Hard Drive?

     

    SATA SSDs are crazy fast for Daz: again, if I take a complex complete scene, delete the figure, and load it, it loads in around just 10 seconds. Hard Drives are much slower than that.
    My problem regards Genesis 8 figures and only those!

    Well that's definitely not all that's involved then. It helps me though. Yes I'm using the internal harddrives on this computer for D/S. One is normal, the other is SSD.

    Yes :)

    A SATA III SSD is more than enough, I guess.

    At this point, it's just a matter of uninstalling useless products and upgrading CPU/RAM.

    ProPose said:

    I don't understand these Loading and Clearing times you guys are getting 

    My G8F loads into a blank scene in about 7 seconds, G8M loads in about the same time. This is on an i7-3820 with 32 GB ram,

    GTX 1060 with 6 GB vram driver ver. 441.87.  No ssd drives, All Sata III

    It depends on the morphs you have installed!

    You probably have not bought many G8 figures/morph packages yet?

    Post edited by LenioTG on
  • "Why are you telling this? We know the GPU doesn't affect loading times :)"

    Because you said this...

    "And what surprised me the most is the 10% reduction in loading times, considering that I've kept the same GPU (RTX 2060)."

    ...which makes it sound like you were correlating GPU and loading times. Sorry if I misunderstood.

  • LenioTGLenioTG Posts: 2,118

    "Why are you telling this? We know the GPU doesn't affect loading times :)"

    Because you said this...

    "And what surprised me the most is the 10% reduction in loading times, considering that I've kept the same GPU (RTX 2060)."

    ...which makes it sound like you were correlating GPU and loading times. Sorry if I misunderstood.

    Corrected, thanks for making me notice, I meant rendering times :D

  • ProPoseProPose Posts: 527
    edited June 2020
    LenioTG said:

    It looks like the hardware upgrades paid off for you... Around 25% less time for G8F and Real-world Scene loading is a nice improvement in time-savings.  I bet you are happy with those numbers!

    Yes! :D

    I'm just quite angry because I decided to get a X570 motherboard, and I didn't expect they would be SO loud. I'll have to switch to a B450 and to build the entire rig again :(

    LenioTG said:

    UPDATE

    The new hardware has arrived, and I've made extensive tests. We're talking about

    • Ryzen 5 3600 (compared to Ryzen 5 1600)
    • RAM 32GB 3600MHz (compared to 16GB 2400MHz)
    • X570 mobo (compared to B350M)

    The results are very interesting IMHO!

    I have repeated each test twice, to make sure the results were reliable. I have tested with the old and with the new hardware, in the same day.
    I have tested 4 different files:

    • A real-world usage scene, with a complete apartment and 4 figures (3 G8F + 1 G8M). I've loaded it, cleared the scene and rendered it up to 200 iterations.
    • Basic G8F: load and clear scene.
    • Basic G8M: load and clear scene.
    • Preload environment without figures: load and clear scene.

    RESULTS

    • Real-world scene
      • Loading: 235s vs 318s (-26%)
      • Rendering: 288s vs 320s (-10%)
      • Clearing: 228s vs 297s (-23%)
    • G8F
      • Loading: 48s vs 64s (-25%)
      • Clearing: 62s vs 79s (-22%)
    • G8M
      • Loading: 25s vs 27s (-7%)
      • Clearing: 17s vs 23s (-26%)
    • Environment
      • Loading: 16s vs 24s (-33%)
      • Clearing: I don't know, it's too fast

    So we can see an average reduction in loading times of 23%, an average reduction of clearing time of 24%. And what surprised me the most is the 10% reduction in loading times, considering that I've kept the same GPU (RTX 2060).

    My Cinebench 20 score has gone up by 51%, and my Passmark Test CPU score has gone up by 51%.
    So, maybe, the loading/clearing times improve by half of the CPU performance gain?
    Has anyone else tested this?

    The video card would have no affect on the loading times as that only comes in to play when you actually render a scene.

    How are you measuring the times? Is there a utility for this? I would like to test my system as well.

    I am curious why you went with Ryzen 5 instead of going up to Ryzen 7? I'm guessing that it was cost based on your location...

    Why are you telling this? We know the GPU doesn't affect loading times :)

    One of the tests I've made was a rendering one, that showed a 10% performance increase. My RTX 2060 was probably bottlenecked by the RAM/CPU before.

    For the rendering times you can see the log.

    For loading/clearing times I'm using a normal stopwatch from when I click on the file until Daz is responsive.

    Zen 3 CPUs are coming soon, so I preferred to take a 3600 for some months so that I'll have the budget to switch to a Ryzen 7/9 at the end of the year.
    Plus, single-thread performance are basically the same, and they say that's what matters in Daz.

    dijitul said:

    A useful option might be a verbose log that can be enabled to record load times for every morph.  A lot of data at first, but may reveal the packages which take the longest load times.

    I also wonder if it's possible to "integrate" all the morphs into a single character file?  Not sure it would save time or even be feasible, so just a passing thought.  Might even be a feature request to have a "cached" load to use until the user manually refreshes/clears it (such as after a new package is installed).  Might allow for faster loading since error-checking and linking would only need to be done once.

    That's actually what I'd like to do: to join the morphs I need for specific characters into just one new custom morph (there are tutorials on YouTube), and then uninstall them.

    But I'm behind my schedule for Patreon, so no more learning/optimizing Daz this month, I need to render! :)

     

    LenioTG said:

    Interesting, the 2 TB NVMe drives are roughly half that here in the US. :(

    You may see some speed improvement with a new MB/CPU/memory but it won't be nearly as much as using an NVMe drive as your boot drive (even if you don't load your entire library onto it). Also there is a point of diminishing returns when it comes to RAM speed. That is to say it probably won't be worth the extra money to go over 3200 and you probably won't see much difference between 2400 and 3200. Having more cores sounds like a great idea but if the software isn't written so support it - and I doubt Daz Studio is - then you won't see any gain beyond the increased clock speed.

    I have an AMD Ryzen 7 2700 8 core / 16-thread CPU on a 470 board with 32 GB RAM, a GTX 1060 Ti, and the aforementioned NVMe drive. The bottleneck IMO - at least in terms of loading - will still be the SATA speed. If the .duf files are compressed then a CPU upgrade will help somewhat.

    Best of luck, and I hope you do see some better performance!

    Lucky you :D

    Consider that that model is PCIe Gen4, and that in Italy all prices are VAT included.

    Yeah but everything other than figures loads crazy fast, so I struggle to think that the problem might be the SATA connection here!

    I already have an NVME as boot drive, I use it for Windows only. That's where Daz is installed.

    I've already purchased a Ryzen 5 3600, 32GB of 3600MHz C16 RAM and a X570 Motherboard (in the future I'll probably get a Ryzen 9 4000something and 2 high-end GPUs), that should allow me to overclock properly. We'll see.

    Hay all, made an interesting discovery. How fast the figures load can have nothing to do with D/S per sae. Check how much 'empty' space is on the drive holding D/S. It works better if there is more than 15GB free, even better if there is more than 20GB free. The problem is how in the world that drive keeps filling up in the first place! But, some claim that's because of all the various updates for the OS,  and its parts.

    And yes I too have deleted a few purchased programs from the C drive - installing some of them onto external drives. Bit of a lag but nothing super note worthy.

    D/S is on the C drive and ALL its content is on the D drive. 3D Coat is entirely on an external drive. Carrara has yet to be reinstalled.

    I highly doubt that's it! I have 600GB of free space in my drive.

    I think you're using a Hard Drive?

     

    SATA SSDs are crazy fast for Daz: again, if I take a complex complete scene, delete the figure, and load it, it loads in around just 10 seconds. Hard Drives are much slower than that.
    My problem regards Genesis 8 figures and only those!

    Well that's definitely not all that's involved then. It helps me though. Yes I'm using the internal harddrives on this computer for D/S. One is normal, the other is SSD.

    Yes :)

    A SATA III SSD is more than enough, I guess.

    At this point, it's just a matter of uninstalling useless products and upgrading CPU/RAM.

    ProPose said:

    I don't understand these Loading and Clearing times you guys are getting 

    My G8F loads into a blank scene in about 7 seconds, G8M loads in about the same time. This is on an i7-3820 with 32 GB ram,

    GTX 1060 with 6 GB vram driver ver. 441.87.  No ssd drives, All Sata III

    It depends on the morphs you have installed!

    You probably have not bought many G8 figures/morph packages yet?

     

    LenioTG said:

    It looks like the hardware upgrades paid off for you... Around 25% less time for G8F and Real-world Scene loading is a nice improvement in time-savings.  I bet you are happy with those numbers!

    Yes! :D

    I'm just quite angry because I decided to get a X570 motherboard, and I didn't expect they would be SO loud. I'll have to switch to a B450 and to build the entire rig again :(

    LenioTG said:

    UPDATE

    The new hardware has arrived, and I've made extensive tests. We're talking about

    • Ryzen 5 3600 (compared to Ryzen 5 1600)
    • RAM 32GB 3600MHz (compared to 16GB 2400MHz)
    • X570 mobo (compared to B350M)

    The results are very interesting IMHO!

    I have repeated each test twice, to make sure the results were reliable. I have tested with the old and with the new hardware, in the same day.
    I have tested 4 different files:

    • A real-world usage scene, with a complete apartment and 4 figures (3 G8F + 1 G8M). I've loaded it, cleared the scene and rendered it up to 200 iterations.
    • Basic G8F: load and clear scene.
    • Basic G8M: load and clear scene.
    • Preload environment without figures: load and clear scene.

    RESULTS

    • Real-world scene
      • Loading: 235s vs 318s (-26%)
      • Rendering: 288s vs 320s (-10%)
      • Clearing: 228s vs 297s (-23%)
    • G8F
      • Loading: 48s vs 64s (-25%)
      • Clearing: 62s vs 79s (-22%)
    • G8M
      • Loading: 25s vs 27s (-7%)
      • Clearing: 17s vs 23s (-26%)
    • Environment
      • Loading: 16s vs 24s (-33%)
      • Clearing: I don't know, it's too fast

    So we can see an average reduction in loading times of 23%, an average reduction of clearing time of 24%. And what surprised me the most is the 10% reduction in loading times, considering that I've kept the same GPU (RTX 2060).

    My Cinebench 20 score has gone up by 51%, and my Passmark Test CPU score has gone up by 51%.
    So, maybe, the loading/clearing times improve by half of the CPU performance gain?
    Has anyone else tested this?

    The video card would have no affect on the loading times as that only comes in to play when you actually render a scene.

    How are you measuring the times? Is there a utility for this? I would like to test my system as well.

    I am curious why you went with Ryzen 5 instead of going up to Ryzen 7? I'm guessing that it was cost based on your location...

    Why are you telling this? We know the GPU doesn't affect loading times :)

    One of the tests I've made was a rendering one, that showed a 10% performance increase. My RTX 2060 was probably bottlenecked by the RAM/CPU before.

    For the rendering times you can see the log.

    For loading/clearing times I'm using a normal stopwatch from when I click on the file until Daz is responsive.

    Zen 3 CPUs are coming soon, so I preferred to take a 3600 for some months so that I'll have the budget to switch to a Ryzen 7/9 at the end of the year.
    Plus, single-thread performance are basically the same, and they say that's what matters in Daz.

    dijitul said:

    A useful option might be a verbose log that can be enabled to record load times for every morph.  A lot of data at first, but may reveal the packages which take the longest load times.

    I also wonder if it's possible to "integrate" all the morphs into a single character file?  Not sure it would save time or even be feasible, so just a passing thought.  Might even be a feature request to have a "cached" load to use until the user manually refreshes/clears it (such as after a new package is installed).  Might allow for faster loading since error-checking and linking would only need to be done once.

    That's actually what I'd like to do: to join the morphs I need for specific characters into just one new custom morph (there are tutorials on YouTube), and then uninstall them.

    But I'm behind my schedule for Patreon, so no more learning/optimizing Daz this month, I need to render! :)

     

    LenioTG said:

    Interesting, the 2 TB NVMe drives are roughly half that here in the US. :(

    You may see some speed improvement with a new MB/CPU/memory but it won't be nearly as much as using an NVMe drive as your boot drive (even if you don't load your entire library onto it). Also there is a point of diminishing returns when it comes to RAM speed. That is to say it probably won't be worth the extra money to go over 3200 and you probably won't see much difference between 2400 and 3200. Having more cores sounds like a great idea but if the software isn't written so support it - and I doubt Daz Studio is - then you won't see any gain beyond the increased clock speed.

    I have an AMD Ryzen 7 2700 8 core / 16-thread CPU on a 470 board with 32 GB RAM, a GTX 1060 Ti, and the aforementioned NVMe drive. The bottleneck IMO - at least in terms of loading - will still be the SATA speed. If the .duf files are compressed then a CPU upgrade will help somewhat.

    Best of luck, and I hope you do see some better performance!

    Lucky you :D

    Consider that that model is PCIe Gen4, and that in Italy all prices are VAT included.

    Yeah but everything other than figures loads crazy fast, so I struggle to think that the problem might be the SATA connection here!

    I already have an NVME as boot drive, I use it for Windows only. That's where Daz is installed.

    I've already purchased a Ryzen 5 3600, 32GB of 3600MHz C16 RAM and a X570 Motherboard (in the future I'll probably get a Ryzen 9 4000something and 2 high-end GPUs), that should allow me to overclock properly. We'll see.

    Hay all, made an interesting discovery. How fast the figures load can have nothing to do with D/S per sae. Check how much 'empty' space is on the drive holding D/S. It works better if there is more than 15GB free, even better if there is more than 20GB free. The problem is how in the world that drive keeps filling up in the first place! But, some claim that's because of all the various updates for the OS,  and its parts.

    And yes I too have deleted a few purchased programs from the C drive - installing some of them onto external drives. Bit of a lag but nothing super note worthy.

    D/S is on the C drive and ALL its content is on the D drive. 3D Coat is entirely on an external drive. Carrara has yet to be reinstalled.

    I highly doubt that's it! I have 600GB of free space in my drive.

    I think you're using a Hard Drive?

     

    SATA SSDs are crazy fast for Daz: again, if I take a complex complete scene, delete the figure, and load it, it loads in around just 10 seconds. Hard Drives are much slower than that.
    My problem regards Genesis 8 figures and only those!

    Well that's definitely not all that's involved then. It helps me though. Yes I'm using the internal harddrives on this computer for D/S. One is normal, the other is SSD.

    Yes :)

    A SATA III SSD is more than enough, I guess.

    At this point, it's just a matter of uninstalling useless products and upgrading CPU/RAM.

    ProPose said:

    I don't understand these Loading and Clearing times you guys are getting 

    My G8F loads into a blank scene in about 7 seconds, G8M loads in about the same time. This is on an i7-3820 with 32 GB ram,

    GTX 1060 with 6 GB vram driver ver. 441.87.  No ssd drives, All Sata III

    It depends on the morphs you have installed!

    You probably have not bought many G8 figures/morph packages yet?

    My G8F Morphs folder properties,  That's for the G8F "Base" figure then you can add the various character morphs the I may want to work with

    Capture.JPG
    289 x 133 - 17K
    Post edited by ProPose on
  • ProPoseProPose Posts: 527

    That's probably why yours take to long to load.  I Load just what I need for a particular character

  • TheKDTheKD Posts: 2,691

    Being a genepool mixer, having to look at what I want to use and mix, and install em as I need em would be slower than a figure taking a few mins to load. If I just used stuff as it is instead of mixing people and outfits I would probably do it that way still. That's how I used V4, and how I used genesis in the beginning. It got old though.

  • LenioTGLenioTG Posts: 2,118
    ProPose said:

    That's probably why yours take to long to load.  I Load just what I need for a particular character

    Thank you for your observation, but we've already discussed that multiple times, and it's not a viable option for story creators that need flexibility in characters generation :)

  • LenioTGLenioTG Posts: 2,118
    edited June 2020

    UPDATE: I've transferred most of my Daz Library to an NVME SSD.
    And I have uninstalled 1/4 of my whole Daz Library, to make it lighter. I've probably uninstalled 1/2-1/3 of the figures.
    I've deleted basically all morphs, except for RareStone and Zev0 most important ones.

    Last tests, after the CPU/RAM upgrade, were

    • G8F
      • Loading: 48s vs 64s (-25%)
      • Clearing: 62s vs 79s (-22%)
    • G8M
      • Loading: 25s vs 27s (-7%)
      • Clearing: 17s vs 23s (-26%)

    Let's test again.

    • G8F
      • Loading: 29s, 30s, 29s (-40%)
      • Clearing: 33s, 34s, 34s (-45%)
    • G8M
      • Loading: 15s, 15s (-40%)
      • Clearing: 12s, 12s (-29%)

    My final considerations here: https://www.daz3d.com/forums/discussion/417291/guide-how-to-reduce-figures-loading-time-a-month-long-experience

    Post edited by LenioTG on
  • LenioTGLenioTG Posts: 2,118
    edited June 2020

    FINAL CONSIDERATIONS ON FIGURES LOADING TIMES

     

    Why are figures loading times important?

     

    Your Daz3D library is installed on an SSD, so you load entire apartments in less than 10 seconds, and poses in a fraction of a second, but loading a single G8F takes an eternity? And even deleting the scene makes you want to switch to cave painting? Then you have the same problem I had, when waiting for figures to load, or for a scene to clear, used to take a huge part of my workflow.

     

    What influences loading times?

     

    1. The most important factor, by a long mile, is the number of the morphs you have installed. A G8 figure, indeed, has those morphs loaded all the time, and it has to build the relationship between those morphs, even if they're not currently used, and so each morph slows down both the loading and the clearing times.
      1. Morph Packages: you probably found yet another morph package for $1.99, that you've never used. Uninstall it. Many morph packages have hundreds of morphs, so they're way worse than single characters.
      2. HD morphs: I think they're worse, regarding loading times, than standard morphs.
      3. Characters: don't buy figures that are similar to other ones you already have. Most of the time t's not even worth keeping them installed for hair, dresses or tattoos that come with them.
    2. CPU and RAM. They work together, so I don't know what had the biggest impact. But I have seen a 25% improvement in loading times, with a 50% improvement in CPU performance.
    3. Storage. It's fundamental. Whether to navigate your content faster, to apply poses, or to load either figures or environments/props. The difference between SSD and HDD is huge. There is yet another difference between NVME and SATA SSDs, but it's not as big as the previous one.

     

    Was I able to improve loading times?

     

    In my tests, I've gone from 136 seconds of loading time for the basic Genesis 8 Female figure, down to 29 seconds (-79%).
    I hadn't tested that, but the 136s already were with an SSD, they were surely even slower with an HDD. Probably almost double than that.
    So I'd say I reduced the loading times by around 90%.

     

    What have I changed to get that improvement?

     

    • I have uninstalled almost 2/3 of my figures, and the vast majority of my morph packages.
      I know, it hurts a lot to let products you have bought take dust, but, if you don't desperately need them, having them installed is a huge malus for how Daz works.
      (Some people install the products they need each time they need them. That's not viable for my workflow, because I make comics with Daz, and so I need a dynamic creative process that's not compatible with browsing DIM every time I need to do anything).
    • I have upgraded my PC, from a Ryzen 5 1600 with 16GB of 2400MHz RAM, to an overclocked Ryzen 5 3600 with 32GB of 3800MHz RAM.
    • I have upgraded my storage first from HDD to SATA III SSD, then to an NVME SSD PCIe 3.0. They got really cheap these days, and, since once installed your library you won't touch it much, for this use cheap QLC will do the trick.

     

    This has been a month-long experience, that has taught me a lot, and that has changed my style of purchasing: now, in order for me to buy a morph package/figure, it must be really worth it.
    Here is the discussion with the whole experience, step-by-step: https://www.daz3d.com/forums/discussion/411446/why-does-it-take-so-long-to-load-figures/p1

    I hope my experience will be helpful to you!
    LenioTG.

    Post edited by LenioTG on
Sign In or Register to comment.