Finding all the data/texture files related to a DSON file

starkadhstarkadh Posts: 52
edited November 2022 in New Users

Hi. I use DAZ for my hobbies since the second release. And I did a terrible mistake.
I installed both DAZ official products and also many other freebies took around the internet. Now, when I wanted to clean my library I simply pressed the right button in the library itself and pressed delete.You know... manually, without Install manager (whom, in any case, couldn't see most of the freebies I installed manually in my library, anyway).
But in such way I deleted only the DSON files and now I find myself with DATA and Texture folder of those models and features I actually use all mixed with several Gb of junk file in the same folders.

I would like clean my HDD a bit, but Install manager - that I started to use too late, I admit - can't even see those files. Is there a way to identify, for every model I have, the related sub-folder in Data and Texture folder so that I can distinguish good files from junk and manually delete the latter. I don't care if it will be a long task.

 

Post edited by starkadh on

Comments

  • ChezjuanChezjuan Posts: 518

    There might be a better way, but for products purchased from the Daz store, you can go to the product library page in My Account, then click on the product, then click "view product page" and finally click Readme and scroll down to the file list. That will show you the files and where they live.

    For items installed from elsewhere, you if you have the ZIP files for them, you could always open them and look at the file paths.

  • starkadhstarkadh Posts: 52
    edited November 2022

    There might be a better way, but for products purchased from the Daz store, you can go to the product library page in My Account, then click on the product, then click "view product page" and finally click Readme and scroll down to the file list. That will show you the files and where they live.

    For items installed from elsewhere, you if you have the ZIP files for them, you could always open them and look at the file paths.

    That's the problem. I can surely do it for the product I took here on the DAZ site, but I can't do it anymore for all the rest. And among what I took elsewhere, I have stuff I can't lost. Even personalized stuff I made make on fiverr only for me.
    And I haven't the files ZIP or RAR anymore, since I had to save space in my Hard Disk back in the day.
    Now I'm getting a new larger SSD, but I wouldn't in any case keep all that trash.
    Also because... Jeez, I started with Michael/Victoria 1... we we haven't had time to get used to genesis 8.1 and are already with Genesis 9. With larger and larger files, 4k textures etc... not talking about my personal HDRI, backdrop images, my personal obj models...

    All the space I can gain is welcomed.

    Post edited by starkadh on
  • PerttiAPerttiA Posts: 10,024

    starkadh said:

    All the space I can gain is welcomed.

    Stopped counting at 20TB's and that was some time and many replaced drives ago cheekyblush 

  • TaozTaoz Posts: 9,963
    edited November 2022

    One way might be locating all the complete stuff you want to keep and then copy it over to a new library, leaving the trash behind.   

    If you have incomplete installs of DAZ products which you want to get rid of, you can reinstall them with DIM and then uninstall them again, that should get rid of all the files.  Provided that they were installed on the default path and not modified (folders moved) afterwards. 

    Post edited by Taoz on
  • NorthOf45NorthOf45 Posts: 5,515

    There is a product available that will do exactly what you want: Content Gatherer. It is not new, but will get the job done. If you are meticulous, you can isolate the content needed for each user-facing file (Poser and DS, although some older Poser files do not have fully expressed paths, omitting things like :Runtime:textures:, assuming Poser would know what to do, so they might need some massaging). You can save a project file (a gatherer list) if you need to alter the list, and zip the files into an archive. Not DIM format, though, but you can do that later with another product Content Package Assist. Sounds like fun times ahead...

  • TaozTaoz Posts: 9,963

    NorthOf45 said:

    There is a product available that will do exactly what you want: Content Gatherer

    It seems to be broken by DS updates or at least difficult to get working correctly.  There are several threads about it:

    https://www.daz3d.com/forums/discussion/530941/content-gatherer-not-working-correctly-alternative
    https://www.daz3d.com/forums/discussion/166076/content-gatherer-problem
    https://www.daz3d.com/forums/discussion/600286/content-gatherer-kaput

    I used it myself a while ago and it was missing several files it was supposed to find, but maybe it wasn't configured correctly.

  • NorthOf45NorthOf45 Posts: 5,515

    Works fine for me, on new DSON files and old Poser files. Some old Poser files often had abbreviated pathnames for the texture folder, omitting the :Runtime:textures (in its various syntax), which CG will not find.

    Maybe re-defining the content folders (Mapped Directories) might clear up some of the symptoms.

  • @starkadh, The only way is to keep the downloaded content files either from Daz and somewhere else, is to copy these to an external drive for safekeeping.

    I do that for the free content, the rest DIM can manage it.

    Usually this is the same mistake most do, believing they will save space, so it does not matter if it is big or small, most content should be saved to external hard disk(s).

  • NorthOf45NorthOf45 Posts: 5,515

    Looking at mindsong's investigation, I can see that Content Gatherer gets the list of content directories (Folders button) from the registry from the General Release configuration (Computer\HKEY_CURRENT_USER\SOFTWARE\DAZ\Studio4), and does not use the beta config (Computer\HKEY_CURRENT_USER\SOFTWARE\DAZ\Studio4PublicBuild). I had removed a couple of temporary library paths for the Studio beta, and the list in Content Gatherer still shows the current config from the General Release. So, if you change your library config, do it also with the General Release, or edit the registry.

  • TaozTaoz Posts: 9,963
    edited November 2022

    NorthOf45 said:

    Looking at mindsong's investigation, I can see that Content Gatherer gets the list of content directories (Folders button) from the registry from the General Release configuration (Computer\HKEY_CURRENT_USER\SOFTWARE\DAZ\Studio4), and does not use the beta config (Computer\HKEY_CURRENT_USER\SOFTWARE\DAZ\Studio4PublicBuild). I had removed a couple of temporary library paths for the Studio beta, and the list in Content Gatherer still shows the current config from the General Release. So, if you change your library config, do it also with the General Release, or edit the registry.

    IMO it would be a better idea to let the user select the libraries to scan.  Making your software depend on other peoples work in any context should be avoided as much as possible for there's a constant risk that their next update will break yours, sometimes in ways that are very difficult or even impossible to fix.

    Post edited by Taoz on
  • NorthOf45NorthOf45 Posts: 5,515

    Taoz said:

    NorthOf45 said:

    Looking at mindsong's investigation, I can see that Content Gatherer gets the list of content directories (Folders button) from the registry from the General Release configuration (Computer\HKEY_CURRENT_USER\SOFTWARE\DAZ\Studio4), and does not use the beta config (Computer\HKEY_CURRENT_USER\SOFTWARE\DAZ\Studio4PublicBuild). I had removed a couple of temporary library paths for the Studio beta, and the list in Content Gatherer still shows the current config from the General Release. So, if you change your library config, do it also with the General Release, or edit the registry.

    IMO it would be a better idea to let the user select the libraries to scan.  Making your software depend on other peoples work in any context should be avoided as much as possible for there's a constant risk that their next update will break yours, sometimes in ways that are very difficult or even impossible to fix.

    No one else's work is involved. I'm talking about one user's context. You can select the libraries to scan, but only if they have been configured in the General Release. If you use different libraries for the beta, or any other installation, that were not assigned in the General Release, they will not be available to be selected and scanned. I guess the original design did not anticipate different registry entries. The contents of the libraries should not be any different from one installation to another, just which libraries would be available.

  • TaozTaoz Posts: 9,963
    edited November 2022

    NorthOf45 said:

    Taoz said:

    NorthOf45 said:

    Looking at mindsong's investigation, I can see that Content Gatherer gets the list of content directories (Folders button) from the registry from the General Release configuration (Computer\HKEY_CURRENT_USER\SOFTWARE\DAZ\Studio4), and does not use the beta config (Computer\HKEY_CURRENT_USER\SOFTWARE\DAZ\Studio4PublicBuild). I had removed a couple of temporary library paths for the Studio beta, and the list in Content Gatherer still shows the current config from the General Release. So, if you change your library config, do it also with the General Release, or edit the registry.

    IMO it would be a better idea to let the user select the libraries to scan.  Making your software depend on other peoples work in any context should be avoided as much as possible for there's a constant risk that their next update will break yours, sometimes in ways that are very difficult or even impossible to fix.

    No one else's work is involved. I'm talking about one user's context. You can select the libraries to scan, but only if they have been configured in the General Release.

    That's what I mean - it's dependent on DS settings and registry entries.  In this case it can be fixed it seems, but it does require some knowledge and work, as the threads show.  Selecting the libraries directly is simple and something anyone can figure out. 

    Anyway I'm talking in general, I've had a lot of problems myself with constantly fixing software I've written that depends on other people's work, so I've developed my own rules for how to minimize the problems and make things as user friendly and maintenance free as possible, which already in itself is difficult in a complex context like this.

    Post edited by Taoz on
  • NorthOf45NorthOf45 Posts: 5,515

    Taoz said:

    NorthOf45 said:

    Taoz said:

    NorthOf45 said:

    Looking at mindsong's investigation, I can see that Content Gatherer gets the list of content directories (Folders button) from the registry from the General Release configuration (Computer\HKEY_CURRENT_USER\SOFTWARE\DAZ\Studio4), and does not use the beta config (Computer\HKEY_CURRENT_USER\SOFTWARE\DAZ\Studio4PublicBuild). I had removed a couple of temporary library paths for the Studio beta, and the list in Content Gatherer still shows the current config from the General Release. So, if you change your library config, do it also with the General Release, or edit the registry.

    IMO it would be a better idea to let the user select the libraries to scan.  Making your software depend on other peoples work in any context should be avoided as much as possible for there's a constant risk that their next update will break yours, sometimes in ways that are very difficult or even impossible to fix.

    No one else's work is involved. I'm talking about one user's context. You can select the libraries to scan, but only if they have been configured in the General Release.

    That's what I mean - it's dependent on DS settings and registry entries.  In this case it can be fixed it seems, but it does require some knowledge and work, as the threads show.  Selecting the libraries directly is simple and something anyone can figure out. 

    Anyway I'm talking in general, I've had a lot of problems myself with constantly fixing software I've written that depends on other people's work, so I've developed my own rules for how to minimize the problems and make things as user friendly and maintenance free as possible, which already in itself is difficult in a complex context like this.

    Okay, you are talking about the author of Content Gatherer. Yes, we are dependent on what he has left us, but as long as we can find a workaround, all is not lost. I shudder to think of what we will lose with DS5... 

  • TaozTaoz Posts: 9,963

    NorthOf45 said:

    Okay, you are talking about the author of Content Gatherer. Yes, we are dependent on what he has left us, but as long as we can find a workaround, all is not lost. I shudder to think of what we will lose with DS5... 

    Yes, that's why I'm not going to write anything content related, until DS5 has been released...

     

  • Taoz said:

    NorthOf45 said:

    There is a product available that will do exactly what you want: Content Gatherer

    It seems to be broken by DS updates or at least difficult to get working correctly.  There are several threads about it:

    https://www.daz3d.com/forums/discussion/530941/content-gatherer-not-working-correctly-alternative
    https://www.daz3d.com/forums/discussion/166076/content-gatherer-problem
    https://www.daz3d.com/forums/discussion/600286/content-gatherer-kaput

    I used it myself a while ago and it was missing several files it was supposed to find, but maybe it wasn't configured correctly.

    I really tried. But I didn't manage to use it. I'm sorry. At the end of the day I utilized an...archaic method.
    I made a new library in a new HDD I had to buy, then I renamed the old Data foulder as "Data_".
    So everything I charged in DAZ gave me a message of error with the position of the "Data" files that daz couldn't find.
    Once I found them I copied manually in the new library. Foulder by Foulder.
    Then I went to check in Runtime/textures since a lot of product are named in the same or similar way in Textures folder. And I copied and paste in the new library that too. Folder by folder.

    What I couldn't identify well, I simply downloaded and installed again in the new library.

    Looooong work. More similar to a house cleaning, with similar fatigue.
    But at the end I had a whole HDD with only junk file. I formatted and I brought back the now clean library to the old HDD.
    Result: After an infinite work, The library that was 490 Gb big (there was old morphs and stuff even by the first Victoria and Michael!) now is 100Gb.

  • Seven193Seven193 Posts: 1,083
    edited January 2023

    It's easy to delete pre-V4 stuff by hand.  They store everything in two folders:
    \Runtime\Geometries\
    \Runtime\Libraries\

    Newer generations don't use those folders anymore.

    Now, if there was a tool that told you how many times a particular product was used (or loaded), that would be useful.  But, I don't think Daz Studio keeps track of that kind of information.

    Post edited by Seven193 on
  • TaozTaoz Posts: 9,963
    edited January 2023
     

    Now, if there was a tool that told you how many times a particular product was used (or loaded), that would be useful.  But, I don't think Daz Studio keeps track of that kind of information.

    In Windows a file's Accessed date is supposed to (it's disabled in registry by default) change whenever it is being opened, which theoretically could be used to track how many times a file is being accessed by monitoring the file.

     

    HKLM\SYSTEM\CurrentControlSet\Control\FileSystem


    It's a REG_DWORD value called NtfsDisableLastAccessUpdate that can be set to 0 or 1.

    From the link:

    Determines whether NTFS updates the last-access timestamp on each directory when it lists the directories on an NTFS volume.

    This entry is designed to prevent the NTFS log buffer in physical memory from becoming filled with timestamp update records. If you have an NTFS volume with a very large number of directories (in excess of 70,000), and Windows 2000 does not respond quickly to dir commands, adding this entry to the registry might make directories list faster.

    0 - When listing directories, NTFS updates the last-access timestamp on each directory it detects, and it records each time change in the NTFS log.

    1 - When listing directories, NTFS does not update the last-access timestamp, and it does not record time stamp updates in the NTFS log.

    Post edited by Taoz on
  • Seven193Seven193 Posts: 1,083

    I don't think that can distinguish between files loaded by a mouse click, and files loaded by the computer through other means, such as loading file resources belonging to a scene.  I would only be interested in loading files initiated by a mouse click.

    If one could list all the products they've used over the past 5 years, and see the ones they hardly used, they could say, "hey, I don't use these anymore, so let's save some hard drive space, and get rid of them".

  • PerttiAPerttiA Posts: 10,024

    Seven193 said:

    It's easy to delete pre-V4 stuff by hand.  They store everything in two folders:
    \Runtime\Geometries\
    \Runtime\Libraries\

    Newer generations don't use those folders anymore.

    Careful there... I've seen freebies for G8 place their textures etc. in Geometries 

  • TaozTaoz Posts: 9,963

    PerttiA said:

    Seven193 said:

    It's easy to delete pre-V4 stuff by hand.  They store everything in two folders:
    \Runtime\Geometries\
    \Runtime\Libraries\

    Newer generations don't use those folders anymore.

    Careful there... I've seen freebies for G8 place their textures etc. in Geometries 

    Paraphrasing: "the good thing about standards is that everyone can make their own."  frown

  • TaozTaoz Posts: 9,963
    edited January 2023

    Seven193 said:

    I don't think that can distinguish between files loaded by a mouse click, and files loaded by the computer through other means, such as loading file resources belonging to a scene.  I would only be interested in loading files initiated by a mouse click.

    If one could list all the products they've used over the past 5 years, and see the ones they hardly used, they could say, "hey, I don't use these anymore, so let's save some hard drive space, and get rid of them".

    With the File Accessed attribute enabled it's easy to scan a library to see which files haven't been accessed within a defined time span, so it's doable if you can determine which products the files belong to.  Scanning preset files for known products, or files referenced within them, might work.

    But it does require that that attribute has been enabled all the time within that time span of course.

    Post edited by Taoz on
Sign In or Register to comment.