[Open Source Release] Daz Render Plugin Framework with Yet Another Lux Plugin
danielbui78
Posts: 333
2021-August-16 UPDATE: New alpha version of yaluxplug released: https://github.com/danielbui78/yaluxplug/releases
Back in 2015, I wrote a Daz Studio Renderman compatible Render Plugin for rendering to Luxrender, which I had intended to be released as Open Source from the beginning. However, I decided to not release it because I did not want to diminish from the success of the other Luxrender-based Daz Plugins that were still being actively developed at that time. Since it has been a few years since those other Luxrender Plugins were updated, I think it's time to open up the Daz Developer scene to an alternative GPU based renderer.
Some interesting things that this plugin could do (with proper luxrender configuration and system scripts): Pause / Resume, Batch Render, Movie Render, Network Cluster Render.
Please Note: This source code has not been updated since 2015. I haven't been able to find my compiled binaries for either Mac or Windows, but I do remember that I had successfully built binaries for 64-bit Intel Mac and Windows from these files. You will need both projects to successfully build the yaluxplug.
Corrections: Each project is still separate. yaluxplug is a Daz Studio Render engine that outputs LXS and PLY files and launches luxrender/luxconsole as a child process. It has easy presets configurations for CPU-only, GPU-only and GPU+CPU rendering. ReadyKitRenderer is a working plugin base that will appear in Daz Studio as a new Render engine and is ready for you to fill in with your custom rendering code or hooks to other rendering software.
Project and Source files have been updated for CMAKE + VS 2019. Github Release binary files with debugging symbols are also available for ReadyKitRenderer.
Daz 3D is part of 
Connect
DAZ Productions, Inc.
7533 S Center View Ct #4664
West Jordan, UT 84084
Licensing Agreement | Terms of Service | Privacy Policy | EULA
© 2024 Daz Productions Inc. All Rights Reserved.
Comments
I am certain many will appreciate it
Thank you, @danielbui78, that is incredibly generous of you. I love this community. I'm sure a lot of AMD users will welcome an OpenCL renderer.
I'm not sure if you are aware but the source code for Reality has been made open source and is available online. I had a tentative idea to use the geometry export procedures from the source as it supports geo-graphs and animations but use custom shaders within DAZ to define the materials, use geometry based IRAY lights, camera and environment settings (HDRI, DOF & exposure, etc) to define the rest of the scene information in much the same way Optane does. Then you wouldn't need a seperate materiials interface which is where I feel Reality was most misunderstood. Personally I was always confident with the way the materials/scene interface worked and saw the reason for it given that Reality was around when Studio was still version 3.
That's really awesome about Reality being Open Source. I think I remember reading about that a couple of years ago, but I must have gotten distracted and forgotten about it, because I would have otherwise posted my source code much earlier. Supporting geo-grafts and animations in a Final Rendering Pipeline is actually pretty straight-forward; as long as you know about DzShape::getAssemblyGeometry and DzScene::getFrame. All you have to do is tell Daz Studio to give you the worldspace vertices, normals and UVs and not worry about skeletons, rigging, morphs, vertex order, etc. which IMO is where most of the headaches of 3D rendering are. The only thing that would be easier, is if you asked for fully-lit screenspace coordinates... but then you would probably no longer be doing an unbiased render.
I agree about the materials interface system in Reality. Although it was very powerful and offered a huge amount of pre-made custom settings for all the major Daz characters for its time, it was not easy to use. Since most modern Daz assets now support IRAY / physically based materials, accurate mathematical conversions should be much easier.
I'm trying to wrap up a few things on the Unofficial DazToUnity Bridge, but hopefully I'll have time to update the yaluxplug repository with CMAKE files and build binaries by the end of the week. I really want to get my Daz to Lux Renderfarm back up and running; I have a lot of centaur videos on my todo list. Long-term, I think what I would do is skip yaluxplug (which is mainly a LXS/PLY exporter) and go back to the ReadyKitRenderer to connect it directly to the LuxCore library or maybe AMD ProRender.
The only thing I couldn't get exported correctly was geometry switching. Reality would only export the default geometry, I think Luxcore also supports strand based hair as I have read that exporting strand hair to an OBJ creates a huge number of polygons. Obviously that was relased after Reality development had stopped.
As for the materials, if there was a standard DAZ shader for each Lux material type that had all the paremeters, then that could be preset by Daz Material presets and stored in standard duf files. Maybe that would ease the conversion to Lux materials.
I'd like to see an export option as I use a Linux machine in a cupboard to render animations 24/7. So I wouldn't need my desktop left on. That would usually involve a little bit of editing as Reality would leave some Windows file paths in the exported files. I'd also edit an animation sequence to use a single materials file as Reality would export the same materials for every individual image. Which could use a chunk of diskspace as it was also collating the textures, IES, HDRI and other files.
I've udpated the project files and source code so that it can be built with CMAKE and VS 2019 with Daz SDK 4.5 and Qt 4.8.1 static binaries. However, there appears to be a bug causing DzShape::getAllRenderPrioritizedMaterials(DzMaterialPtrList) to memory fault when exiting the method which called it. Maybe a garbage collection issue? I rewrote that section to use DzShape::getMaterial(index) and it runs... except everything is being rendered inside out. Perhaps this is due to my workaround, or perhaps this is a bug that was introduced right before I stopped working on the project back in 2015. The good news is that all of the Daz Renderer integration and the Luxrender configuration code to start up in GPU only rendering mode are still working. I would post binaries, but I don't know the utility in being able to render inside out objects/people. BTW, the default branch is now "2021-update": https://github.com/danielbui78/yaluxplug
sounds like that transporter on Galaxy Quest
I fixed the bug I mentioned before. Software and Hybrid are working, but OpenCL/GPU rendering appears to be GPU RAM limited more than expected for my GPU (RX580 8GB), ex: genesis 2 female will GPU render when no hair or clothes, but adding either will cause the render to fail. Maybe my drivers or setup are not optimal? Download on github: https://github.com/danielbui78/yaluxplug/releases/tag/v0.1
Redacted
Sorry, the bugfix that I was referring to was the one caused by my workaround for the original DzMaterialPtrList crash bug. I don't have any solution other than using another method to get data for now. That original bug is definitely a very interesting one, though. I am able to use the DzMaterialPtrList and iterate through the entire Materials List and process all the data. It only crashes when I return from the method which called DzShape::getAllRenderPrioritizedMaterials(DzMaterialPtrList). What is even more frustrating is that this function worked 100% back in 2015 on Mac and PC.
Redacted
The problem appears to not be a GPU RAM limit, but a system RAM limit. Specifically during Kernel Compilation. It seems to only be affecting my AMD OpenCL drivers. I installed Intel OpenCL drivers, and the kernel compiler is working fine there with any size scene or complexity. In previous versions of the AMD OpenCL driver from 2015, there would be a pause of up to 60-90 seconds depending on scene complexity. During this time, the system RAM usage would go up and down as much as 4-5 GB as each kernel was compiled in memory and then written to disk/cached. With my current AMD OpenCL drivers, the kernel compilation starts eating up system RAM well passed 5GB, and continues to go up past 40GB. My system starts stalling out at 20GB, as the swap drive starts going crazy and the Windows UI stops updating for several minutes at a time. Since LuxCoreDemo and LuxMark can successfully compile similarly complex scenes, I think it's just some sort of deprecation / incompatibility with how things were kernel compiled in 2015. I'll try to figure out how to convert to the new CFG / SCN format and adapt to any new scene definition guidelines to prevent the AMD OpenCL kernel compiler from going out of control and crashing.
QUESTION: is anyone successfully using AMD-based OpenCL GPU rendering in 2021? I have not been able to do it with any complex scenes using any Lux-based plugin for Daz since 2015.
It may be worth corresponding with Dade on the Lux forums. He did say he would be able to offer some support on updating Reality to luxcorerender
https://forums.luxcorerender.org/viewtopic.php?f=9&t=1467
Although you are not technically doing that, the endgame is the same.
Things are coming along very well. It took me a very painful and frustrating 20+ hours over the weekend to get the basic features back up and running with luxcore 2.5, but the renders are now looking better than ever (for a small subset of models and shaders). I am not focusing on visual quality of the shader/material conversion yet. I'm just trying to make broad strokes to get rudimentary support in for most of the major shaders.
@Reggie68: thanks, I will be sure to contact Dade, especially when I attempt to implement SSS or Dual Spectral Lobe support.
yaluxplug (with luxcore 2.5):
IRay reference:
Here is a render of the LuxCore engine flexing on IRay with reflections. The only modifications from the previous render was changing the "Lighting Model" on the Daz Default Shader from "Plastic" to "Glossy (Plastic)". Glossiness was already at 100%. I also gave IRay a fighting chance by increasing increasing Reflection Strength to 100%. EDIT: these renders were done with the "DAZ Default Shader" material set for the Plane primitive. See comparison below for "Iray Uber" specific material set.
yaluxplug (with luxcore 2.5):
IRay reference:
REMATCH: with Iray Uber material and custom Iray settings. Sorry, I don't remember the exact settings, Glossiness-related properties to 100%, Glossiness color to 100%, Roughness-related properties to 0%. Reflection-related properties to 100%. NOTE: yaluxplug is currently just treating "Iray Uber" shader materials as unknown materials, and using default settings. Using a default glossy2 luxcore material for it and then setting the roughness (1-glossiness) value to it. All reflection-related luxcore material settings are default.
yaluxplug:
IRay:
Redacted
This is 100% 1-to-1 comparison. All materials and settings were exactly the same, aka default materials and shaders from Daz Studio: I just switched render engines and clicked render. As stated above:
Steps to reproduce:
EDIT: I added a second comparison test with "Iray Uber" shader instead of Daz Default Shader in the post above. Iray does well there.
Redacted
hehe, actually, based on that information, my first render would be the best comparison since niether LuxCore or Iray have native support for DAZ Studio Default shader. The second render gives Iray an unfair advantage because Iray Uber shaders are designed to work best with the internal settings of Iray. None of those shaders (DAZ Studio Default and Iray Uber) have any native / internal level support for LuxCore, and I have not put any extra coding to tweak visual quality when using those shaders other than to map the most common settings (diffuse maps, specular maps, roughness, glossiness) onto the LuxCore material settings.
Redacted
Iray is the default, and has been since it was added as far as I know.
Rudimentary LuxCore support is now in place for Daz Studio Default Shaders, omUberSurface shaders, Iray Uber shaders and PBRSkin shaders. I think this should cover most assets from Genesis 2 to Genesis 8. When I say rudimentary, I mean diffuse maps, specular maps and bump maps should be loaded and displayed for most things. Normal maps and Translucency maps are implemented but not activated in the current release because of visual bugs that need to be worked out. Dual Specular Lobe is half implemented (only first lobe is loaded and rendered), I will try to get around to fixing bugs and implementing the rest maybe next week. GPU Rendering Performance is currently NOT impressive... I need to implement a materials lookup table to remove redundant materials. Maybe this will improve performance, because there are a LOT of redundant textures and materials in the scene file.
If you have been waiting patiently for a binary preview release, you can go download the latest working alpha at: https://github.com/danielbui78/yaluxplug/releases/tag/v0.2
Requirements
Installation and Setup
-Download yaluxplug-v02.7z and unzip it into your DazStudio\plugins folder.
-Download and Install LuxCoreRender v2.5 Standalone and then the LuxCore API SDK v 2.5.
-Copy the luxcoreconsole and luxcore.dll from the SDK into the LuxCoreRender standalone package. You can delete the rest of the SDK folder and files.
-Run Daz Studio and go to the Render Settings Pane. Click Advanced tab and select Lux executable drop-down. Then choose the folder where you have the luxcoreconsole.exe file.
-Select the Rendering device: CPU, CPU+GPU, GPU only.
-Set the maximum texture size to use.
(The additional "yaluxplug-v02-debug-pdb.7z" file contains debugging information for use with Visual Studio 2019. It is not needed for standard use of plugin.)
Instructions
-Load your scene.
-Luxcore is compatible with IRay's Tonemapper Options and Environment Options nodes.
-The property settings in the Tonemapper Options node will over-ride what is in the yaluxplug Advanced Render Settings tab.
-Recommend setting IBL (image based lighting) aka HDRI lightsource in the Environment Options.
-Select yaluxplug in the Render Settings.
-Set up the "halt" conditions to complete rendering in the Advanced tab.
-Click Render.
-A Render Preview window will open and function similar to the IRay Render Preview window. Clicking Cancel in this window will stop the render, but you can still preview and save the image.
-A Log Window will open similar to Luxus, which contains the console output from luxcoreconsole.exe.
-Wait for the halt conditions to complete, or click "Cancel" in the Render Preview window to stop the rendering.
-If you click Cancel in the small Daz rendering dialog box, it will close the preview window. There is also a similar Stop button on the Log window.
Known Issues
-Many Settings in the Advanced Render Settings tab are from the old Luxrender build and not working with LuxCore.
-Only rudimentary Diffuse, Specular and Bump map support are implemented.
-Normal maps, Dual Lobe Specular, Translucency, Subsurface Scattering are not yet implemented.
-Some surfaces/submeshes are not properly hidden for geoshells and some geografts.
-GPU Performance is not yet optimized.
FYI, normal maps, bump maps, metallic maps, dual lobe specular, translucency, subsurface scattering, PBR skin shader support have all been implemented... I have been mainly working on automatic material conversion algorithms, tone-mapping calibrations and accurate visual reproduction of Iray effects on the LuxCoreRender engine. I'll try to post more renderings when I have time.
UPDATE: New alpha version of yaluxplug released: https://github.com/danielbui78/yaluxplug/releases
yaluxplug: Yet Another Lux(Core) Plugin for Daz Studio
New in this version
-SSS/Translucency support
-Better Dual Lobe Specular support
-Better PBRSkin and IrayUber emulation
-Optional Iray Render behaviors (optional addition of Tonemapper and Environment nodes)
Requirements
or
Installation and Setup
-Download yaluxplug-v03.7z and unzip it into your DazStudio\plugins folder.
-Follow directions on the LuxCoreRender v2.5-Experimental-1 release page to download and install the custom LuxCoreRender for yaluxplug: https://github.com/danielbui78/LuxCore/releases/tag/v2.5-experimental-1
-....Or Download and Install LuxCoreRender v2.5 Standalone and then the LuxCore API SDK v 2.5. Copy the luxcoreconsole and luxcore.dll from the SDK into the LuxCoreRender standalone package. You can delete the rest of the SDK folder and files.
-Run Daz Studio and go to the Render Settings Pane. Click Advanced tab and select Lux executable drop-down. Then choose the folder where you have the luxcoreconsole.exe file.
-Select the Rendering device: Software (CPU only), Hybrid (CPU+GPU), or GPU only. Recommend Hybrid mode for best performance.
-Set the maximum texture size to use.
Instructions
-Load your scene.
-Luxcore is compatible with IRay's Tonemapper Options and Environment Options nodes.
-The property settings in the Tonemapper Options node will over-ride what is in the yaluxplug Advanced Render Settings tab.
-Recommend setting IBL (image based lighting) aka HDRI lightsource in the Environment Options.
-Select yaluxplug in the Render Settings.
-Set up the "halt" conditions to complete rendering in the Advanced tab.
-Click Render.
-A Render Preview window will open and function similar to the IRay Render Preview window. Clicking Cancel in this window will stop the render, but you can still preview and save the image.
-A Log Window will open similar to Luxus, which contains the console output from luxcoreconsole.exe.
-Wait for the halt conditions to complete, or click "Cancel" in the Render Preview window to stop the rendering.
-There is also a similar Stop button on the Log window.
Known Issues
-AMD OpenCL drivers are known to have many bugs. These may cause black textures, blocky textures, or other visual artifacts. Recommend disabling Normal maps or enabling "Normal as Bump". May need to disable Bump and Normal maps to avoid all artifacts.
-Due to OpenCL driver issues, AMD GPU may be slower than CPU rendering. Hybrid mode may have best performance on AMD systems.
-Several OVERRIDE options are available in Advanced Render Settings to use custom render pathways for compatibility with some Genesis 8 and older models. Try OVERRIDE Saturation for skin that renders without SSS or are too pale. Try OVERRIDE Translucency/Transmission for skin that renders with unnatural colors (too orange, green, blue, etc).
-DrawGround may need to be disabled in Environment node if feet are partially missing or clipped off in renders.
Hi.
Nice project !!!
I have a problem with the spot light. Not working. Only the distance light.
Thank You.
Thanks for the feedback! I just confirmed what you are seeing -- there are definitely bugs in the export of spotlights and point lights to LuxCore. The transformation matrix is incorrectly scaled and I think the light power/gain is incorrect as well. This bug actually seems to be an original bug carried over from my Luxrender plugin, or maybe Luxrender and LuxCore just handle light positioning differently. I will post an update here when it is fixed, but it may take me some time.
Thank You for the fast response !!!
Other way everything working fine with my radeon rx 6700 xt
Thanks !
Version 0.4.1 with the spot light and point light hotfixes is out: https://github.com/danielbui78/yaluxplug/releases/tag/v0.4.1
Also, there is mac support.
Reality stopped working for me a few days ago, was working, now when I go to launch nothing happens, no window, no error, nada, anyone else experiencing this?
I am wondering if Paolo has stopped maintaining the authentification server or something, emailed him but no reply. So I am considering buying a new rig and switching to iray but the money I have to spend to squeeze the performance out of iray that I currently get on my old system is quite insane, so I am thinking about trying your plugin. One question atm: does yaluxplug allow for export only so I can close everything down, reboot and launch just luxrender?
Yes. This happened to me once and it turned out to be related to installing a product (AMD APP SDK) which upgraded the default Windows OpenCL.dll driver from version 2.x to version 3.x. Go into your \Winddows\System32\ folder and find the OpenCL.dll. Right-click it and select "Properties". A File Properties dialog should pop up, select the Details tab. If the File version is not "2.2.1.0" then it was probably "upgraded" by another OpenCL package. Unfortunately, uninstalling the AMD APP SDK did not fix this for me. I had to dig up the original file out of my hard-drive backups. I then put it into a special folder labeled "OpenCL 2.0 driver" and also copied it into the Reality_DS.exe folder to hopefully make sure it never gets "upgraded" again.
yaluxplug does not currently let you specify an "export" folder, but you can find the exported render files in your Daz Studio temp folder. For me, this folder is in: "\Users\[USERNAME]\AppData\Roaming\DAZ 3D\Studio4\temp". Exported scene files are named "dXXXX". Meshes are exported as binary PLY files in the same folder. Exported images are placed in a subfolder named "yaluxCache". You'll have to copy the desired scene and PLY files out before exiting Daz Studio, otherwise they will get deleted.
Note that in the updated versions of yaluxplug, I hard-coded a separate pathway for LuxCoreRender output and have not re-enabled the LuxRender output pathway yet. I think yaluxplug v0.1 Alpha may still have the original LuxRender executable support: https://github.com/danielbui78/yaluxplug/releases/tag/v0.1 However, there were several bug fixes and to the render system since that version, including fixes for geografts and geoshells. So you may want to try with both versions and luxrender and luxcorerender setups to see which works best for you. Also, yaluxplug's design goal was to be a user-friendly plug-in replacement for Iray and not Reality. So most of the Reality custom lights and surface/texture settings will not work with yaluxplug.
I just confirmed that my Reality version is still working fine with the latest Windows 10 update and Daz Studio 4.15 Pro and the original OpenCL version 2.0 driver. Hope you get your installation working again. Good luck!