New Release: Bringing Daz figures To Life in UE5
dave_0aa47f5a80
Posts: 68
Starting with an FBX character exported from Daz3D, we produce a brand-new, custom set of Unreal compliant JCMs by effectively sampling mesh deformations over a wide range of body positions.
We then rebuild the skeleton from scratch to produce a character optimised for animation in Unreal Engine. This is a completely automated process performed by a Python script run in Maya in conjunction with a custom Daz script. These two scripts represent many months of work!
Demo Movie
Technical Overview Movie
Despite some minor differences in the skeleton structure, the process works seamlessly with all Daz Genesis 8 and 8.1 figures, both male and female, with or without anatomical elements. The system also works perfectly with the higher resolution FBX characters exported by the DazToUnreal plugin.
Once the converted character has been manually imported into the Unreal Editor, applying JCMs using posedrivers, setting up physics, converting standard UE control rigs and other animation tasks all become relatively straightforward. Compelling, realistic mesh deformations across a wide range of body positions that are not possible with the standard Daz JCMs become achievable.
Detail
There are two parts to the conversion process. The first is a Maya Python script that takes the FBX file exported from Daz3D and rebuilds the skeleton for optimal UE compatibility:
~ Converts the joint rotation order to XYZ
~ Reorients all the joints, pointing X down the joint and making Z the primary swing angle
~ Reorients the root node
~ Converts all inline twist bones to leaf twist bones
~ Adds the standard Unreal IK joints (optional)
~ Adds muscle joints for 'jiggle' effects (attach physics bodies to drive morphs, optional)
~ Adds extension joints to finger and toe tips for VR grabbing and precise IK solutions (optional)
The second part of the system is a Daz script that, when used in conjunction with the Maya script, produces a definitive set of JCMs by effectively sampling mesh deformations for a wide range of body positions. The resulting JCMs are optimised for the UE's linear skinning method - Daz uses the dual quaternion skinning method, so this is necessary for good results.
The character mesh is also combined with the clothing meshes during the process, so all exported JCMs (and other morphs) are applied to all meshes at once. This virtually eliminates poke-through issues and makes the application of JCMs at runtime a far more efficient process, especially when used in conjunction with Unreal's posedriver technology.
We also produced a cut-down version of the skin materials optimised to minimise draw calls, enhanced them with micro-normal and micro-roughness maps taken from the Metahuman project and added configurable 'Selective AO' masks for problem areas like nostrils and ears.
An IK retargetting rig, standard backwards / forwards control rig, ground following rig, eye tracking rig and semi-automated method for characterising the figure in MotionBuilder have also been produced.
Background
The system was originally developed so that we could include Daz characters in the VR meeting application we are developing, as we wanted to make the extensive Daz3D catalogue more accessible for avatar creation. It has since turned into a project in its own right. We are considering offering either a Daz figure conversion service or selling the system. There would be some work to get the system to a marketable state, so we thought we'd showcase it to try and gauge the level of interest. We'd very much welcome your comments and feedback!
Daz 3D is part of 
Connect
DAZ Productions, Inc.
7533 S Center View Ct #4664
West Jordan, UT 84084
Licensing Agreement | Terms of Service | Privacy Policy | EULA
© 2024 Daz Productions Inc. All Rights Reserved.
Comments
Interesting, one of the most tedious things about using Daz characters on Unreal Engine is dealing with the skeletons.
After long hours of trial-and-error, I was able to find workarounds for the main issues with the skeletons but it is a tedious, time consuming, process.
Having a conversion tool would be awesome; Maybe Daz 3D could be interested; they could integrate the conversion to their Daz to Unreal bridge; that would be awesome.
I would be reluctant to use a conversion service thought; I use custom models, I desing and morph-in my characters' look, they are part of my copyrighted works.
I don't use "out of the box" Daz characters and I'm not comfortable with uploading my private designs anywhere. Not until I release a complete UE5 game.
I don't know what your upload policy would be but ensuring that custom, private, designs will not be leaked or "shared" would be a requirement for me... and for many.
Just something to consider.
Make it like some Daz studio plugins or independent utilities being sold in Daz store, and it's better to be independent of third party software or bridges. Because users may prefer different 3D software, Maya has the official support from Epic with Livelink, Blender is free and popular with very useful plugins, also users can use different bridges like DazToUnreal or Diffeomorphic or some experimental pipelines. This is awesome and would be the best seller if it's user friendly with documentation and video demonstrations.
Thank you for your feedback!
@VividImaginator:
I hear you on the trial and error comments - it was after spending a good deal of time trying to animate my fingers in VR using Valve Index Knuckles that I decided to look at reorienting the joints. I wasn't planning to spend so much time on the project back then, but it's done now.
I completely understand your concerns about uploading your files for conversion. Whilst we are considering all options to monetise the system, I'm also not sure I'd really want to start offering that as a service either, as now that we have Daz based avatars we can use I'm very keen to get back to developing the open source, decentralised VR meeting app we have had on hold for a while now.
@ VividImaginator, @acatmaster
It would be great to get the system integrated into the DTU bridge, but since most of the heavy lifting is done by the Maya Python script the system is only really available as a tool for Maya users, at least in it's current form. In theory it would be possible to port it to a C++ plugin, but I taking on such a big project at the moment is not really a priority.
I'd be happy to consult on such a project is someone else wanted to do so though.
Bigger Picture
As hardware gets faster and software optimisations like Nanite and Lumens continue to evolve, I see huge potential for Daz3D characters being used as avatars in the near future. I believe the early success of platforms like Second Life are due in no small part to their focus on fostering an ecosystem of user generated content - unlike many of the limited, in-app avatar creation options offered by some of the currently proposed next generation metaverse solutions.
There is already a huge range of Daz characters / apparel available and Daz Studio offers many creative possibilities beyond that. I think it would be great if both regular users and professional avatar creators could use Daz Studio to create avatars and seamlessly upload them to metaverse platforms - this is what we're aiming for with our app and this is the main reason I have developed this conversion system.
a little later to the party but still here, it's really awesome specially if it could be used as a sort of "addon" for the daztounreal plugin and could work without the maya extra step, if your team would talk with david and daz team to see how to make then work together it will be really perfect, we really need the jcms improviment and fix for the rotation/orientation stuffs for daz characters it would be awesome.
@Ellessarr
Thank you for the feedback :-)
I one hundred percent agree with what you say - all the features I've built into the system were born out of necessity for achieving compelling results when animating Daz characters in Unreal.
I'd be very open to talking to David (am guessing Vodhanel?) and the Daz team about a possible collaboration / ways to implement my process outside of Maya.
awesome!!!!!, that is something i would really love to see, a colab would make both have a awesome work and we finally get a perfect daz to unreal transition for the characters, with all jcsm, bones corrections, also maybe awesome physics and mats, making the characters goes to unreal in the most perfect optmized way.
specially without the need of a third part program like maya or blender as it was supposed to be, you just export the character using the plugin and it go direct to unreal with everything working.
Hey Dave
As a mocap specialist and studio that uses Daz characters a lot, this is very, very interesting to us. How can we get in touch?
thanks
@Bryan Steagall - I have sent you a PM via Daz3D forum's messaging.
being curious, the talk with david or daniel are having any adavances??
Thank you to everyone who has provided feedback. Based on the encouraging comments I've received here, on youtube and in private messages / emails, I have decided to start looking at ways in which I can make this available for Daz users.
There are quite a few considerations to work through and discussions to be had, but I intend to get this out there as soon as possible. I'll post updates here as things progress.
I've been working on getting the DTL system ready for release. I've updated the joint renaming scheme to exactly match UE's, made a few improvments to the JCM specfications and am in the process of giving the updated version a thorough testing. The current priority right now is writing documentation and producing tutorial movies. I've made the first tutorial movie. It shows how to convert Unreal Engine 'Quinn' character's animations, animation blueprints and control rig as found in the standard third person template that ships with the UE editor.
The entire process of replacing Unreal Engine's Mannequin with a freshly converted Daz character takes around five minutes:
Retargetting a converted Daz character in UE5
If anyone feels so inclined, I'd really appreciate any feedback in terms of how easy the tutorial is to follow or if there's anything that could be made more clear.
For the next tutorial I'm planning to show how to quickly and easily convert a Body IK rig, complete with forward and reverse solving. Because of the high compatibility of Daz figures that have been converted using DTL, the process of converting the rig takes around a minute. Once set up, this rig can be used to create and modify animations in the Unreal editor. I plan to demonstrate how to convert the rig and how to use the new rig for both animation editing (mocap) and animation creation (keyframe animation) on the converted Daz figure.
I haven't tried it yet, but that tutorial looks great! Thanks for putting it together.
@Senor_Monkey - Thanks for the feedback. I should probably make it clear that the Daz character in the tutorial is a heavilly customised Daz character that has been adapted for animation in Unreal Engine using a system I have not released yet.
Very interesting.. I'm also working on a VR project with Daz Characters, would love to know more about this...
@pasnoc_19191c30f0 - I have sent you a PM
I also need the custom JCMs for Epic skeleton compatible characters, would like to know what skeleton would be used by the plugin, the UE4 manny or the UE5 mannequin skeleton. More detail about the plugin is much appreciated.
In this tutorial, I show how to convert Unreal Engine's standard Full Body IK rig for use with a DTL character. I then show how to create, edit and export animations using the rig.
Creating, editing and exporting animations with Control Rig
Hi @catmaster - The DTL system exports a skeleton with the exact same naming convention as the UE5 Manny and Quinn skeletons. Although these do not have toe joints, there are references to toe joint names in the standard UE full body control rig I used for the above tutorial movie, so I have used those with the aim of future proofing the joint names.
Excellent! I used the smooth modifier in DAZ, then the DazToUnreal Bridge, and used JCMS. I found that smoothing didn't work in UE5, and the export was very slow. I use baking smoothing in DAZ, but it is very slow, exporting JCMS does not output smoothing effect, how can I solve it. thanks
Where can you download and use the DAZTOUE you made?
I'm not aure I can help with your smoothing question as I use Maya for any smoothing operations I do.
Regarding the DTL conversion system, I'm working on getting it released as fast as I can. I'll be posting updates here.
Can't wait! Solves so many problems. You'll sell a million if the price isn't exorbitant.
This is amazing! Looking forward to it.
@delectro_9f6dc82a97 and @faseceroo - thank you!
In preparation for release, I've put together a short movie showcasing a couple of features I'd not shown yet:
A couple of features and capabilities
We're working full speed to get this released!
one question about how this stuff gonna work, you gonna need a third party program like maya?? it will be integrated to the pluging or to daz or to unreal, how things gonna work?
In it's current form the system does most of the conversion using a script in Maya. I can't say too much at the moment, but I can say that various ways to either provide a cloud based conversion service or to create plugins that all users can use are being explored.
i would love a cloud or integration to or direct unreal or in daz or at last in blender, cuz maya is not the best option for indies, due to high prices.
any other way other than only with maya.
Maya has an Indie version at less than $300 per year. It's not heavily advertised and not easy to find on their website. I love Blender and am currenlty using it, but depending on how this DTL system works, I would consider switching back. I've read that Maya integration with Unreal is better than Blender so if Unreal is the future of real-time rendering and game making, Maya might be a better option anyway.
https://makeanything.autodesk.com/maya-indie
@Krys Kryngle - that is very useful information, thank you so much
@Ellessarr - I appreciate it's not an ideal solution, but I'd be interested to hear your thoughts on a $300 Maya license
with the current recession and high valued dollar, still a lot expensive at last here on where i'm living, maya prices still high for many peoples even with the "indie", blender still a much better option or even zbrush price is better than maya, autdodesk prices still a lot too high.
If the UE5 animation features are as good then most of the work can be done inside UE5 and there would be no need for Blender or Maya.
That means using Blender or Maya is needed and better than posing characters with Control Rig, and the Control Rig is more suitable for the IK modifications rather than making animations.