DNM effiecency

Learn to make aircraft, maps and more with all of the tools. All the tutorials you will ever need are located here!
NOTE THAT HACKING THE MAIN YSFLIGHT .EXE IS FORBIDDEN
Post Reply
User avatar
Neocon
First Class Membership
First Class Membership
Posts: 3840
Joined: Wed Oct 19, 2011 5:01 am
Favorite Aircraft: Baron 58
Location: Tennessee
OS: Win 10
Has thanked: 1608 times
Been thanked: 1727 times
Contact:

DNM effiecency

Post by Neocon » Tue Oct 13, 2015 2:36 am

I'm posting this as a person who doesn't know a lot about how Blender works and a modder who uses Gepoly (x and o), DNMViewer, and Notepad to make and work on models. I've noticed a lot of inefficiencies in DNM models lately. Among them are duplicate surf files (PCK entries), overly-complicated text, unused null entries in the parts tree, and misnamed surf files.

I'll start with the duplicate PCK entries. An example would be the wheels. Let's say you have a plane with three wheels that are identical. There is no reason to have duplicate surf file PCK entries for each wheel. You can have a single PCK wheel.srf... shared among several entries in the parts tree, where you have SRF "wheel_main_l", SRF "wheel_main_r", , and SRF "wheel_nose"; and use the coordinate system to rotate them as needed. I don't know if that is possible in Blender or if Blender automatically makes a whole new PCK entry for each duplicated wheel. If you can share the same PCK entry for multiple SRF entries in the parts tree, that would save quite a bit on file size.

Next is the issue of overly-comlicated text. One of the biggest offenders I've seen is the letter "O". Not only are they overly complicated in the number of verticies, but it seems they are being turned out as a series of triangle faces. When I make a letter, I generally try to to use more than three or four faces. For example, with the letter "C" I might use only two or three faces, so for the letter "O" I would only use four faces. The letter "T" would only have two faces, although I've gotten away with only one face before. Triangluarized "O"s seem to reflect light oddly sometimes, on top of the file size issue. Each face takes up several lines of text in the .dnm file, and an "O" with 100 faces is going to take up a significant amount of space, which in turn increases the file size and reduces performance in YSF. If you can, it is probably better to simplify the letters, which can most likely be done without reducing the quality of the letters. I've noticed that the word "'Ohana" used on the 'Ohana planes takes up as many kb's of memory as the rest of the plane combined. Add on top of that it is used twice, so it has a duplicate PCK entry. Removing the duplicate and pointing both SRF entries in the parts tree alone reduced the file size of the 737 by 1/3. Simplifying the remaining text reduced it again, resulting in a final file size of half of its original size, which should improve frame rates and decrease lag online.

I've noticed a few .dnm files that have a bunch of SRF "nullxxx.srf" entries in their file trees. What is strange is that, A. they didn't have child SRF's attached, so they obviously had no purpose of existing in the first place, and B. they had coordinates placing them several kilometers from the center of the object. They were easy to clean up. My reason for mentioning them is that just cleaning up loose ends like those reduced the file size of the .dnm significantly (about 10-15%!), and only took a couple of minutes. Please clean up your work before releasing it. Silly things like that can make a big difference both in download size and performance in YSF.

Finally, I've noticed that in some models there will be a PCK entry where the surf file has a name like "mesh.001". Somehow, both YSF and DNMViewer seem to comprehend that "mesh.001" is a .srf file, even though it lacks the ".srf" in its name. This has a negative impact on people like me who modify planes by using dnmexp.pl and then importing them into gepolyx. Gepolyx will only import .srf files, and Windows reads them as "mesh.001" with no file type. I then have to rename it to "mesh.001.srf" so it will be recognized by gepoly. When I get ready to return it to the .dnm, I either have to change the name of the surf in the parts tree to have the ".srf" at the end, or I have to remove the ".srf" from the file name after I paste it in. It only takes an extra few seconds, so I live with it. What concerns me is that a future update to YSF may stop recognizing surf files without the ".srf" file type in its name in the PCK entry of the .dnm.


In conclusion, I hope those of you who make or modify models for YSF will take steps to make your files more efficient, which will result in a better experience for everyone using them.

User avatar
u2fly
Senior Veteran
Senior Veteran
Posts: 1248
Joined: Fri Oct 02, 2015 9:27 am
Favorite Aircraft: Antonov An-225 Mriya
Location: Ukraine
OS: Linux
Has thanked: 745 times
Been thanked: 276 times
Contact:

Re: DNM effiecency

Post by u2fly » Tue Oct 13, 2015 4:54 am

Thanks, interesting.
Now i try collect all info about YS formats in one nice book, so will add this info too.
Here is PDF with table of content
/!\ READ YSFLIGHT HANDBOOK (online + PDF) | updated 2019/04/25

U.S. Senator John McCain wrote:
“Spending #NewYearsEve w/ brave #Ukrainian Marines at a forward combat outpost - we stand w/ them in their fight against #Putin's aggression”
http://pic.twitter.com/vkz0gdozVV
(31 DEC 2016)

User avatar
Dragon_Mech
Veteran
Veteran
Posts: 571
Joined: Wed Jul 15, 2015 12:18 pm
Favorite Aircraft: F-14, F-15D&E, SR-71, King Air 200, B737
Location: Mansfield, Mo. USA
OS: Windows 7 Ultimate
Has thanked: 94 times
Been thanked: 127 times

Re: DNM effiecency

Post by Dragon_Mech » Tue Oct 13, 2015 7:23 am

This is some great info Patrick. i'm just getting started in modifying aircraft and i don't want to release bloated mods. :)
Owner, Dragonic Machine Industries
Image

NightRaven
Senior Veteran
Senior Veteran
Posts: 1843
Joined: Tue Feb 07, 2012 10:00 am
Favorite Aircraft: .
OS: windoze 10 64-bit
Has thanked: 538 times
Been thanked: 608 times

Re: DNM effiecency

Post by NightRaven » Tue Oct 13, 2015 7:47 am

1) Alt-D in Blender instead of Shift-D. This duplicates only the mesh (positions of vertices, faces, colour) but gives it different object data, allowing 2 objects to be exactly the same with different positions and/or orientations without significantly increasing file size.

2) Decimate modifier in Blender, or you can set DefRes to a lower value before converting the text into a mesh. This reduces the vertex count significantly for text. You can also switch out the default font for another less complex font. I notice that many mods made by less experienced people have this issue.

3) Completely dependant on modder competence.

4) See my "Some tips" thread.
Also this causes D3D to mess up and crash on less capable computers. This issue is endemic in HQ mods, even in decaff's aircraft.

User avatar
Flake
Core Staff
Core Staff
Posts: 4350
Joined: Thu Feb 10, 2011 8:47 pm
Favorite Aircraft: Boeing F/A-18A
Location: Australia
OS: Windows 10 (x64)
Has thanked: 835 times
Been thanked: 1238 times
Contact:

Re: DNM effiecency

Post by Flake » Tue Oct 13, 2015 8:05 am

Patrick31337 wrote:Next is the issue of overly-comlicated text. One of the biggest offenders I've seen is the letter "O". Not only are they overly complicated in the number of verticies, but it seems they are being turned out as a series of triangle faces. When I make a letter, I generally try to to use more than three or four faces. For example, with the letter "C" I might use only two or three faces, so for the letter "O" I would only use four faces.
Regarding this you do save a few lines in the SRF file, but YSF Does have to recalulate the convexnised poly.

4-5 vertecies per poly is about the max you can get away with before it starts to use very slow calculation methods. In fact this is what causes the confusion and peep hole effects between DX and GL as they render differently.

3 vertex polys can be instantly transformed to the OpenGL / DirectX pipeline. (The 3d engines of graphics cards work in triangles only.

That said, I am a huge fan of using 4 vertex polys (Quadrilaterals) as they make repainting / modding, especially in GePolyX much easier.

The solution is what NR said: Simply your models. The letter O doesn't need 500 faces in it.

I use a tolerance of ~5-20 degrees for corner smoothness myself, depending on the importance of the part.

For example: 90 degree smoothness for a wheel results in square tyres and is not acceptable. I say 10 degrees is acceptable though. (36 sub-divisions). The smoother you try and make a part or decal, the more polys you need. The trick is not necessarily to sacrifice quality, but to re-image what you plan to do to work within the constraints.
I am an accountant working full-time (and some). I'm not here as often as I would like to be. Send a message if you need me. There are a few people in the community who can get in contact with me urgently if you need - don't be afraid to ask. I just don't check here as frequently as I used to. Sorry!

User avatar
decaff_42
Staff
Staff
Posts: 3545
Joined: Sat Oct 29, 2011 7:23 pm
Favorite Aircraft: SR-71 Blackbird
Location: Eastern United States
OS: Mac OSX
Has thanked: 617 times
Been thanked: 1326 times
Contact:

Re: DNM effiecency

Post by decaff_42 » Tue Oct 13, 2015 10:51 pm

This is an awesome collection of knowledge which I think we can expand on even further. DNM efficiency is something that I think we all desire, and I would love to do some experiments to see how much we can save in DNM file size.

I'd like to address some things that have been brought up and add my own penny to the conversation.

(1) Text in YSFlight
As Nightraven has mentioned, reducing the vertex count of text before conversion to a mesh in Blender is a MUST. You can easily see the impact with some quick data from Blender
Screen shot 2015-10-13 at 6.12.02 PM.png
Screen shot 2015-10-13 at 6.12.02 PM.png (23.41 KiB) Viewed 1842 times
As you can see, compared to the default DefRes = 12, we can save 80% in faces by going thru and doing some quad-making (ctrl -> j ) and cleaning up the mess that makes, you can save upwards of 45% on the baseline text of a particular DefRes. Since this is ONLY for the default Text word, you can expect higher savings with words that have more curved portions, perhaps as much as 90-95% when you are done working on it. In terms of just faces, the DefRes=4 was 76% less than the default, and the DefRes = 3 was 80% less. As you can see below the detail difference is not extreme between the last two, but perhaps the best balance between detail and performance is the DefRes = 4.
Screen shot 2015-10-13 at 6.05.16 PM.png
If there are lots of duplicate features there could be HUGE file size savings.
Here's some data:

For a given vertex, you save one line in the DNM file straight out. This will save you between 24 and 27 characters, each takes up x number of bytes a piece.

Code: Select all

V -0.0569 -0.0111 -0.1021 R
Removing one face will save you 5 lines in the DNM file and will save you more than 70 characters.

Code: Select all

F
C 210 186 148
N 0.0154 0.0342 0.1058 0.4774 0.0585 0.8767
V 4 42 68
E


If we look at the number of lines saved, it is really easy to see how this will add up.
DefRes12 = 839 Lines
DefRes4Quad = 358 Lines
DefRes4Decaff = 283 Lines
DefRes3Decaff = 241 Lines

In terms of lines and number of characters, there was a 70% savings in the DefRes3Decaff vs the DefRes12 default text.


(2) Duplicate SRFs
As Nightraven mentioned, the Alt-D duplication does reduce the vertex count significantly as the linked objects will have the same SRF name in the DNM file, but refer to a common mesh. This has the added benefit in Blender of creating a link between the two so if you edit one (paint, modify, etc) you edit them all. This is how I was able to make the nozzle for the F-20 work well, as the animations were carried over too! In the DNM file, all the 12 petals refer to a single mesh listed in the SRF section of the DNM file. You can see this in the CLA definition stage with FIL line under the SRF line.

Code: Select all

SRF "petal_1_panel.029"
FIL Mesh.500
…
SRF "petal_1_panel.028"
FIL Mesh.500
I am unsure if duplicated items like this could be animated differently (like with landing gear), but I will take a look.

(3) PCK Mesh.### Entries
I took a look thru the DNM of the F-20 that I exported from Blender, and every SRF in the SRF section was a mesh.###. As I mention above, these are assigned to SRFs in the CLA definition section of the Blender-generated DNM file that I have in front of me.


(4) Null SRFs
I am not sure where these are coming from, however when I did a search thru my F-20 DNM, I did not find any.



I shall do some experimentation to see if I can reduce the dnm size of the F-20 models I am working on by taking further advantage of the Alt-D duplication and report back on results when I have them.


EDIT: I am able to confirm that Alt-D duplicated objects can be animated differently. However, since they are duplicated, you loose the ability to clear object rotations via the Apply Scale/Rotation to Object Data feature (Ctrl->A-1)

If you have already duplicated without linking, you can go back and link them by selecting both objects and pressing Ctrl->L and select Mesh Data. I did this to the F-20C and got a 2% file-size savings. There is more simlarity in the F-20D, so I think That will save even more!

EDIT 2:
F-20D saved 20% on the DNM file size. Thanks to aggressive duplication in the cockpit, The D model is at 2.4mb where as the C model is at 2.2mb after the linking of duplicated objects. This is awesome news. I will have to re-work a few things with the models I have already painted, but this kind of savings are golden! :D
Check out my website YSDecaff for all my tutorials and addon releases.
Visit by Bitbucket page for Python codes: https://bitbucket.org/decaff42/

Civilian Aviation Server Flight Log

User avatar
Neocon
First Class Membership
First Class Membership
Posts: 3840
Joined: Wed Oct 19, 2011 5:01 am
Favorite Aircraft: Baron 58
Location: Tennessee
OS: Win 10
Has thanked: 1608 times
Been thanked: 1727 times
Contact:

Re: DNM effiecency

Post by Neocon » Wed Oct 14, 2015 12:21 am

Something that can also reduce the file size by a small amount is converting the color codes from rgb to YS. Again, it is a small amount. 573 kb to 554 kb small. :lol: Of course, the more faces you're dealing with the bigger the difference it will make.

NightRaven
Senior Veteran
Senior Veteran
Posts: 1843
Joined: Tue Feb 07, 2012 10:00 am
Favorite Aircraft: .
OS: windoze 10 64-bit
Has thanked: 538 times
Been thanked: 608 times

Re: DNM effiecency

Post by NightRaven » Wed Oct 14, 2015 12:32 am

decaff_42 wrote:(3) PCK Mesh.### Entries
I took a look thru the DNM of the F-20 that I exported from Blender, and every SRF in the SRF section was a mesh.###. As I mention above, these are assigned to SRFs in the CLA definition section of the Blender-generated DNM file that I have in front of me.
While this does not normally affect performance per se, it does, like I have said before, cause D3D YS to crash on less capable computers. So I would say renaming the mesh with the .srf extension is good practice.

User avatar
Neocon
First Class Membership
First Class Membership
Posts: 3840
Joined: Wed Oct 19, 2011 5:01 am
Favorite Aircraft: Baron 58
Location: Tennessee
OS: Win 10
Has thanked: 1608 times
Been thanked: 1727 times
Contact:

Re: DNM effiecency

Post by Neocon » Wed Oct 14, 2015 1:56 am

Would it be possible to make a pearl script that can automatically recognize a surf name without .srf and add the .srf to the end? I've seen some lwl.srf.002-type names, so the result would be lwl.srf.002.srf, which would look funny, but would still work.

User avatar
Flake
Core Staff
Core Staff
Posts: 4350
Joined: Thu Feb 10, 2011 8:47 pm
Favorite Aircraft: Boeing F/A-18A
Location: Australia
OS: Windows 10 (x64)
Has thanked: 835 times
Been thanked: 1238 times
Contact:

Re: DNM effiecency

Post by Flake » Wed Oct 14, 2015 6:13 am

Patrick31337 wrote:Something that can also reduce the file size by a small amount is converting the color codes from rgb to YS. Again, it is a small amount. 573 kb to 554 kb small.
This one really is a matter of personal preference. Personally I like having full ARGB color palette but some addons don't even need it.

back end wise, that does need to be converted back to hex to be used by DX and GL though (like, 1/1000th of a second?) but yeah, there's that. It's a bitwise operation though so it's very fast.

Also, regarding decaff's post...

Decaff has my seal of approval in regards to YSF knowledge - I'm the guy to speak to re: netcode and things. Decaff is the guy to speak to regarding most other things. Decaff, you are a tank. I love your work.
I am an accountant working full-time (and some). I'm not here as often as I would like to be. Send a message if you need me. There are a few people in the community who can get in contact with me urgently if you need - don't be afraid to ask. I just don't check here as frequently as I used to. Sorry!

Post Reply

Who is online

Users browsing this forum: No registered users and 0 guests