Asset Validation
Tonyhilliam (Talk | contribs) |
|||
| (5 intermediate revisions by 3 users not shown) | |||
| Line 11: | Line 11: | ||
2. That warnings must be fixed. Warnings are generated when the game believes that there may be reason for the content creator to review their asset. It is purely a courtesy to highlight cases which are known to be frequently problematic. This does not necessarily mean that the content is faulty, or that anything needs to be changed. Modifying your own created content to resolve warnings is encouraged, but not mandatory. Modifying content created by others to resolve warnings is strongly discouraged as it is likely to introduce compatibility issues for no benefit. | 2. That warnings must be fixed. Warnings are generated when the game believes that there may be reason for the content creator to review their asset. It is purely a courtesy to highlight cases which are known to be frequently problematic. This does not necessarily mean that the content is faulty, or that anything needs to be changed. Modifying your own created content to resolve warnings is encouraged, but not mandatory. Modifying content created by others to resolve warnings is strongly discouraged as it is likely to introduce compatibility issues for no benefit. | ||
| + | |||
| + | :: Note that it is also a myth that "today's warning is tomorrow's error". In some cases we warn people about errors prior to making the changes mandatory in future versions, but this is a guide not a rule. | ||
| + | |||
| + | == Validation Error Messages == | ||
| + | The list of error messages that can be created during the validation process is at [[:Category:Validation_Errors|Validation Errors]]. | ||
== Recent Validation Changes == | == Recent Validation Changes == | ||
The following changes are reasonably new additions that many users may not yet have experience with. They are included here to highlight the requirements and to advise users that they are legitimate problems. | The following changes are reasonably new additions that many users may not yet have experience with. They are included here to highlight the requirements and to advise users that they are legitimate problems. | ||
| − | * | + | * Content is now loaded by the game engine during validation, in addition to the existing practice of checking the content files for known errors. This means that some number of errors which previously only occurred at runtime will now be detected during validation. |
| − | + | * Limited sanity checks for incorrect use of [[material sharing]] are now present. | |
| − | + | ||
| − | + | == Recent Issues == | |
| − | + | At build 119492 we processed a bug report where items were listing as "Faulty" (eg. in Content Manager) but "View Errors & Warnings" did not reveal any problems, or where the outcome of "View Errors & Warnings" would vary from run to run. (It was also possible that these items could appear in CM as "Installed" and then only show as Faulty after a db repair.) | |
| − | + | ||
| − | + | This was determined to be due to the presence of legitimately faulty mesh files in the asset, which were not referenced in the config.txt file of the asset in question. We found that this configuration caused the mesh files to bypass validation. | |
| − | + | ||
| − | + | In the case where the mesh file is in itself faulty (e.g. has missing textures) then any attempt to reference that mesh from a second asset would cause that second asset to flag as faulty, as expected. That the first asset was not flagged faulty in this scenario is perhaps not ideal, but isn't really a big deal since it never uses the mesh in question. | |
| + | |||
| + | However, there exists a more problematic case, where the mesh is valid when considered individually, but where it is in conflict with other meshes in the same asset. In this scenario, validation of either asset alone could result in the validation passing (since each asset might use one of the meshes, but never both simultaneously) but bulk validation which processed both simultaneously or in a short period of each other, as well as actual in-game use, could reveal problems. | ||
| + | |||
| + | This is typically seen as a material conflict (VE186), or potentially in-game as the mesh having a different appearance depending on the exact load order. | ||
| + | |||
| + | The simple fix for this scenario would be to validate all of the mesh files contained in an asset when validating the asset, whether or not the asset uses those mesh files. | ||
| + | |||
| + | Unfortunately, while this does indeed catch the problem case that we were investigating, it also catches numerous cases where an asset simply contains outright-faulty mesh files. While this is technically a valid outcome, it means that a lot of assets become flagged as faulty due to a mesh file which is never used in practice. This is something that should definitely be cleaned up over time, but not something that we feel justifies breaking a significant amount of content over. | ||
| + | |||
| + | Given the number of assets affected and the low likelihood of anyone actually seeing a problem, we've made this error a warning for assets below trainz-build 5.1. From 5.1 and above, the error will be shown and therefore content creators will be given the necessary information to fix the problem. | ||
| + | |||
| + | We will continue to review this moving forward, with the aim to slowly reduce the prevalence of this fault in the legacy content set. | ||
Latest revision as of 10:06, 7 March 2022
All Trainz assets must be correctly formed in order to work properly across a wide range of software and hardware configurations. To aid in this process, Trainz includes asset validation functionality which checks each item of content for a range of known flaws. This validation process is performed both on your local Trainz installation, and also as part of the Download Station Upload Process. Assets which fail validation cannot be used in-game.
This page documents recent changes to the asset validation process. It is not intended as a complete list of possible failure cases, nor does it seek to provide information on how to fix each error condition. If a fault is discovered in your own content, you should review the relevant Content Configuration documents to understand the rules and restrictions which are relevant to the asset in question.
Contents |
[edit] Common Misunderstandings
There are two common misunderstandings that our users often have regarding asset validation:
1. That built-in content is exempt from validation, and may be faulty. This is incorrect; all built-in content is validated at the N3V Games offices using the same tools as are available to end users. We do not perform validation of built-in content on end-user machines because (1) it is slow, and (2) the content cannot be modified and was known to be good at the time it was made built-in.
- It is important, however, to note that built-in content does not contain source textures. This means that a user who attempts to clone or edit built-in content will be faced with a faulty result unless they provide their own source textures. This does not indicate that the built-in content was faulty, but rather that the user's manipulations have left the game with a faulty copy of the asset.
2. That warnings must be fixed. Warnings are generated when the game believes that there may be reason for the content creator to review their asset. It is purely a courtesy to highlight cases which are known to be frequently problematic. This does not necessarily mean that the content is faulty, or that anything needs to be changed. Modifying your own created content to resolve warnings is encouraged, but not mandatory. Modifying content created by others to resolve warnings is strongly discouraged as it is likely to introduce compatibility issues for no benefit.
- Note that it is also a myth that "today's warning is tomorrow's error". In some cases we warn people about errors prior to making the changes mandatory in future versions, but this is a guide not a rule.
[edit] Validation Error Messages
The list of error messages that can be created during the validation process is at Validation Errors.
[edit] Recent Validation Changes
The following changes are reasonably new additions that many users may not yet have experience with. They are included here to highlight the requirements and to advise users that they are legitimate problems.
- Content is now loaded by the game engine during validation, in addition to the existing practice of checking the content files for known errors. This means that some number of errors which previously only occurred at runtime will now be detected during validation.
- Limited sanity checks for incorrect use of material sharing are now present.
[edit] Recent Issues
At build 119492 we processed a bug report where items were listing as "Faulty" (eg. in Content Manager) but "View Errors & Warnings" did not reveal any problems, or where the outcome of "View Errors & Warnings" would vary from run to run. (It was also possible that these items could appear in CM as "Installed" and then only show as Faulty after a db repair.)
This was determined to be due to the presence of legitimately faulty mesh files in the asset, which were not referenced in the config.txt file of the asset in question. We found that this configuration caused the mesh files to bypass validation.
In the case where the mesh file is in itself faulty (e.g. has missing textures) then any attempt to reference that mesh from a second asset would cause that second asset to flag as faulty, as expected. That the first asset was not flagged faulty in this scenario is perhaps not ideal, but isn't really a big deal since it never uses the mesh in question.
However, there exists a more problematic case, where the mesh is valid when considered individually, but where it is in conflict with other meshes in the same asset. In this scenario, validation of either asset alone could result in the validation passing (since each asset might use one of the meshes, but never both simultaneously) but bulk validation which processed both simultaneously or in a short period of each other, as well as actual in-game use, could reveal problems.
This is typically seen as a material conflict (VE186), or potentially in-game as the mesh having a different appearance depending on the exact load order.
The simple fix for this scenario would be to validate all of the mesh files contained in an asset when validating the asset, whether or not the asset uses those mesh files.
Unfortunately, while this does indeed catch the problem case that we were investigating, it also catches numerous cases where an asset simply contains outright-faulty mesh files. While this is technically a valid outcome, it means that a lot of assets become flagged as faulty due to a mesh file which is never used in practice. This is something that should definitely be cleaned up over time, but not something that we feel justifies breaking a significant amount of content over.
Given the number of assets affected and the low likelihood of anyone actually seeing a problem, we've made this error a warning for assets below trainz-build 5.1. From 5.1 and above, the error will be shown and therefore content creators will be given the necessary information to fix the problem.
We will continue to review this moving forward, with the aim to slowly reduce the prevalence of this fault in the legacy content set.
